Settings and Parameter Store

Last Updated: March 12, 2025

This document will help you configure settings and securely manage sensitive information across environments. vets-api uses configuration file(s) to manage environment-specific and secret values. Developers can reference these values via the Settings object that is created by the config gem. Environment-specific and secret values are stored in Parameter Store Parameters.

Background

Config files

config/settings.yml
- Used in all deployed environments including Preview Environments
config/settings/test.yml
- Used in the testing environment (local and CI)
config/settings/development.yml
- Used for local development

settings.local.yml

To change a setting on just your machine, use config/settings.local.yml. This file is rarely used and not committed to the repo. It’s intended to be used a machine-specific configuration file. You can use this config file to turn on clamav or specify your betamock cache directory for local development on your machine. You should not add any Settings to this file.

The file settings.local.yml will not be loaded in tests to prevent local configuration from causing flaky or non-deterministic tests. Environment-specific files (e.g. config/settings/test.local.yml) will still be loaded to allow test-specific credentials.

Environment-specific and secret values

A setting is a key/value pair within the Settings object that is derived from the config files. Usually, the value either differs between environments or is secret and can’t be committed the repo.

Secrets can be IP addresses, passwords, API keys, tokens, etc.
Environment-specific values could be a hostnames, environment names, urls
A value can also be secret and environment-specific such as for Amazon Web Services (AWS) credentials

Values that are not secret and are the same in all environments including local development and testing should be constants and not added to a config file or Parameter Store.

Settings convention

Settings are organized by module or team
- Team organization is used for Settings not in a specific module and not owned by the Platform.
- Existing Settings that don’t conform to this convention will be addressed at a later date by the Platform in Phase 2 of the transition plan.
Environment variables are named by converting the Setting in dot notation
- . are converted to __ (double underscore)
  - Settings.module.service.key => ENV['module__service__key']
- The name is used to infer which Parameter value to use.
All Settings must be present in settings.yml to exist
Use ENVs in config files if the value is either:
- Secret (e.g., aws_secret_access_key)
- Deployed environment-specific (email template id)
Use hardcoded values in the config files if the value is both:
- Non-secret
- The same for all deployed environment
- Different between local and deployed environments

ENV keys in config/settings.yml MUST be all lower case.

Parameters

The Parameter Store, Param Store for short, is a tool within AWS Systems Manager for configuration data management and secrets management. Parameters are used to store that information. Parameters have four components a name, type, value, and version. Name is also called path. The type is always SecureString. The value is always stored in the Parameter as a string.

The path has four components:

/dsva-vagov/vets-api/
the environment (dev, staging, sandbox, and prod)
the setting type (i.e., env_vars)
the setting keys converting . to /

For more information on requesting access to AWS console please review Request access to tools on the Platform Documentation

Every setting using the Param Store MUST have a Parameter for each deployed environment regardless if it’s being used or not. This is to ensure predictability and consistency between environments.

Add a Setting

Step 1: Determine the key name

Example:Setting.module.service.api_key with the value of key_abc123

Since the api_key is a secret, the value in settings.yml will be an environment variable

Step 2: Add the key/value alphabetically to all config files

YAML

# config/settings.yml
---
common:
  vsp_environment: <%= ENV['common__vsp_environment'] %>
module:
  service:
    api_key: <%= ENV['module__service__api_key'] %>
travel_pay:
  sts:
    account_id: <%= ENV['travel_pay__sts__account_id'] %> 

# config/settings/development.yml
---
common:
  vsp_environment: localhost
module:
  service:
    api_key: fake_api_key
travel_pay:
  sts:
    account_id: fake_account_id

# config/settings/test.yml
---
common:
  hostname: www.example.com
module:
  service:
    api_key: fake_api_key
travel_pay:
  sts:
    account_id: fake_account_id

Step 3: (Optional) Add the value to the Param Store

If you’re using an environment variable for your settings, you’ll need to add the value to the Param Store as Parameters. First, determine the paths for each environment, using the example Setting.module.service.api_key, the following paths are required:

/dsva-vagov/vets-api/dev/env_vars/module/service/api_key
/dsva-vagov/vets-api/staging/env_vars/module/service/api_key
/dsva-vagov/vets-api/sandbox/env_vars/module/service/api_key
/dsva-vagov/vets-api/prod/env_vars/module/service/api_key

Note how the dot notation is converted to slashes. It MUST match exactly in order to generate the correct environment variable specified in the settings.yml file.

Then using AWS CLI or AWS Console create a Parameter for each deployed environment

CODE

aws ssm put-parameter --name /dsva-vagov/vets-api/dev/env_vars/service/api_key --value key_abc123 --type SecureString --overwrite

For more information on using AWS Console, please review the AWS User Guides on Parameter Store

Deploying changes in Parameters

In order to apply any changes (including additions, updates, rollbacks), the vets-api deploy must be restarted. Parameter Store updates will not trigger the deploy to restart and pods to be replaced. Any changes made in the Parameter Store will not be deployed until the next ArgoCD sync in applied.

For more information on deployments, please review the Platform Documentation on Deployments.

Please open a ticket in #vfs-platform-support if changes are urgently needed. Backend support can restart the deploy and apply changes between normal deploys.

Delete a Setting

Step 1: Remove all occurrences of the Setting from vets-api

Step 2: Remove the Setting from the config files

config/settings.yml
config/settings/development.yml
config/settings/testing.yml

Step 3: Merge with master

Step 4: Copy the associated Parameters to archive path

/dsva-vagov/vets-api/archived/...

Archived parameter will be deleted after 30 days

Step 5: Delete the original Parameters

After the updated code is deployed and the Parameter is copied to /archive, you can safely delete the original Parameter. It’s important to ensure the updated code has been deployed or the Parameter value will be nil.

Rollback a Setting value

When a hardcoded value change(s) need to be rollbacked, follow the same PR revert procedures as a normal PR.

For rolling back a Parameter, delete the latest version of the Parameter. Then wait for a deploy to change the value in the pods.

Debugging

The two most likely reasons for the wrong value or no value for a Setting are misspellings or the pods haven’t been refreshed yet.

Verify the correct spelling and convention of the Setting key and ENV in config/settings.yml
1. Indentation is very important for yaml, so verify proper indentation
2. A common mistake is using a single underscore when a double underscore is needed
Check the Parameter path for spelling and conventions. A common spelling mistakes is using a singular env_var instead of the required env_vars.
Verify the value in the Parameter is correct.
Verify a deployment has occurred since the Setting or Parameter was last changed.
Check the full list of ENV variables (not including the values) is viewable through ArgoCD in the ssm-env-vars “Kubernetes Secret”.
Contact support

Screenshot of Kubernetes Secret in ArgoCD — Kubernetes Secret in ArgoCD

Best practices

Environment name in Setting name

RUBY

# bad
Settings.module.sandbox_key
Settings.module.staging_key

# good
Settings.module.key

Dot notation

Although the values can be accessed in a similar manner to hashes, it’s best to only use dot notation.

RUBY

# bad
Settings.dig('example', 'url')
Settings['example']['url']
Settings[:example][:url]

# really bad
Settings.send(:redis).dig('app_data')[:url]

# good
Settings.example.url

Mixing business logic with environment configuration

It’s a best practice not to mix business logic with environment configuration because you won’t be able to test the code until it’s deployed to production.

RUBY

# bad
if Settings.vsp_environment == 'production'
  do_something('prod_value')
else 
  do_something_else('dev_value')
end

# good 
do_something(Settings.do_something.arg)

It’s not always possible, but try to limit the differences as much as possible. For example, if you don’t want to send an email in local development, add a guard clause just before actually sending the email.

RUBY

# bad
class UsersController < ApplicationController
  def create
    @user = User.new(user_params)

    if @user.save
      UserMailer.with(user: @user).hello_email.deliver_now if Rails.env.production?
      render :show, status: :created, location: @user
    end
  end
end

  # ...
end

# good
class UserMailer < ApplicationMailer
  default from: "theteam@example.com"

  def hello_email
    return unless Rails.env.production?
    
    @user = params[:user]
    @url  = "http://example.com/login"
    mail(to: @user.email, subject: "Hello")
  end
end

Setting isn’t required

Tracking the environment(s) itself

Settings that re-purpose to the deployed environment name, are typically not necessary. This can include making the distinction between lower environments and production. It’s not ideal (see best practice above) to use the environment like this, but it’s better than adding another Setting for nothing.

RUBY

# bad
send_email unless Settings.my_service.mock
only_do_if_deployed if Setting.my_service.enabled

# good
send_email if Settings.vsp_environment == 'production' 
only_do_if_deployed if Rails.env.production?

# worse: if they track vsp_environment
Settings.my_service.environment_name
Settings.my_module.env

Underdeveloped feature

Features that aren’t production ready, aren’t settings. These settings should be used with Feature Flippers. Here’s the Platform Documentation on Feature Flippers: https://depo-platform-documentation.scrollhelp.site/developer-docs/feature-toggles-guide

RUBY

# bad
Settings.my_new_feature_enabled

# good
Flipper.enabled?(:my_new_feature)

Environment name in the value

Some non-secret values where the environment name or another setting is the only difference between values may not need to be in config/settings.yml.

RUBY

# bad
Settings.my_module.internal_url => 
  'https://api.va.gov/my_interal_url'
  'https://sandbox-api.va.gov/my_interal_url'
  'https://staging-api.va.gov/my_interal_url' 
  'https://dev-api.va.gov/my_interal_url' 

# good
def url
  "#{Settings.hostname}/my_internal_url"
end

# bad
Settings.my_module.file_path => 
  'some/path/development-file.txt'
  'some/path/staging-file.txt'
  'some/path/sandbox-file.txt'
  'some/path/production-file.txt' 
  
# good
def file
  "some/path/#{Settings.vsp_environemnt}-file.txt"
end

# best - remove the prefix
def file
  "some/path/file.txt"
end

Not really a Setting

Non-secret settings that only vary slightly between environments could be constants. This is could be the case for Settings with integer values and some hardcoded Settings. For example, a read_timeout doesn’t need to vary between deployed and local environments.

RUBY

# bad
external_client:
  read_timeout: 10
github_organization: department-of-veterans-affairs

# good
READ_TIMEOUT = 10
GITHUB_ORGANIZATION = department-of-veterans-affairs

Help and feedback

Suggest content changes to this page.
Submit new Platform Website content.
Get help from the Platform Support Team in Slack.
Submit a feature idea to the Platform.