Store a secret in Parameter Store
The Platform uses AWS SSM Parameter Store to store secrets and other variables that may be necessary for services to function properly. For the purposes of this document, a “secret” can be a password, API key, or other string of information that your application may need to communicate with other applications. This guide provides details and best practices for storing and managing Platform secrets, including:
How secrets are organized
All service secrets are stored in the dsva-vagov/<service-name>/ path.
All shared team secrets (secrets that belong to a team and not to a service) will be in a top-level path leading with the team name, e.g.: cms, devops, frontend-team. The team who creates a path will have full permission to edit any secret in that path.
The Platform Operations team will manage the /devops path and store shared variables that other teams may need to access to ensure we only manage the credential in a single place, e.g.: VA_VSP_BOT_GITHUB_TOKEN, etc.
The Platform Operations team will create and manage service accounts in AWS and will give those service accounts READONLY permissions for the paths they require for the team’s operation.
Secrets are organized in a tree structure with the below hierarchy:
/dsva-vagov
/<service-name> e.g.: vets-api, vets-website, etc.
/TOP_LEVEL_VARIABLE_1
/TOP_LEVEL_SECRET_1
/dev
/VARIABLE_2
/SECRET_1
/staging
/VARIABLE_1
/SECRET_2
/prod
/VARIABLE_3
/SECRET_3
/sandbox
/VARIABLE_4
/SECRET_4
/cms
/dvp
/devops
/frontend-team
….
Managing secrets
The following instructions assume that you have the AWS Command Line Interface (AWS CLI) and jq installed.If you don’t have the AWS CLI set up, view our instructions to set up a session token with the AWS CLI before attempting to manage secrets.
First, retrieve the secrets and store them locally.
aws ssm describe-parameters > ./parameters.json
View a list of parameter names to check if the parameter you want to add already exists.
jq '.Parameters[]?.Name' ./parameters.json | less
Find a secret for a specific service
All the commands have the same result. This example shows a few different use cases for jq
:
jq '.Parameters[]? | select(.Name | test("^/dvp/")).Name' ./parameters.json
jq '.Parameters[]? | select( (.Name/"/")[1] == "dvp" ) | .Name' ./parameters.json
jq '.Parameters[]? | select( (.Name|split("/"))[1] == "dvp").Name' ./parameters.json
Find a secret for a specific environment
jq '.Parameters[]? | select(.Name | test("^/[^/]+/staging")).Name' ./parameters.json
jq '.Parameters[]? | select( (.Name/"/")[2] == "staging" ) | .Name' ./parameters.json
jq '.Parameters[]? | select( (.Name|split("/"))[2] == "staging").Name' ./parameters.json
All of our secrets currently use the same key for encrypting values, but since that may not always be the case, you can retrieve the key and information (with some info redacted) using the following example:
jq '[.Parameters[]? | select(.Type=="SecureString") | .KeyId] | unique
[
"alias/aws/ssm"
]
aws kms describe-key --key-id alias/aws/ssm
> {
> "KeyMetadata": {
> "AWSAccountId": "",
> "KeyId": "",
> "Arn": "",
> "CreationDate": 1534194438.007,
> "Enabled": true,
> "Description": "Default master key that protects my SSM parameters when no other key is defined",
> "KeyUsage": "ENCRYPT_DECRYPT",
> "KeyState": "Enabled",
> "Origin": "AWS_KMS",
> "KeyManager": "AWS",
> "CustomerMasterKeySpec": "SYMMETRIC_DEFAULT",
> "KeySpec": "SYMMETRIC_DEFAULT",
> "EncryptionAlgorithms": [
> "SYMMETRIC_DEFAULT"
> ],
> "MultiRegion": false
> }
> }
How to add, update, and delete secrets
Adding a secret
This command will create an encrypted value. When running the command, it’s given a name, value, type, and tags. All can be done in a single command:
aws ssm put-parameter \
--name "/test/devops/example" \
--value 'P@sSwW)rd' \
--type 'SecureString' \
--key-id 'alias/aws/ssm' \
--tags 'Key=Name,Value=example' 'Key=Creator,Value=Ryan'
> {
> "Version": 1,
> "Tier": "Standard"
> }
Updating a secret
The following example shows how to update a secret value in a single command:
aws ssm put-parameter \
--name "/test/devops/example" \
--description 'An example key to be deleted later' \
--value 'P@sSwW)rd' \
--type 'SecureString' \
--key-id 'alias/aws/ssm' \
--overwrite
> {
> "Version": 2,
> "Tier": "Standard"
> }
Retrieving a secret
Example with decryption
Using the example above, let’s retrieve the full value of the secret. The values that return will be encrypted unless you specify --with-decryption
on the command line:
aws ssm get-parameter --name '/test/devops/example' --with-decryption
Example with multiple parameters
The following example will allow you to retrieve multiple secrets at once by wrapping the name of the secret in single quotes with a space between each parameter.
aws ssm get-parameters --names '/test/devops/example1' '/test/devops/example2'
Example with revision history
The following example allows you to view the revision history for your secret values:
aws ssm get-parameter-history --name '/test/devops/example' --with-decryption
> {
> "Parameters": [
> {
> "Name": "/test/devops/example",
> "Type": "SecureString",
> "KeyId": "alias/aws/ssm",
> "LastModifiedDate": 1636052332.861,
> "LastModifiedUser": "",
> "Value": "P@sSwW)rd",
> "Version": 1,
> "Labels": [],
> "Tier": "Standard",
> "Policies": [],
> "DataType": "text"
> },
> {
> "Name": "/test/devops/example",
> "Type": "SecureString",
> "KeyId": "alias/aws/ssm",
> "LastModifiedDate": 1636052462.067,
> "LastModifiedUser": "",
> "Description": "An example parameter to be deleted",
> "Value": "P@sSwW)rd",
> "Version": 2,
> "Labels": [],
> "Tier": "Standard",
> "Policies": [],
> "DataType": "text"
> }
> ]
> }
Deleting a secret
Once you no longer need a secret, you can delete it by using the following examples.
Single secret
aws ssm delete-parameter --name '/test/devops/example'
Multiple secrets
aws ssm delete-parameters --names '/test/devops/example1' '/test/devops/example2'
Additional resources
https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html
In addition to the using AWS CLI, there are 2 alternative ways to interact with AWS Parameter Store:
Use the AWS Web Console. This is the simplest method for which we’ve provided the above examples.
GitHub - smblee/parameter-store-manager. This method uses a cross-platform desktop application that provides an UI to easily view and manage AWS SSM parameters.
Help and feedback
Create an issue ticket to suggest changes to this page