Skip to main content
Skip table of contents

Data Encryption in Vets API

It’s understandably important to encrypt PII stored in Vets API, so we’ve adapted a few technologies to simplify the process and ensure ongoing security. We encrypt our data following the Envelope Encryption pattern, and store the Root Key, or “Customer Managed Key (CMK)”, in AWS. AWS’s Key Management Service (KMS) rotates this key annually, and Vets API has a Sidekiq job scheduled to re-encrypt all of our data with the new key soon after.

Technologies Used

  • Lockbox gem - Library used for encrypting data. The README describes how to create migrations and update your models accordingly for various types of data

  • AWS Key Management Service (KMS) - AWS Service used for storing and rotating the Root Key/Master Key. Once per year (Oct 11th as of writing this), the key’s cryptographic secret (“material”) is rotated, while keeping a record of previous versions for use in decrypting older data. More details can be found here.

  • kms_encrypted gem - Library using KMS for key storage and Lockbox for handling encryption. The Getting Started section of the README includes information on migrations and on updating your models. See below for notes on how we’ve patched the #has_kms_key method to abstract a few requirements away from the developer.

How to Setup Encryption in your Model

While the details may vary based on the type of data you are encrypting (see READMEs above for details), you can generally use the following pattern:

  • Migration - Here’s an example of how the OauthSession Model was updated to encrypt the user_attributes field.

    • Line 2 adds a new encrypted attribute, user_attributes.

      • Any number of attributes can be added

    • Line 3 adds the required data key used by the kms_encrypted gem

      RUBY 
      def change  
  add_column :oauth_sessions, :user_attributes_ciphertext, :text
  add_column :oauth_sessions, :encrypted_kms_key, :text
end

  • Model - Here’s an example of how the OauthSession Model was updated to encrypt the user_attributes field.

    • Line 3 adds the required has_kms_key method from the kms_encrypted gem. Do NOT add any arguments to this method (it’ll raise an error), as we have patched this method to implicitly pass in arguments for consistency across the app. See below.

    • Line 4 specifies which fields are encrypted, following the pattern from the Lockbox README

      • Note that multiple attributes can be define on a single line (i.e. has_encrypted :attribute_1, :attribute_2, key: :kms_key, **lockbox_options

        RUBY 
        class OAuthSession < ApplicationRecord
  ...
  has_kms_key
  has_encrypted :user_attributes, key: :kms_key, **lockbox_options
  ...
end

How to Encrypt Existing Data

Summary

You need to create multiple PRs. One to add the new encrypted columns and one to remove the old unencrypted columns.

Step 1: Add the Encrypted Columns

  • Migration - Follow the Migration Step in the How to Setup Encryption in your Model section

  • Model - Add migrating: true argument to the has_encrypted method as described above

RUBY 
class ExistingModel < ApplicationRecord
  
  has_kms_key
  has_encrypted :data, migrating: true, type: :json, key: :kms_key, **lockbox_options

end

Step 2: Update the Existing Data

Fortunately, you don’t have to worry above data migrations. During the nightly KMS Key Rotation Jobs, each record will get updated when the keys for that record are updated.

Step 3: Remove the Dld Unencrypted Column

  • Confirmation - After the nightly job is done, confirm the data in your model’s table has been encrypted.

  • Migration - Then create a migration to safely remove that column from your table

RUBY 
class DropDataInExistingModel < ActiveRecord::Migration[7.1]
  def change
    safety_assured { remove_column :existing_models, :data, :jsonb }
  end
end

  • Model - You can remove migrating: true from the has_encrypted method

Custom KMS Versioning

Our application uses custom versioning within the data key of every record, that describes the year in which the Root/Master Key rotated. Our KMS Root Key is rotated on October 11th, so we use October 12th as our cutoff date. Therefore, every record created after Oct 12th (00:00) will have the year in which the rotation took place prepended to the encrypted_kms_key (i.e. v2023:asdf1324....) up until the following October 12th, at which time it will begin prepending with the following year. For example, in July of 2024 a new record would have v2023 prepended to their encrypted_kms_key, while records created after the 2024 rotation, say November, would have v2024. Upon October 12th, 2024, a Sidekiq job will kickoff programmatically rotating all records still using the v2023 encryption material.

Automating the KMS Key Rotation

While AWS KMS automatically updates the Root/Master key on a yearly cadence, the records themselves are not. When KMS rotates the key, it is updating the cryptographic material/secret it uses to encrypt the records. KMS saves all previous versions of the cryptographic material in perpetuity, so that it can always be used to decrypt data, even once it has been rotated.

Soon after the Master key is rotated, we kick off a sidekiq job that batches encrypted records and kicks off secondary jobs to apply the kms_encrypted method, rotate_kms_key! that decrypts and re-encrypts the record with the updated cryptographic material.

Notes about our kms_encrypted patch

During the implementation of the automated KMS Key Rotation, we decided to use custom versioning to more easily search and identify records that have/have not received the most recent encryption material. In the kms_encrypted.rb initializer, you can see that we patch the KmsEncrypted::Model module with the KmsEncryptedModelPatch, in which we do a couple things:

  • We explicitly call the default method (super) with “kms_options", so that future developers don't have to worry about including the same logic

  • We calculate the kms_version based on the KMS Root/Master Key rotation date (Oct. 11th EOD)

  • We enumerate previous_versions so that any version that may be prepended to encrypted data will point to the same Master Key. This was especially important during the initial implementation when there were a variety of versioning schemes being used, but now it’ll just be a safeguard in place for the case that a team initially creates data without using this pattern, and later needs to adopt it.

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.

Contact Support Close