Overview

GitHub Actions (for vets-website and content-build) and Jenkins (for vets-api) perform the following tasks after a pull request is merged into main

  1. Build main branch to create an deployment artifact (.tar file)

  2. Deploy to development and staging using deployment artifact

  3. Create a release

    in GitHub from main, tag artifacts of that commit SHA hash with release name

  4. Deploy to production using deployment artifact according to automated deployment schedule

A big assumption in this process is that the main branch should always be deployable. As such, the deployment to the staging environment is configured to happen automatically and can be used to see what something would look like in a production-like environment for any kind of manual testing/verification.

Automated deployment schedule

All listed times are eastern timezone and are scheduled Monday–Friday (excluding holidays). All deployments are executed from the main branch in each repository.

View the Frontend Dashboard to see the deployment status of a commit.

Application

Changes in by

Deployment Start

Release Information

GitHub Action Workflow/Jenkins Job

vets-website

2:00pm ET

3:00pm ET

vets-website release history

Daily Production Deploy (vets-website)

content-build

2:00pm ET

3:00pm ET

content-build release history

Daily Production Release (content-build)

vets-api

2:00pm ET

3:00pm ET

vets-api release history

vets-gov-autodeploy-vets-api

content-build (content only)

n/a

9am–12pm Hourly, 1:45pm, 4pm, 5pm ET

content-build latest release

Content Release

Automated process details

  • Every work day at the configured time, the Daily Production Deploy (vets-website) and Daily Production Deploy (content-build) workflows will run.

  • The Daily Production Deploy workflow in vets-website has several jobs that run, primarily including the

    • Create Release job, which tags the release for production, and the

    • Deploy job, which deploys the release artifact

  • The Daily Production Release workflow in content-build only tags the release. The next Content Release workflow that runs will use the latest release.

  • These jobs should not be triggered manually.

Rollbacks

If a production deployment introduces issues that affect Service Level Objectives (SLOs) established for the project, you may restore service to users by rolling back to a previous deployment. This is accomplished by triggering a new Daily Production Deploy workflow in GitHub Actions using a previous release tag. Typical deployment times are under 5mins.

  1. Identify the release you want to rollback to by visiting the vets-website or content-build release log(s)

  2. Click on the commit ID in the left column of the release you want to reference

  3. Copy the commit ref (it will be a long string like: 7c74702605561a33a5a6edbe46a95ac43dddb1df)

  4. Visit the vets-website or content-build Daily Production Deploy workflow(s)

  5. At the top of the previous workflow runs, click “Run workflow”

  6. Enter the ref value into the “Deploy specific commit” field

  7. Click "Run workflow"

If SLOs are not affected and a fix is not critical, no rollback will be issued. Instead the fix should be applied through the standard development workflow.

Additional information


Feedback and support