Deployments

Last Updated: December 16, 2024

This page details the automated build and deployment process for vets-website, content-build, vets-api, forward proxy (HaProxy) and reverse proxy (OpenResty).

If you need info on forward or reverse proxy deployments, head down to the those sections of this page:

• Forward proxy deployment

• Reverse proxy deployment

Application deployments

This section contains information about application deployments like vets-website, content-build, andvets-api. This section covers the steps performed by GitHub Actions and Jenkins after a pull request is merged into the main branch, including artifact creation, deployment to development and staging environments, release creation, and production deployment. The document also outlines the deployment schedule, rollback procedures, and continuous deployment details.

GitHub Actions (for vets-website and content-build) perform the following tasks after a pull request is merged into main

1. Build main branch to create a deployment artifact (.tar file)

2. Deploy to development and staging using deployment artifact

3. Create a release in GitHub from main, tag artifacts of that commit SHA hash with release name

4. Deploy to production using deployment artifact according to automated deployment schedule

A big assumption in this process is that the main branch should always be deployable. As such, the deployment to the staging environment is configured to happen automatically and can be used to see what something would look like in a production-like environment for any kind of manual testing/verification.

See Vets API EKS deploy process for how vets-api is deployed.

Automated deployment schedule

All listed times are eastern timezone and are scheduled Monday–Friday (excluding holidays). All deployments are executed from the main branch in each repository.

View the Frontend Dashboard or the Backend Dashboard to see the deployment status of a vets-website or vets-api commit, respectively.

Automated process details

• Every work day at the configured time, the Daily Production Deploy (vets-website) and Daily Production Deploy (content-build) workflows will run.

• The Daily Production Deploy workflow in vets-website has several jobs that run, primarily including the

  • Create Release job, which tags the release for production, and the

  • Deploy job, which deploys the release artifact

• The Daily Production Release workflow in content-build only tags the release. The next Content Release workflow that runs will use the latest release.

• These jobs should not be triggered manually.

Rollbacks

If a production deployment introduces issues that affect Service Level Objectives (SLOs) established for the project, you may restore service to users by rolling back to a previous deployment. This is accomplished by triggering a new Daily Production Deploy workflow in GitHub Actions using a previous release tag. Typical deployment times are under 5mins.

1. Identify the release you want to rollback to by visiting the https://github.com/department-of-veterans-affairs/vets-website/releases or content-build release log(s)

2. Click on the commit ID in the left column of the release you want to reference

3. Copy the commit ref (it will be a long string like: 7c74702605561a33a5a6edbe46a95ac43dddb1df)

4. Visit the vets-website or content-build Daily Production Deploy workflow(s)

5. At the top of the previous workflow runs, click “Run workflow”

6. Enter the ref value into the “Deploy specific commit” field

7. Click "Run workflow"

If SLOs are not affected and a fix is not critical, no rollback will be issued. Instead the fix should be applied through the standard development workflow.

Continuous deployment

In addition to the daily production deploy, certain applications in vets-website can be deployed to production via continuous deployment. Applications on the allow-list are also deployed to production when changes to the application are merged into main and the CI workflow run for the commit is successful. To learn more about continuous deployment, see the documentation on continuous deployment.

Forward proxy (HaProxy) deployments

Below you’ll find a table that outlines the deployment schedule for forward proxy (HaProxy).

For more on forward proxy deployments, head over to the vsp-platfrom-fwdproxy Github page. There you’ll find information on triggers, testing, workflows, manuals, pull requests, and SSM port forwarding.

Reverse proxy (OpenResty) deployments

Below you’ll find a table that outlines the deployment schedule for reverse proxy (OpenResty).

For more on reverse proxy deployments, head over to the vsp-platfrom-revproxy Github page. There you’ll find information on triggers, testing, workflows, manuals, pull requests, and SSM port forwarding.

Additional information

Help and feedback

• Suggest content changes to this page.

• Submit new Platform Website content.

• Get help from the Platform Support Team in Slack.

• Submit a feature idea to the Platform.