Skip to main content
Skip table of contents

Cypress Integration Test Workflow - for CI failures

Last Updated: October 24, 2024

Introduction

Engineers working with vets-website need to be able to quickly iterate on integration test specifications when there are e2e failures in the Continuous Integration workflow. Due to differences between local and CI execution environments Cypress tests do not always produce the same results. The Cypress Integration Test Workflow allows users to more readily diagnose tests that are failing in the CI workflow without having to wait for the full CI workflow to run, which can take anywhere from twenty minutes to an hour to complete each time.

The Cypress Integration Test Workflow MVP is now available here.

Steps for using the workflow:

  1. From the above link, you will arrive at the workflow’s page in vets-website, featuring the workflow name and a list of linked, past workflow runs.

the workflow title is shown, with its file name showing as a subtitle. You are at the right page if you see a message about the workflow having a Workflow Dispatch event trigger.

Workflows can be found in the Actions tab of a given repository

  1. On the right side of the page, find the Run workflow button and click it. You should see something matching the screenshot below.

Upon clicking the Run workflow button, a modal will appear, with four required user inputs.

The latest appearance of the modal at the time of writing

  1. You will provide a set of inputs to dictate the parameters for a new workflow run:

    1. You will use the workflow from the main branch of vets-website, unless you are testing changes with the workflow itself.

    2. The Commit SHA field will require a full SHA, not the truncated one visible in most places where it can be found in vets-website. Recent commits may need time for CI to produce and archive a build for their given SHA.

      1. This can be most conveniently captured by navigating to the Code tab found on the vets-website repository page, then clicking the commit count button. In the screenshot below, it appears as “29,873 Commits.”

        The Code tab of the vets-website repository. The desired button has no border until focused.

        If the page default of main is left unchanged, that button defaults to this page: https://github.com/department-of-veterans-affairs/vets-website/commits/main/

        The page will load a list of recent commits and their SHAs, which represent a squashed branch that has been merged into main. A branch menu, which defaults to “main,” is visible above the list, and can be used to specify your current branch. As seen in the screenshot below, each commit includes a copy button, as shown in the screenshot below. Simply click it to copy the chosen commit’s SHA to your clipboard. Now you can return to the Cypress Integration Test Workflow’s page/tab to paste that into the input field.

        Each commit line has a copy button that will copy the full (not shown) SHA to your clipboard.

        The SHA shown beside the copy button is truncated for the UI and will not be useful for the workflow

      2. Alternatively, you can reach the commit SHAs from the compare page of your branch in the vets-website repo:

        1. https://github.com/department-of-veterans-affairs/vets-website/compare/[your branch name here]…main

        2. Click the copy icon associated with your commit as done with the other method, then paste it into the workflow input field.

c. Next is the Are your tests failing in […] input. You will now help the workflow build the specPattern for your app-specific tests by specifying one of the following directories:

CODE 
vets-website/src/applications

or

CODE 
vets-website/src/platform

d. In the final input field, provide the name of the app/product directory that contains your failing test specs, as it would appear in the path above. For example: appeals if your e2e test failures originate from specs stored in vets-website/src/applications/appeals.

i. Some universal e2e specs, such as any site-wide tests, are stored in vets-website/src/platform/ and will need the directory name provided in the same way as above.

  1. Once you have provided your inputs, click Run Workflow. image of the green Run Workflow button The page will refresh. Your newly queued workflow may not immediately appear. Simply wait for the page to update the listing.

Run Times

Workflow run times will vary, based on your use case. Running the e2e specs for products like the 10-10EZR form may take more than 15 minutes, while smaller ones may take as little as 7.

Once your workflow is complete, review the test results. If there are any specs consistently failing, the workflow can be run again once your latest product code changes have been committed and pushed to the repo. Just remember to replace the build SHA input with one from your commits as you push them.

Goals for this workflow

Ultimately, waiting on CI to rerun tests with each push is a slow momentum-killer. Our goal is to provide a more rapid means of evaluating and correcting the problems behind e2e test failures. As this workflow is currently in an MVP state, suggestions/feedback are not only welcome, but requested.

Please don’t hesitate to reach out to the Platform Reliability Team with any questions or feedback.

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.

Contact Support Close