There are many different scenarios where Cypress end-to-end tests may fail. It’s important to understand both why and how to solve these issues. Test failures are blockers in GitHub Actions deployment workflows in every instance, so these solutions will help your code move forward.
Step 1: Diagnose the problem
There’s a few different scenarios in which a test may fail. The solution varies on a case by case basis depending on what caused the failure in the first place. We have many tools available to help diagnose the cause of the failure including:
GitHub Actions logs output in the workflow showing the results of the tests.
GitHub commit history will show any code that may have impacted the test result.
Step 2: Select a solution
Dependent on what your discovery was, the solution may vary. There are three common areas that test failures will fall into:
Code was changed in the application itself that impacted the test results. (See: Solution A)
The test spec was updated and it does not work with the application. (See: Solution A)
Content Build’s most recent build contained a content update that was not in the build that the PR branch was tested against. (See: Solution B)
Diagnose the merged code that caused the failure. There are many tools to assist with this such as Cypress Mochawesome reports that will show a video of the test failure as well as our git history.
Once the problematic code is located, revert the commit that caused the failure.
Notify the appropriate parties that the problematic code is removed and the deploy may be re-run.
Temporarily disable the test by adding a
itblock in the test spec. A key part of this solution is that the test is only disabled temporarily. Please resolve the issue and enabled the test again.
Create a task/ticket to update the test to account for the new content
Reenable the test ASAP.