Accessibility defect severity rubric

This document will provide a common language for determining accessibility defect seriousness. It will capture the following things:

1. Defect severity

2. Timeline to fix the defect

3. Potential users impacted by the defect

This rubric aligns to both the axe-core rule descriptions severity descriptions and human impact.

Launch-Blocking Issues

❗️ Must be fixed before production launch.

a11y-defect-0 and a11y-defect-1 issues have the potential to make an application unusable for a significant group of users.

a11y-defect-0

❗️ Potentially Harmful. Must be fixed immediately.

These issues have the potential to cause life-threatening situations and should be remediated before work starts or continues on a11y-defect-1 issues. Our audience has a higher incidence of traumatic brain injury (TBI) and post-traumatic stress disorder (PTSD), so we need to correct these issues immediately.

Types of issues

• Applications must not have high flicker rates (could trigger epileptic seizures)

  • Three Flashes or Below Threshold: Understanding SC 2.3.1

• Motion or movement should not cover a high percentage of the screen. This could lead to possible vestibular issues.

  • WCAG2.1 Understanding Success Criterion 2.2.2: Pause, Stop, Hide

  • A List Apart: Accessibility for Vestibular Disorders: How My Temporary Disability Changed My Perspective

Potential impact

• Users who have photosensitivity or seizure disorders

• Users who have vestibular issues like motion sickness

a11y-defect-1

❗️ Critical. Must be fixed before launch.

These issues have the potential to severely disrupt the user experience and must be fixed before the Platform will certify a staging accessibility review.

Types of severity 1 issues in web applications and pages

• Focus must be managed for single-page applications

  • Name, Role, Value: Understanding SC 4.1.2

• Keyboard must not trap focus

  • No Keyboard Trap: Understanding SC 2.1.2

  • Examples of Success Criterion 2.1.2

  • Content must be reachable without a pointer device like a mouse:

  • UI or form controls must be reachable by TAB or SHIFT + TAB

  • UI or form controls must be reachable by screen reader shortcuts or menu options

• Violations reported by the axe browser plugin must be corrected. Teams may run additional checks using toosl such as:

  • WAVE

  • SortSite

  • AATT

• Videos must have captioning

  • Captions (Prerecorded): Understanding SC 1.2.2

• Color contrast issues must be corrected

  • Contrast (Minimum): Understanding SC 1.4.3

• Form inputs must have semantic labels

  • Labels or Instructions: Understanding SC 3.3.2

• Content being added or removed from the page must announce a change to assistive devices

• Improper aria attributes or missing IDs must be corrected. See the rules of ARIA use.

• Elements that can receive focus must have a yellow focus halo. This includes buttons, links, form inputs, textareas, and custom components. The following CSS rule should be present on the element that has keyboard focus. See WCAG2.1 Understanding Success Criteria 2.4.7: Focus Visible

  CSS

  outline: 2px solid #f9c642;

Types of severity 1 issues in PDF documents

• Document tables are missing a header

Potential impact

• Users who navigate by keyboard

• Users who navigate by screen reader, magnification software, or other assistive device

• Users who have low-vision, partial vision, or are blind

• Users who are hard of hearing or deaf

• Users with short-term memory or other cognitive issues

Post-launch Issues

⚠️ Should be fixed as soon as possible

a11y-defect-2 and a11y-defect-3 issues have the potential to degrade the user experience and should be fixed as soon as possible. These issues may have a short or long time horizon; some are violations, others are suggested improvements.

a11y-defect-2

⚠️ High. Should be fixed in 1-2 sprints post-launch.

These issues have the potential to disrupt the user experience and should be fixed before production launch if possible, or prioritized in the first 1-2 sprints post-launch. These issues will not delay your product launch.

Types of issues

• Page <title> tags must update on every route (URL) change. This includes single-page apps (SPAs).

  • <h1> should match the <title> or be included in the <title>

    • Example title: Check your appeals status | VA.gov

    • Example heading: Check your appeals status

  • Page Titled: Understanding SC 2.4.2

• Zoomed layouts must support use without horizontal scrolling, or have difficult to read text. Horizontal scrolling is allowed for data tables, images, or other rich media.

• Color blindness issues must be corrected. A common example is color being used to distinguish links from plain text.

• Labels or calls to action must be descriptive to assistive devices. Some examples include:

  • A button to edit a form in place. The button says "Edit", but doesn't answer "Edit...what?" to a screen reader. The user is asked to work back through the page and listen for a heading or key text to identify what this button might do.

  • A button labeled "Learn more" that expands an accordion. Again, screen reader users are left to figure out what they might be learning more about.

  • This issue is listed as a a11y-defect-2 because non-descript labels can reduce the overall experience for screen reader users. These labels usually appear in repeated UI controls, increasing the cognitive load on users.

  • This issue can be remediated in a couple of ways:

    HTML

    /*
     * Best if you need to support language translation
     */
    <button>
      <span aria-hidden="true">Edit</span>
      <span class="sr-only">Edit street address</span>
    </button>

    HTML

    /*
     * Less markup, but does not support language translation
     * <https://adrianroselli.com/2019/11/aria-label-does-not-translate.html>
     */
    <button aria-label="Edit street address">Edit</button>

  • Accessible labels must be included in visual labels

  • WCAG2.1 Understanding Success Criterion 2.5.3: Label in Name

• Applications must have a logical source order

• Applications must have elements that receive focus in an order that preserves meaning and usability for sighted, keyboard, and assistive technology users. All users should be able to determine where their keyboard focus is quickly, and understand when that focus changes based on changes in the user interface.

• HTML should use meaningful landmarks. These include:

  • <banner>

  • <navigation>

  • <main>

  • <aside>

  • <footer>

  • divs with role attributes.

• HTML should have properly nested headings. Adding an <h4> as the first heading underneath an <h2> is an example of improper nesting. The WebAIM WAVE or headingsMap browser plugins provide visual representations of heading nesting.

Potential impact

• Users who navigate by keyboard

• Users who navigate by screen reader, magnification software, or other assistive device

• Users who have low-vision, partial vision, or are blind

• Users who are hard of hearing or deaf

• Users with short-term memory or other cognitive issues

a11y-defect-3

⚠️ Medium. Should be fixed in 1-3 sprints post-launch.

These issues have the potential to disrupt the user experience and should be fixed as time allows in the first 1-3 sprints post-launch.

Types of issues

• Invalid HTML (non-semantic, badly formed) must be fixed

  • Parsing: Understanding SC 4.1.1

• Pages should have a way to skip navigation blocks of three or more links

  • Bypass Blocks: Understanding SC 2.4.1

• Errors in early release tools like axe-coconut should be fixed

• Best practice warnings in automated scanning tools like aXe browser plugin, SortSite, AATT should be fixed

Potential impact

• Users who navigate by screen reader, magnification software, or other assistive device

• Users who have low-vision, partial vision, or are blind

• Users who navigate by keyboard

a11y-defect-4

✔️ Low. Consider fixing or exploring in 2-4 sprints post-launch.

These issues and items to consider can improve the experience for assistive technology users and those with other challenges. These issues might include research and design iteration, so they have a longer timeline for entering production.

Types of issues

• Less semantic HTML (using <ul> instead of <ol> for sequential instructions, for instance) should be fixed

  • Parsing: Understanding SC 4.1.1

• Best practice warnings in early release tools like axe-coconut should be fixed

Potential impact

• Users who navigate by keyboard

• Users who navigate by screen reader, magnification software, or other assistive device

• Users who have low-vision, partial vision, or are blind

• Users who are hard of hearing or deaf

• Users with short-term memory or other cognitive issues

Help and feedback

• Suggest content changes to this page.

• Submit new Platform Website content.

• Get help from the Platform Support Team in Slack.

• Submit a feature idea to the Platform.