Modernized VA.gov Accessibility Strategy
The purpose of this document is to co-create a single, cross-discipline, cross-contract, VA-wide accessibility strategy document we can point to as a north star to answer:
How we define accessibility for VA.gov
How we’ll know we’re successful (or not), and
How we’ll measure and keep track of our progress.
How did we make this?
We started this effort through a prioritization exercise based on problems our accessibility specialists identified.
What does this cover?
For now, this strategy document covers the modernized VA.gov site (specifically, the parts that the Office of the Chief Technology Officer [OCTO] is responsible for) and the VA: Health and Benefits native mobile app.
How will this change?
Everything in this strategy is intended to be flexible and may change over time.
Mission
Empower product teams to collaborate with disabled Veterans and caregivers by enhancing the culture and tools used to co-create and maintain services that are accessible beyond compliance.
Vision
Every disabled Veteran and caregiver has guaranteed access to equitable, easy to use, self-service tools without needing to request accommodations1.
Key Success Metrics
We plan to review the metrics below using the Staging Review Dashboard in Domo. Changes in metrics should be reviewed by accessibility leads to evaluate effectiveness of tactics, inform new tactics, create new conversations, or re-evaluate broader accessibility strategy.
None of these key metrics are permanent and are meant to be discussed and modified over time by the team.
Percentage of research studies with the accessibility label
Helps us measure the inclusion of disabled Veterans in our research.
Why? Most VA.gov products have not been tested with disabled Veterans who use assistive technology or have compounded disabilities.
How will we track this?
We are using the accessibility label,
accessibility, in the research repo. Currently, the criteria for the accessibility label is up to the researcher. It's most often applied to research specifically looking at accessibility concerns or when the majority of participants use assistive technology (AT). It's also sometimes being applied when research includes any AT user participants.In the future, we plan on collaborating with the research ops team to either (a) automate inputs for requested assistive technology users or (b) manually track how many Veterans emerge in studies through the recruitment checker or by following studies in the research ops board.
We track monthly data from our recruiting partner as to how many AT user participants attended research sessions.
For 2023: 223 of 987 moderated studies were with AT users (23%)
For 2022: 182 of 826 moderated studies were with AT users (22%)
How will we monitor and use it?
If the number consistently dips over time, it’s a sign we’re not including Veterans with disabilities and that we should take a closer look at studies.
This metric can help open up conversations related to blockers around Veterans with disabilities who aren’t able to participate in research, for example, inaccessible prototypes.
Submetrics
In the future we may be able to keep track of what types of assistive tech are being tested, specifically. We’ll work with research ops to standardize how we’re tracking these.
Percentage of staging reviews with no major issues (defects 0-2) written
Helps us measure at large if accessibility is succeeding at shifting left (regardless of what tactics are being used).
Why? Some Collaboration Cycle reviews still produce major defects.
How will we track this? The Staging Review Dashboard has this data.
How will we monitor and use it?
If this number increases, it’s a signal that our tactics for shifting left are working.
If this number decreases, it can open conversations around exploring different tactics for shifting left. For example, we might ask “are there any patterns among the issues being detected by type and team?” We are interested to see if there is a notable difference between teams with and without embedded accessibility specialists.
Number of unique VFS teams that contact accessibility specialists
Helps us measure how proactive or involved VFS teams are in collaborating with accessibility specialists early and often.
Why? In an ideal world, VFS teams should be more incentivized to learn and seek out effective accessibility practices from accessibility specialists.
How will we track this?
The Content, Accessibility, and Information Architecture (CAIA) team keeps track of how many people attend their open office hours and what they are asking about.
Accessibility specialists from CAIA and the Benefits teams are writing an Accessibility Cookbook to document recurring discussions and decisions.
How will we monitor and use it?
If the number of unique questions decreases significantly, we'll be motivated to look into the causes e.g. is it motivation based? Are teams aware that office hours exist? Etc. The spreadsheet also allows us to look at the sub-metrics related to what teams are asking questions, where they’re asking questions, and what kinds of questions they’re asking.
Percentage of accessibility specialists with a balanced workload
Helps us measure if accessibility specialists are: working sustainably, not suffering from “burnout”, and feel satisfied with their work.
Why? Accessibility specialists burn out quickly and have been historically overworked.
How will we track this? Weekly, on a Google spreadsheet or as part of the CAIA team metrics.
How will we monitor and use it?:
Our accessibility leads intend to use this to keep track of specialist workloads and know when to step in to help and mitigate burnout before it happens.
As an accessibility practice, this can help inform decisions to assign or reassign tasks for VA.gov wide accessibility efforts.
Support hiring decisions
In the future we may want to consider adding on to this by using a count of backlog tickets and time to completion.
[ 🛠️ In Progress] Average time to close out accessibility bugs
Helps us measure if accessibility is being prioritized by teams.
Why? Many accessibility issue tickets aren’t prioritized and don’t get fixed in a timely manner. We have 500+ issues that remain unresolved.
How will we track this? We’re working with the Platform Analytics team on building Domo dashboards around issue ticket lifespans with the intention of being able to filter by accessibility.
How will we monitor and use it? When established, we can use this metric over time to start conversations on which teams are (or aren’t) prioritizing accessibility and why.
Percentage of components and patterns we consider to have deficient (or sufficient) accessibility guidance
Why? This will help us understand how embedded accessibility is in the design system which scales across to all (Veteran Facing Services) VFS teams.
How will we track this? Accessibility information is provided on each component page in the VA design system.
How will we monitor and use it?:
If the metric remains stagnant or trends in a negative direction, it’s a possible sign that many new inaccessible components are being added or the remediation of existing elements isn’t being prioritized.
In the future, we may also want to expand this to tracking severity of issues within the design system. That depends on if there will be a consistent review of components (e.g. the number of tickets, and their severity, will fluctuate more based on the consistency and spread of audits and less on the actual number of issues present that have yet to be discovered).
Strategies
These broader strategies will help us form tactics which we’ll evaluate based on the key metrics listed above.
Some strategies listed here are intended for a future state of accessibility at the VA and may not be applicable now. For example, we don’t have the ability to collect certain key metrics to evaluate the accessibility of internal tools, methods, and processes.
1. Self-service benefits accessed on VA.gov by disabled Veterans are co-designed with disabled Veterans.
Problem this is meant to solve:
We don’t have any standardized method or practice to measure and learn if our accessibility recommendations are having a positive impact on the user experience for disabled Veterans.
While we’re confident that our recommendations can create access, a truly inclusive, easy to use self-service experience is most effectively designed with users.
Mitigates us from creating disability dongles (well-intended, but ineffective solutions for disabled people)
2. VA.gov is the standard for accessibility beyond compliance in both the private and public sector.
Problem this is meant to solve:
Aside from (arguably) gov.uk, there aren’t many excellent “standards” for inclusive services for the disabled experience. Referencing subpar, inequitable experiences from the private industry may hurt our efforts more than help them.
Aiming to become the new standard instead of just catching up to a low bar can enable us to think even further beyond compliance. We believe this will empower us to focus more on the disabled Veteran experience, not just an automated count of technical issues.
Sharing what we learn with other agencies can (long term) help create a more equitable American democracy.
3. All product teams have the needed space, time, knowledge, and power to embed accessibility as part of their key responsibilities.
Problem this is meant to solve:
Accessibility resources are currently scattered across VA.gov with little to no maintenance or organization.
Some teams don’t know where to get accessibility guidance, who to reach out to, etc.
Some teams may not be aware that accessibility is a part of their key responsibilities when launching products. (They may also be unaware of what those specific responsibilities are.)
Accessibility is not (yet) an organized practice. It is currently a loosely organized group of specialists collaborating on Slack.
4. The systems and tools that allow work to happen also enable, not block, accessibility beyond compliance.
Problem this is meant to solve:
In order for us to employ disabled employees (who may themselves be Veterans), we need to make internal ops (tools, methods, etc.) and culture more accessible and inclusive to them. This is important as it's both a legal mandate (which Congress wants prioritized) and a critical component in inclusive and participatory design. Hiring more disabled Veterans empowers our teams to have more perspectives centered around lived experience, and ultimately leads to more equitable products and services.
Accessibility specialist work is prone to burnout. Without clear expectations on scope, resources, and support, we may end up experiencing high levels of turnover.
Contributors
It takes a lot of people with lots of different, diverse perspectives to make this work. Credit go to the following individuals for writing, reviewing, or providing direction and support:
Martha Wilkes (our executive sponsor)
Angela Fowler
Brian DeConinck
Tiffany Pender
Laura Flannery
Brea Blackwelder
Courtney Chambers
Meli Manak
Jared Cunha
Ryan Smith
Sarah Koomson
Josh Kim
David Kennedy
Eli Mellen
Shawna Hein
Mickin Sahni
Mike Gifford
Jenny Wang
Anish Kshatriya
Clayton Zook
Liz Lantz
Matt Dingee
Naomi Marcussen
Shira Goodman
Danielle Thierry
Tim Wright
Michelle Middaugh
Nina Anusavice
Christopher Walker
Cory Trimm
Want to become a contributor? Let us know at #accessibility-ops.
Footnote
Accessibility can be thought of as a continuum. Inaccessible experiences exclude people. Accommodations are modifications made for disability on an as-needed basis. Accessible experiences provide equal access to everyone without disability tax, emotional tolls, microaggressions, or psychological burdens.
Help and feedback
Get help from the Platform Support Team in Slack.
Submit a feature idea to the Platform.