This checklist can help guide you through all of the tasks associated with conducting research on VFS projects.
1. Planning for research
3 - 4 weeks before research begins
- Make a research folder within the Product folder
Create a research folder (If you don’t currently have one) in your product's folder on GitHub (Add file > create new file > product/research/README.md). The “README.md” does not need to contain any information.
- Make a research folder for your initiative-specific research
Create an initiative-specific research folder (naming convention: YYYY-MM-Research-Name) within the research folder. Create a markdown file named research-plan.md and save it in the initiative-specific research folder you created. Then, copy and paste from the research plan template into your new file. Fill in your plan using the template; adjust as needed. Store your conversation guide and research plan in this folder. Remember to discuss your research plan with your team.
TEMPLATE: research plan
Research Folder Example:
Product-One (folder) >
Research (folder) >
YYYY-MM-Research-Initiative-One (folder) >
Research Plan (document)
Conversation Guide (document)
Static Screenshots (Sketch file)
Research Findings (document)
- Prepare your prototype or mock-up
Have the thing you’re testing ready! The Collaboration Cycle should conduct a Midpoint Review a week (minimum) before you start your research. They make sure it meets VA.gov quality standards before putting it in front of veterans.
- Create a conversation guide
First, create a markdown file named conversation-guide.md and save it in your research folder (i.e. product-name/research/conversation-guide.md). Then, copy and paste from the conversation guide template into your new file, and adjust the template as needed. Work with your team to complete the conversation guide.
TEMPLATE: conversation guide
- Make your research inclusive and accessible
Consider ways to make your research more inclusive and accessible. Some ideas are to set inclusivity requests for recruitment, like using the MVS sampling method; include mobile or have a separate study for mobile usability testing; and even if you won’t have a coded version, there are still assistive technology users you can recruit to test your design.
2. Research review
At least 10 days before research begins
- Add your study to the repo board
Create a new ticket using the research repo ticket template.
- Submit your research for review
See instructions on how to get your research approved and to kickoff participant recruitment.
- You may also choose to consult on the accessibility of your planned research
While not required, you may reach out to VA.gov accessibility specialists, #accessibility-help on Slack, for advice on effective and ethical research design for disabilities and assistive technology.
3. Participant recruitment
At least 7 days before research begins
- Initiate your recruitment plan
Once your research is approved, move your Research Repo card to the “Current” column. Then, if you are using Perigean for recruiting, Shane Strassberg will send the kickoff email signaling Perigean to begin recruiting.
4. Run a pilot session
Anytime before your first research session
- Identify your pilot participant(s)
Identify who will participate in your pilot session. We recommend recruiting a developer from your team to get them involved early in the process.
- Set up the pilot session
Set a meeting time with your pilot. Include the guidelines about participating in a pilot with the invite so folks know what to expect.
- Run the pilot session
Run pilot session and note where any adjustments could make future sessions run smoother.
- Make any adjustments to your conversation guide
Make sure note-taker(s) and moderator(s) have the most recent version before sessions begin.
5. Conduct sessions
Before research sessions
As Perigean schedules sessions
- Assign and confirm notetaker(s)
This should include a member of your team but could include others. Ask your notetaker(s) to review guidance on taking notes.
- Prepare for note-taking
Create a “session-notes” folder within your research study Github folder and store each session as a .md document. Copy the conversation guide over to your .md file before each session so that you have a structured way to capture your notes.
- Invite observers
Invite your team and stakeholders to sessions. Limit your observers to 3-4 per session so participants aren’t overwhelmed. You should empower participants by allowing them to decide if they are comfortable having observers. Ask observers to review information about observer guidelines and attendance etiquette.
- Prepare as moderator
Be familiar with the conversation guide, study implement(s), and prototype(s). Review tips on moderating, guidelines for recording sessions,and guidance on participant and researcher safety and exit strategies.
During research sessions
- Start a thread in #feedback-backchannel (ex: Dependency Verification Usability P1 starts in 10 minutes :thread: )
- Type “observer instructions” in the channel at the beginning of your day of sessions to give them tips
- Make sure you mute all notifications, and close unnecessary applications
- Make sure all attendees (except participants) are muted
- Have your conversation guide somewhere accessible that the participant cannot see
- Make sure to hit record after permission is granted!
- Keep handy the guides for using Zoom in case the participant needs help during the session
- Refer to tips on moderating and responding to emergencies
- At the end of your session, take a look at the #feedback-backchannel and see if anyone brought up questions that you can address with your participant
- You may choose to take quick notes, but your primary duty as moderator is to run a smooth session. That’s why you have a notetaker!
After research sessions
- Check-in with notetakers and observers to capture their top 3 observations. Tip: ask them to put these in the session #feedback-backchannel thread, or a Mural you created for this!
- Have a daily or weekly debrief with your team to help digest sessions
- Summarize what you’ve heard by writing up key findings/themes. You may use the Topline Summary template.
- Share this summary with the team and observers to see if they feel anything was missing based on their observations
- Download any recordings you need, but only keep them for as long as necessary to complete synthesis. Then securely dispose of them. Review and ensure you understand the guidelines on recording user research
Do not upload any original session recordings to the cloud (Github, Dropbox, Google Docs, etc.) or any public or private server. The risk of accidentally disclosing PII outweighs any benefit provided by sharing original session recordings.
6. Synthesize data
Use whatever method and tool that works for you and your team. The key things you’ll want to get out of your research in order to write up your research findings are:
- 5-10 top findings/themes along with supporting quotes
- Determine whether you proved your hypotheses true or false
- Come up with recommendations based on your findings, including supporting evidence.
7. Share your findings with colleagues
- Write up your research findings
Use the research findings template to document your findings. This should be stored in your GitHub research folder. If you’d like to include a video clip of a participant, make sure to scrub all PII and PHI. Remember VFS work is about the digital experience, so that should be the focus of your findings since that’s what we can act on.
- Update the research repo board
Follow these guidelines for what to do when your research is complete.
- Share your findings with other colleagues
Share your research with your team and stakeholders and others. If the template above doesn’t fit well for some audiences, you may choose to use this VA presentation template.
- Share with the broader design practice
Reach out to Naomi Marcussen on Slack to sign up to share your findings with others at the weekly Design + Content + Research meeting. Use this template to create your presentation.
- Contribute to the Design System
Does your research have design system implications? Find out how to contribute to the design system.