This checklist can help guide you through all the tasks associated with conducting research on VFS projects.
1. Planning for research
3 - 4 weeks before research begins
- Make a research folder
Create a research folder in your product's folder on GitHub (Add file > create new file > product/research/README.md). All project-related research & elements should live in this folder, including the following: research plan, conversation guide, outlines for tree test, card sort, notes, synthesis documentation, research findings, and any readout presentations
See examples: Authenticated Experience, Health record
- Create a research plan
Create a markdown file named research-plan.md and save it in that folder (i.e. product-name/research/research-plan.md). Copy and paste from the research plan template into your new file. This will become the doc you use to plan your study, and also the documentation that project teams, research, UX lead, leadership, et al, can use to find your research in the future. Add anything you need to add to make this document work for you. If your team creates research issues and/or epics, you can cross-reference your xxxresearchplan.md file in that as well. Discuss your research plan with your team.
TEMPLATE: research plan
- Prepare your prototype or mock-up
Have the thing you’re testing ready and reviewed by the Collaboration Cycle a week (minimum) before you start research. They will make sure it meets the VA.gov quality standards.
- Create a conversation guide
Create a markdown file named conversation-guide.md and save it in that folder as well (i.e. product-name/research/conversation-guide.md). Copy and paste from the conversation guide template into your new file. Work with your team as your flesh this out.
TEMPLATE: conversation guide
2. Research review
1 - 2 weeks before research begins
- Add your study to the repo board
Learn how to add new research for approval to the repo board.
- Submit your research for review
Learn how to get your research approved to kickoff participant recruitment.
- You may also choose to consult on the accessibility of your planned research
While not required, you may reach out to VSP Accessibility team, #vagov-accessibility on Slack, for advice on effective and ethical research design for disabilities and assistive technology.
3. Participant recruitment
- Update the repo board
Once your research is approved, move your research card to the current column. If you are using Perigean, they will start recruiting for your study at this point.
4. Run a pilot session
Anytime between when your research is submitted and the start of research
- Identify your pilot participant
Identify who will participate in your pilot session. We recommend recruiting a developer from your team to get them involved early in the process.
- Set up the pilot session
Set up a meeting for your pilot. Include the guidelines about participating in a pilot with the invite so folks know what to expect. Make sure Perigean is aware so they can provide a Zoom account to practice with.
- Run the pilot session
Run pilot session and make any adjustments to your conversation guide that are needed
5. Conduct sessions
Before remote usability sessions
As Perigean schedules sessions
- Assign and confirm notetaker(s)
This should include a member of your team, but could include others. Ask your notetaker(s) to review guidance on taking notes.
- Prepare for note-taking
Create a “session-notes” folder within your research study Github folder and store each session as a .md document. Copy the conversation guide over to your .md file before each session so that you have a structured way to capture your notes
- Invite observers
Invite your team and stakeholders to the sessions. Limit your observers to 3-4 per session so participants aren’t overwhelmed. You should empower participants by allowing them to decide if they are comfortable having observers. Ask observers to review information about observer guidelines and attendance etiquette.
- Prepare as moderator
Be familiar with the conversation guide, study implement(s), and prototype(s). Review tips on moderating, guidelines for recording sessions,and the guidance on participant and researcher safety and exit strategies.
During research sessions
- Start a thread in #feedback-backchannel (ex: Dependency Verification Usability P1 starts in 10 minutes :thread: )
- Type “observer instructions” in the channel at the beginning of your day of sessions to give them tips
- Make sure you mute all notifications, and close unnecessary applications
- Make sure all attendees (except participant) are muted
- Have your conversation guide somewhere accessible that the participant cannot see
- Make sure to hit record after permission is granted!
- Keep handy the guides for using Zoom in case the participant needs help during the session
- Refer to tips on moderating and responding to emergencies
- At the end of your session, take a look at the #feedback-backchannel and see if anyone brought up questions that you can address with your participant
- You may choose to take quick notes, but your primary duty as moderator is to run a smooth session. That’s why you have a notetaker!
After the sessions
- Make sure to have a daily or weekly debrief with your team to capture their top 3 - 5 observations
- Download any recordings you need, but only keep for as long as necessary to complete synthesis. Then securely dispose of them.
- Summarize what you’ve heard by writing up key findings/themes
- Share this summary with the team and those that observed the sessions. Ask if they feel anything was missing based on their observations.
Do not upload any original session recordings to the cloud (Github, Dropbox, Google Docs, etc.) or any public or private server. The risk of accidentally disclosing PII outweighs any benefit provided by sharing original session recordings.
6. Synthesize data
Use whatever method and tool that works for you and your team. The key things you’ll want to get out of your research in order to write up your research findings are:
- 5-10 top findings/themes along with supporting quotes
- Determine whether you proved your hypotheses true or false.
- Recommendations based on your findings along with supporting evidence.
7. Share your findings with colleagues
- Write up your research findings
Use the research findings template to document your findings. This should be stored in your GitHub research folder. Remember our work is about the digital experience, so the focus of your findings should be items that we can act on.
- Update the research repo board
Follow these guidelines for what to do when your research is complete.
- Share your findings with colleagues
Create presentations as needed to share with your team and stakeholders. You could use the Monday meeting research presentation template or this generic VA presentation template to get started.
- Share with the broader design practice
Reach out to Naomi Marcussen on Slack to sign up to share your findings with other designers, content, and research folks at the weekly Design + Content + Research meeting. Use the Monday meeting research presentation template when presenting to this group.
- Contribute to the design system
Does your research have design system implications? Find out how to contribute to the design system.