This checklist can help guide you through all the tasks associated with conducting research on VFS projects.

1. Planning for research

3 - 4 weeks before research begins

  • Make a research folder
    Create a research folder in your product's folder on GitHub (Add file > create new file > product/research/README.md). All project-related research & elements should live in this folder, including the following: research plan, conversation guide, outlines for tree test, card sort, notes, synthesis documentation, research findings, and any readout presentations
    See examples: Authenticated Experience, Health record
  • Create a research plan
    Create a markdown file named research-plan.md and save it in that folder (i.e. product-name/research/research-plan.md). Copy and paste from the research plan template into your new file. This will become the doc you use to plan your study, and also the documentation that project teams, research, UX lead, leadership, et al, can use to find your research in the future. Add anything you need to add to make this document work for you. If your team creates research issues and/or epics, you can cross-reference your xxxresearchplan.md file in that as well. Discuss your research plan with your team.
    TEMPLATE: research plan
  • Prepare your prototype or mock-up
    Have the thing you’re testing ready and reviewed by the Collaboration Cycle a week (minimum) before you start research. They will make sure it meets the VA.gov quality standards.
  • Create a conversation guide
    Create a markdown file named conversation-guide.md and save it in that folder as well (i.e. product-name/research/conversation-guide.md). Copy and paste from the conversation guide template into your new file. Work with your team as your flesh this out.
    TEMPLATE: conversation guide

2. Research review

1 - 2 weeks before research begins

3. Participant recruitment

  • Update the repo board
    Once your research is approved, move your research card to the current column. If you are using Perigean, they will start recruiting for your study at this point.

4. Run a pilot session

Anytime between when your research is submitted and the start of research

  • Identify your pilot participant
    Identify who will participate in your pilot session. We recommend recruiting a developer from your team to get them involved early in the process.
  • Set up the pilot session
    Set up a meeting for your pilot. Include the guidelines about participating in a pilot with the invite so folks know what to expect. Make sure Perigean is aware so they can provide a Zoom account to practice with.
  • Run the pilot session
    Run pilot session and make any adjustments to your conversation guide that are needed

5. Conduct sessions

Before remote usability sessions

As Perigean schedules sessions

  • Assign and confirm notetaker(s)
    This should include a member of your team, but could include others. Ask your notetaker(s) to review guidance on taking notes.
  • Prepare for note-taking
    Create a “session-notes” folder within your research study Github folder and store each session as a .md document. Copy the conversation guide over to your .md file before each session so that you have a structured way to capture your notes
  • Invite observers
    Invite your team and stakeholders to the sessions. Limit your observers to 3-4 per session so participants aren’t overwhelmed. You should empower participants by allowing them to decide if they are comfortable having observers. Ask observers to review information about observer guidelines and attendance etiquette.
  • Prepare as moderator
    Be familiar with the conversation guide, study implement(s), and prototype(s). Review tips on moderating, guidelines for recording sessions,and the guidance on participant and researcher safety and exit strategies.

During research sessions

  • Start a thread in #feedback-backchannel (ex: Dependency Verification Usability P1 starts in 10 minutes :thread: )
  • Type “observer instructions” in the channel at the beginning of your day of sessions to give them tips
  • Make sure you mute all notifications, and close unnecessary applications
  • Make sure all attendees (except participant) are muted
  • Have your conversation guide somewhere accessible that the participant cannot see
  • Make sure to hit record after permission is granted!
  • Keep handy the guides for using Zoom in case the participant needs help during the session
  • Refer to tips on moderating and responding to emergencies
  • At the end of your session, take a look at the #feedback-backchannel and see if anyone brought up questions that you can address with your participant
  • You may choose to take quick notes, but your primary duty as moderator is to run a smooth session. That’s why you have a notetaker!

After the sessions

  • Make sure to have a daily or weekly debrief with your team to capture their top 3 - 5 observations
  • Download any recordings you need, but only keep for as long as necessary to complete synthesis. Then securely dispose of them.
  • Summarize what you’ve heard by writing up key findings/themes
  • Share this summary with the team and those that observed the sessions. Ask if they feel anything was missing based on their observations.

Do not upload any original session recordings to the cloud (Github, Dropbox, Google Docs, etc.) or any public or private server. The risk of accidentally disclosing PII outweighs any benefit provided by sharing original session recordings.

6. Synthesize data

Use whatever method and tool that works for you and your team. The key things you’ll want to get out of your research in order to write up your research findings are:

  • 5-10 top findings/themes along with supporting quotes
  • Determine whether you proved your hypotheses true or false.
  • Recommendations based on your findings along with supporting evidence.

7. Share your findings with colleagues