This checklist can help guide you through all of the tasks associated with conducting research on VFS projects.

1. Planning for research

3 - 4 weeks before research begins

  • Make a research folder within the Product folder
    Create a research folder (If you don’t currently have one) in your product's folder on GitHub (Add file > create new file > product/research/README.md). The “README.md” does not need to contain any information.
  • Make a research folder for your initiative-specific research
    Create an initiative-specific research folder (naming convention: YYYY-MM-Research-Name) within the research folder. Create a markdown file named research-plan.md and save it in the initiative-specific research folder you created. Then, copy and paste from the research plan template into your new file. Fill in your plan using the template; adjust as needed. Store your conversation guide and research plan in this folder. Remember to discuss your research plan with your team.
    TEMPLATE: research plan

Research Folder Example:

Product-One (folder) >
Research (folder) >

YYYY-MM-Research-Initiative-One (folder) >

Research Plan (document)

Conversation Guide (document)

Static Screenshots (Sketch file)

Research Findings (document)

YYYY-MM-Research-Initiative-Two >

YYYY-MM-Research-Initiative-Three >

*For additional examples: Combined Debt Portal Research Studies, Public Websites

  • Prepare your prototype or mock-up
    Have the thing you’re testing ready! The Collaboration Cycle should conduct a Midpoint Review a week (minimum) before you start your research. They make sure it meets VA.gov quality standards before putting it in front of veterans.
  • Create a conversation guide
    First, create a markdown file named conversation-guide.md and save it in your research folder (i.e. product-name/research/conversation-guide.md). Then, copy and paste from the conversation guide template into your new file, and adjust the template as needed. Work with your team to complete the conversation guide.
    TEMPLATE: conversation guide
  • Make your research inclusive and accessible
    Consider ways to make your research more inclusive and accessible. Some ideas are to set inclusivity requests for recruitment, like using the MVS sampling method; include mobile or have a separate study for mobile usability testing; and even if you won’t have a coded version, there are still assistive technology users you can recruit to test your design.

2. Research review

At least 10 days before research begins

  • Add your study to the repo board
    Create a new ticket using the research repo ticket template.
  • Submit your research for review
    See instructions on how to get your research approved and to kickoff participant recruitment.
  • You may also choose to consult on the accessibility of your planned research
    While not required, you may reach out to VA.gov accessibility specialists, #accessibility-help on Slack, for advice on effective and ethical research design for disabilities and assistive technology.

3. Participant recruitment

At least 7 days before research begins

  • Initiate your recruitment plan
    Once your research is approved, move your Research Repo card to the “Current” column. Then, if you are using Perigean for recruiting, Shane Strassberg will send the kickoff email signaling Perigean to begin recruiting.

4. Run a pilot session

Anytime before your first research session

  • Identify your pilot participant(s)
    Identify who will participate in your pilot session. We recommend recruiting a developer from your team to get them involved early in the process.
  • Set up the pilot session
    Set a meeting time with your pilot. Include the guidelines about participating in a pilot with the invite so folks know what to expect.
  • Run the pilot session
    Run pilot session and note where any adjustments could make future sessions run smoother.
  • Make any adjustments to your conversation guide
    Make sure note-taker(s) and moderator(s) have the most recent version before sessions begin.

5. Conduct sessions

Before research sessions

As Perigean schedules sessions

  • Assign and confirm notetaker(s)
    This should include a member of your team but could include others. Ask your notetaker(s) to review guidance on taking notes.
  • Prepare for note-taking
    Create a “session-notes” folder within your research study Github folder and store each session as a .md document. Copy the conversation guide over to your .md file before each session so that you have a structured way to capture your notes.
  • Invite observers
    Invite your team and stakeholders to sessions. Limit your observers to 3-4 per session so participants aren’t overwhelmed. You should empower participants by allowing them to decide if they are comfortable having observers. Ask observers to review information about observer guidelines and attendance etiquette.
  • Prepare as moderator
    Be familiar with the conversation guide, study implement(s), and prototype(s). Review tips on moderating, guidelines for recording sessions,and guidance on participant and researcher safety and exit strategies.

During research sessions

  • Start a thread in #feedback-backchannel (ex: Dependency Verification Usability P1 starts in 10 minutes :thread: )
  • Type “observer instructions” in the channel at the beginning of your day of sessions to give them tips
  • Make sure you mute all notifications, and close unnecessary applications
  • Make sure all attendees (except participants) are muted
  • Have your conversation guide somewhere accessible that the participant cannot see
  • Make sure to hit record after permission is granted!
  • Keep handy the guides for using Zoom in case the participant needs help during the session
  • Refer to tips on moderating and responding to emergencies
  • At the end of your session, take a look at the #feedback-backchannel and see if anyone brought up questions that you can address with your participant
  • You may choose to take quick notes, but your primary duty as moderator is to run a smooth session. That’s why you have a notetaker!

After research sessions

  • Check-in with notetakers and observers to capture their top 3 observations. Tip: ask them to put these in the session #feedback-backchannel thread, or a Mural you created for this!
  • Have a daily or weekly debrief with your team to help digest sessions
  • Summarize what you’ve heard by writing up key findings/themes. You may use the Topline Summary template.
  • Share this summary with the team and observers to see if they feel anything was missing based on their observations
  • Download any recordings you need, but only keep them for as long as necessary to complete synthesis. Then securely dispose of them. Review and ensure you understand the guidelines on recording user research
    Do not upload any original session recordings to the cloud (Github, Dropbox, Google Docs, etc.) or any public or private server. The risk of accidentally disclosing PII outweighs any benefit provided by sharing original session recordings.

6. Synthesize data

Use whatever method and tool that works for you and your team. The key things you’ll want to get out of your research in order to write up your research findings are:

  • 5-10 top findings/themes along with supporting quotes
  • Determine whether you proved your hypotheses true or false
  • Come up with recommendations based on your findings, including supporting evidence.

7. Share your findings with colleagues