Participating in research as either a notetaker or observer can be invaluable for understanding user needs and product requirements. Before you take part, review the guidelines for how to participate in research as an observer or notetaker.

Joining introductions

  • You may change your video conference display name to be just your first or preferred name

    • We encourage you not to change your name to “observer”

  • Sign into the session with your microphone muted

  • If you aren’t admitted directly

    • Wait in the Zoom waiting room while the moderator begins the session

    • If the participant approves your presence, as in the majority of cases, the moderator will soon admit observers into the session

    • If there are issues, the moderator should be in contact via slack

  • At the moderator’s prompt, unmute your camera and microphone and briefly introduce yourself with

    • Your first or preferred name

    • Your role with VA.gov

  • When that’s done

    • Mute your microphone

    • Turn off your camera

    • Confine all messaging to the #feedback-backchannel thread for the session

      • Don’t message within the video conference app

      • Don’t expect the moderator to interact with you

How to be a good research observer

As an observer, you will play an important, offstage role

  • Create a safe, neutral, and distraction-free atmosphere for the moderator and participant

  • Pay close attention to what happens during the session

  • Join a debrief session and discuss what you observed

  • Prepare your top 3 takeaways

    • these are the top three items we should not forget or surprises in each session

    • write these in the #feedback-backchannel thread for the session or in another location the moderator designates

At this time, do not jump to solutions or interpret user behavior too much. Keep focused on what the user said and did. Exact quotes and observations should be captured, we’ll brainstorm solutions later. Right now, stay focused on the present session and understanding the participant’s reality.

  • Example Observation (Good): “The participant typed ‘science fiction’ in the search box.”

  • Example Inference (Bad): “The participant doesn’t like to use the navigation.”

  • Example Opinion (Worse): “The navigation is not noticeable.”

How to be a good notetaker

For every session, there should always be a designated notetaker. If you are the designated notetaker, your responsibility is to take (as near to as possible) verbatim notes.

Note-taking guidelines

  • Taking verbatim notes is literally transcribing what the participant and moderator said and did

  • It doesn’t have to be a live, full transcript, but it should be as close to exact as possible. After taking notes, go back and fill in areas that were not captured exactly during the session

  • The note taker should be taking notes only. Avoid interpretation and save that for debriefs and session reports

  • Something that seems unimportant could surface as a pattern after a couple of interviews. So, capture everything in the raw notes

  • In order to help the moderator identify where personally identifiable information is spoken or displayed on screen and later remove it from the recording, please make a stamp of "PII" in each spot where it occurs in the transcript

  • “Type really fast and be kind to yourself.” - Sophia. Don't correct yourself while typing, don’t worry about typos. You can go back and fix this later, but try to be as accurate as possible. Time stamp areas you get lost and go back to the recording to fill in those details

  • Keep all communications with in the #feedback-backchannel thread for the session

    • Don’t expect the moderator to track the channel while moderating the session

    • The moderator should only be expected to read this at the end when opening up to questions from observers

    • Notetaker and moderator should be partners in this session

      • Notetaker should compile questions from the channel for the moderator

Note-taking template

  • If you have a template that works for you, share it!

  • Consideration: Reframer integration

  • Regardless of how you take notes, be sure to capture: P#, date, time, test name

Role: Stakeholders

  • Stakeholders (external to DSVA, not assigned as a note taker) are not required to take verbatim notes, but are encouraged to do so if they would like to.

  • Stakeholders should be reminded before sitting in on sessions that they will be asked to provide 3-5 takeaways for each session to the moderator. Remind them that this is one data point of many that will be captured throughout the process.

Sample email to stakeholders

Subject: Usability Testing for X Product

Hello,

Thank you for your interest in attending a usability testing session for x product.

About usability testing: Usability testing is a method used to see how easy to use something is by testing it with real users. Users will be asked to complete tasks to see where they encounter problems and experience confusion. Usability testing will help us identify how usable or intuitive our product is. The important thing to remember about usability testing is that we aren't testing the user, we are testing our product!

You can find more information on usability testing and why we only need 5 test users here: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ .

As an attendee:

- You will be a silent viewer (please remember to mute yourself when you join the virtual meeting)

- Take observation notes (focus on what the participant says and does, don't jump to solutions or interpret user behavior)

- Note the top three things you found most interesting or insightful or confusing

- If you have questions/comments for the researcher during the sessions, you can send the researcher any notes you took

The researcher will review the notes and top takeaways from all of the sessions to look for certain patterns. If patterns are identified, design changes will be discussed.

You have been invited to attend the following Veteran feedback session on x date. The goal of the session is x. This session will start at x time and should last about 45 minutes. Each session involves one Veteran and should include no more than 6 team members.

Thanks,