Participating in research as either a notetaker or observer can be invaluable for understanding user needs and product requirements. Before you take part, review the guidelines for how to participate in research as an observer or notetaker.
You may change your video conference display name to be just your first or preferred name
We encourage you not to change your name to “observer”
Sign into the session with your microphone muted
If you aren’t admitted directly
Wait in the Zoom waiting room while the moderator begins the session
If the participant approves your presence, as in the majority of cases, the moderator will soon admit observers into the session
If there are issues, the moderator should be in contact via slack
At the moderator’s prompt, unmute your camera and microphone and briefly introduce yourself with
Your first or preferred name
Your role with VA.gov
When that’s done
Mute your microphone
Turn off your camera
Confine all messaging to the #feedback-backchannel thread for the session
Don’t message within the video conference app
Don’t expect the moderator to interact with you
How to be a good research observer
As an observer, you will play an important, offstage role
Create a safe, neutral, and distraction-free atmosphere for the moderator and participant
Pay close attention to what happens during the session
Join a debrief session and discuss what you observed
Prepare your top 3 takeaways
these are the top three items we should not forget or surprises in each session
write these in the #feedback-backchannel thread for the session or in another location the moderator designates
At this time, do not jump to solutions or interpret user behavior too much. Keep focused on what the user said and did. Exact quotes and observations should be captured, we’ll brainstorm solutions later. Right now, stay focused on the present session and understanding the participant’s reality.
Example Observation (Good): “The participant typed ‘science fiction’ in the search box.”
Example Inference (Bad): “The participant doesn’t like to use the navigation.”
Example Opinion (Worse): “The navigation is not noticeable.”
How to be a good notetaker
For every session, there should always be a designated notetaker. If you are the designated notetaker, your responsibility is to take (as near to as possible) verbatim notes.
Taking verbatim notes is literally transcribing what the participant and moderator said and did
It doesn’t have to be a live, full transcript, but it should be as close to exact as possible. After taking notes, go back and fill in areas that were not captured exactly during the session
The note taker should be taking notes only. Avoid interpretation and save that for debriefs and session reports
Something that seems unimportant could surface as a pattern after a couple of interviews. So, capture everything in the raw notes
In order to help the moderator identify where personally identifiable information is spoken or displayed on screen and later remove it from the recording, please make a stamp of "PII" in each spot where it occurs in the transcript
“Type really fast and be kind to yourself.” - Sophia. Don't correct yourself while typing, don’t worry about typos. You can go back and fix this later, but try to be as accurate as possible. Time stamp areas you get lost and go back to the recording to fill in those details
Keep all communications with in the #feedback-backchannel thread for the session
Don’t expect the moderator to track the channel while moderating the session
The moderator should only be expected to read this at the end when opening up to questions from observers
Notetaker and moderator should be partners in this session
Notetaker should compile questions from the channel for the moderator
If you have a template that works for you, share it!
Consideration: Reframer integration
Regardless of how you take notes, be sure to capture: P#, date, time, test name
Stakeholders (external to DSVA, not assigned as a note taker) are not required to take verbatim notes, but are encouraged to do so if they would like to.
Stakeholders should be reminded before sitting in on sessions that they will be asked to provide 3-5 takeaways for each session to the moderator. Remind them that this is one data point of many that will be captured throughout the process.
Sample email to stakeholders
Subject: Usability Testing for X Product
Thank you for your interest in attending a usability testing session for x product.
About usability testing: Usability testing is a method used to see how easy to use something is by testing it with real users. Users will be asked to complete tasks to see where they encounter problems and experience confusion. Usability testing will help us identify how usable or intuitive our product is. The important thing to remember about usability testing is that we aren't testing the user, we are testing our product!
You can find more information on usability testing and why we only need 5 test users here: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ .
As an attendee:
- You will be a silent viewer (please remember to mute yourself when you join the virtual meeting)
- Take observation notes (focus on what the participant says and does, don't jump to solutions or interpret user behavior)
- Note the top three things you found most interesting or insightful or confusing
- If you have questions/comments for the researcher during the sessions, you can send the researcher any notes you took
The researcher will review the notes and top takeaways from all of the sessions to look for certain patterns. If patterns are identified, design changes will be discussed.
You have been invited to attend the following Veteran feedback session on x date. The goal of the session is x. This session will start at x time and should last about 45 minutes. Each session involves one Veteran and should include no more than 6 team members.