Attendee etiquette

As an attendee (notetaker and/or observer):

  • You’ll be a silent viewer

    • mute yourself and turn your camera off when you join the virtual meeting

    • change your name to “note taker” or “observer” instead of your name or remove your last name from the list

To communicate with the moderator and other attendees, use #feedback-backchannel:

  • Don’t expect the moderator to interact with you. It can be really distracting for the moderator to interact with the channel while moderating.

  • Thread all conversations per session.

  • Two basic things happen on the feedback channel:

    • moderator feedback

    • observers talking to each other

  • The moderator should only be expected to look at the end when opening the session up for questions to observers. Don’t expect the moderator to track the channel while moderating the session

  • Before each session ping feedback rules before each session

  • Notetaker and moderator should be partners in this session.

    • Notetaker should compile questions from the channel and give them to the moderator. If not in the same room, notetaker should direct message the moderator with the list of channel questions.

Role: Notetaker

For every session, there should always be a designated notetaker. If you are the designated notetaker, your responsibility is to take (as near to as possible) verbatim notes.

Notetaking guidelines

  • Taking verbatim notes is literally transcribing what the participant and moderator said and did.

  • It doesn’t have to be a live, full transcript, but it should be as close to exact as possible. After taking notes, go back and fill in areas that were not captured exactly during the session.

  • The note taker should be taking notes only. Avoid interpretation and save that for mini-debriefs / session reports.

  • In order to help the moderator identify where personally identifiable information is spoken or displayed on screen and later remove it from the recording, please make a stamp of "PII" in each spot where it occurs in the transcript.

  • “Type really fast and be kind to yourself.” - Sophia. Don't correct yourself while typing, don’t worry about typos. You can go back and fix this later, but try to be as accurate as possible. Time stamp areas you get lost and go back to the recording to fill in those details.

Notes template

  • If you have a template that works for you, share it!

  • Consideration: Reframer integration.

  • Regardless of how you take notes, be sure to capture: P#, date, time, test name

Role: Observer (internal team members)

An observer is someone who attends a session but is not the designated notetaker. Observers should be reminded before sitting in on sessions that they will be asked to provide 3-5 takeaways (see below) for each session to the moderator. Observers may, but are not required to, take verbatim notes.

Observer guidelines

  • Top 3 takeaways: Write down the top three things we should not forget and the top three surprises after each session. The moderator will solicit your feedback after each session.

  • Do not jump to solutions or interpret user behavior too much. Keep your notes focused on what the user said and did. Exact quotes and observations should be captured, not your interpretation of what they said. Remember, this is about the user, not you. Their reality is what we are here to understand.

    • Example Observation (Good): “The participant typed ‘science fiction’ in the search box.”

    • Example Inference (Bad): “The participant doesn’t like to use the navigation.”

    • Example Opinion (Worse): “The navigation is not noticeable.”

    • Send notes to researcher afterward so s/he can synthesize

  • Something that seems unimportant could surface as a pattern after a couple of interviews. So, capture everything in the raw notes.

  • Remember, the patterns across participants are what is important. This is why we brainstorm solutions later. Try to stay focused on notetaking. One blue sky daydream can result in 5 minutes of missed notes.

Role: Stakeholders

  • Stakeholders (external to DSVA, not assigned as a note taker) are not required to take verbatim notes, but are encouraged to do so if they would like to.

  • Stakeholders should be reminded before sitting in on sessions that they will be asked to provide 3-5 takeaways for each session to the moderator. Remind them that this is one data point of many that will be captured throughout the process.

Sample email to stakeholders

Subject: Usability Testing for X Product

Hello,

Thank you for your interest in attending a usability testing session for x product.

About usability testing: Usability testing is a method used to see how easy to use something is by testing it with real users. Users will be asked to complete tasks to see where they encounter problems and experience confusion. Usability testing will help us identify how usable or intuitive our product is. The important thing to remember about usability testing is that we aren't testing the user, we are testing our product!

You can find more information on usability testing and why we only need 5 test users here: https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ .

As an attendee:

- You will be a silent viewer (please remember to mute yourself when you join the virtual meeting)

- Take observation notes (focus on what the participant says and does, don't jump to solutions or interpret user behavior)

- Note the top three things you found most interesting or insightful or confusing

- If you have questions/comments for the researcher during the sessions, you can send the researcher any notes you took

The researcher will review the notes and top takeaways from all of the sessions to look for certain patterns. If patterns are identified, design changes will be discussed.

You have been invited to attend the following Veteran feedback session on x date. The goal of the session is x. This session will start at x time and should last about 45 minutes. Each session involves one Veteran and should include no more than 6 team members.

Thanks,