Planning unmoderated studies

Last Updated: May 5, 2025

Unmoderated research usually involves participants completing simple tasks (such as card sorting or tree testing) or answering questions (such as a survey). This can help gather feedback from more people more quickly, increasing a study's statistical significance. This article walks you through how to plan for unmoderated research and things to keep in mind.

Following the Paperwork Reduction Act (PRA) in your research

PRA requirements have been clarified to say that usability tests like card sort, tree test, and surveys do not need PRA clearance. Usability research is exempt from PRA because it uses direct observation (e.g., usability testing, interviews, co-design) and asks structured, semi-structured, or open-ended questions to improve a paper or digital form.

What does this mean for unmoderated research?

This means that:

• there are no PRA limits on the number of participants you recruit through Perigean

• you don’t have to split your study into cohorts of 10 or less

• you don’t have to include separate links in your research plan

Research tools and participant privacy

Researching for VA requires you to use tools or services under a paid license. Free tools collect user data. Should a research participant share PII or PHI while using a free service, it puts our research practice at risk.

An Optimal Workshop account is available for anyone working on VA.gov to use. You can use any available tool, including surveys, card sorting, or tree testing.

Learn how to access the VA.gov team Optimal Workshop account.

Setting up an unmoderated study

1. Read through the PRA primer documentation.

2. Create a research plan.

3. Create the study implement (survey, card sort or tree test) in Optimal Workshop.

4. Ask for the unique identifier that Perigean gives to participants at the beginning of your study.

   1. To add a prompt in the beginning, navigate to the Study flow tab, then the Anonymous box in the Participant Identifier section in the left column. Then, select Other from the Format dropdown on the right side of the page.

      

   2. Enter a prompt in the Question text field, such as “Please enter the unique code Perigean provided you.”

      

4. Then, ask for the unique identifier again at the end of your study.

   1. Navigate to the Post-study questions on the left and click on the + button. Then, select Single-line text in the right panel.

b. Enter a prompt in the Single-line text box, such as “Please enter the unique code Perigean provided you to receive compensation.”

5. Add the study link to your research plan.

6. Pilot the study, including the instructions and activity—especially make sure the participant IDs come through as expected.

Recruiting for unmoderated studies

1. First, submit your research proposal to the Research Review touchpoint

2. Once your research is approved and ready, Perigean can begin recruitment

   1. Perigean will send out their recruitment email to veterans to gauge interest

   2. Veterans respond to the email

   3. Perigean sends veterans an instructional email, which includes the cohort study link and a unique participant code

3. Veteran participant completes the study and provides the code in the questionnaire

4. Researcher logs into Optimal Workshop and sends codes to Perigean at the end of each day to gauge participation

   1. It’s important to only send codes to Perigean for participants who’ve substantially participated

   2. If a participant abandoned the task, entered their code but didn’t answer any questions, ask Perigean to follow up with them

5. Perigean continues recruitment as needed

6. Let Perigean know when you’ve closed the study

Tips for conducting unmoderated studies

• Plan ahead to optimize time

  • Leave the sort open for 14 days if possible (one sprint) and use the time to work on other initiatives while the study is open

• Recruit for double the number of participants you want

  • Abandonment is much higher for unmoderated research as compared to moderated studies

• Communicate regularly with Perigean to measure participation and continue recruitment as needed

• It’s still important to pilot-test unmoderated studies!

  • Use a fake participant code and make sure all parts of the study work correctly

• Use the Maximum Variation Sampling (MVS) method to make sure your study is inclusive

• Note that some types of studies in Optimal Workshop are not accessible to users with screen readers (e.g. card sort, tree test, and first click testing)

• Write thorough instructions for the study

  • For example, many participants have said OptimalSort instructions were difficult to understand

  • Consider a hybrid sort, if applicable, to give participants a better understanding of what to do

• When conducting an unmoderated card sort, don’t create more than 30 cards

  • Sorting that many cards requires a high cognitive load and a lot of time, meaning increased abandonment of the exercise

• Do not require participants to go to more than one link to complete the activity

  • This study design decreases participation, increases abandonment

Help and feedback

• Suggest content changes to this page.

• Submit new Platform Website content.

• Get help from the Platform Support Team in Slack.

• Submit a feature idea to the Platform.