Participant ratings let your team evaluate individual participants after a research session on a star scale of 1 to 5 based on the quality of their feedback and participation in the study. Ratings make it easier to identify participants who are a strong fit for future research by helping you to:

• Spotlight the most thoughtful, reliable contributors

• Prioritize high-performers when multiple participants qualify

• Build stronger cohorts over time

• Share context across team members in Rally

What are participant ratings?

Participant ratings are a quick way to capture how effectively a participant was able to take part in a research session.

When you rate a participant, you can record signals related to:

• Attendance and preparedness

• Ability to follow study instructions

• Technical readiness

• Engagement and clarity during the session

Ratings can surface participants who contribute clear, impactful feedback. When used consistently, they help your team spot trends over time, prepare for future sessions with more context, and maintain continuity across roles and studies.

How to rate a participant

Rate a participant directly after a session, while it's still fresh:

1. Navigate to Interviews > Past for your study.

2. Click Rate & Confirm Attendance.

3. Confirm whether the session happened by selecting Yes or No show.

4. Select a star rating.

5. Optionally, select Add note to add context.

6. Select Submit to save.

If the session had a standby participant, you can confirm attendance, rate, and add a note for each person separately in the same window.

After you submit a rating

• The participant's average rating appears on their Person Profile, along with how many ratings it's based on and their show rate.

  • The average includes every rating the person has received, across all studies in your workspace.

  • Individual session ratings appear under the Notes tab.

• To edit a rating, select Edit Rating & Attendance, make your changes, and select Submit.

Use ratings in your workspace

Ratings are stored in the Participant Rating property, so you can use them anywhere you work with properties. For example:

• Views and filters: filter for a rating of 4 or higher to quickly find high-quality participants. Learn more in Views.

• Table columns: add Participant Rating as a column on people tables to quickly see a person's rating.

• Governance rules: set up an engagement rule that automatically sets participants with a rating of 2 or lower to Do Not Contact. Learn more in Governance.

What to consider when rating

When choosing a rating, focus on whether the participant was able to meet the expectations of the study as designed, given the setup and guidance provided. For consistency, your team should agree on what each rating level means. For example, 5 stars might mean you'd interview them again, they followed all instructions, and they gave clear feedback.

The goal of a rating system is less about judging a participant and more about helping researchers make informed decisions: reducing operational risk, improving study fit, and supporting fair inclusion decisions.

You can consider the following factors when establishing a rating system:

Attendance reliability

• Did the participant show up on time?

• Did they give advance notice for lateness or cancellation?

• Was the absence a no-show or a legitimate conflict?

Instruction follow-through

• Did they follow study instructions?

• Did they complete required tasks?

• Did they use required setup (camera on, screen share, prototype access, etc.)?

Technical readiness

• Was audio and video functional?

• Was their connection stable enough for the session?

• Could they troubleshoot basic issues or follow guidance?

Level of engagement

• Did they respond thoughtfully rather than with one-word answers?

• Did they stay reasonably focused on tasks?

• Did they participate for the full session?

Communication clarity

• Could they explain experiences and decisions?

• Did they ask clarifying questions when instructions were unclear?

Misrepresentation

• Did they provide inaccurate or contradictory background information?

• Did they fail a screener in ways that suggested gaming?

• Do they appear to participate in an unusually high number of studies?

• Are they providing scripted or repetitive answers?

Facilitation

Always consider where responsibility for the success of a session may fall, and factor the following elements as applicable:

• How clear were the instructions provided to the participant?

• Were expectations clearly set?

• Did any elements of facilitation prevent the participant from engaging fully in the session?

What should not factor into ratings

When rating, make sure to focus on session success and study fit, not on judging the participant as a person or agreeing with their opinions.

Participant ratings should never be based on:

• Agreement with opinions, design direction, or product feedback

• Helpfulness to a team’s narrative or goals

• Personality traits

• Demographic attributes

• Whether feedback was positive or negative

Including these factors introduces bias and undermines fair participation.

FAQs

Does Rally send a reminder to rate participants after a session?

Yes. The interview host will receive a notification in Rally after a session, reminding them to confirm attendance and rate the participant.

If you have Slack notifications enabled, RallyBot will also send you a message after the interview. Select Rate & add note to rate the participant and add a note without leaving Slack.

Who receives the notification to rate a participant?

Only the host of the interview receives the reminder. Guests and observers don't.

Can a participant receive more than one rating per interview?

No. Each interview can only have one rating per participant. To update a submitted rating, select Edit Rating & Attendance, make your edits, and select Submit.

Can I rate participants in surveys or unmoderated tests?

No. Ratings are tied to interview sessions, so they're only available for interview studies.