Structured Interview Scoring: A Guide With Examples

In short

Tracy AI Interviewer scores every candidate on the skills and weights you set for the role.

Each result has a recording, a timestamped transcript and a written assessment per question, so the score can be checked.

Below: how structured interview scoring works, from anchors to disagreement and common errors.

Next: load your criteria into one role and read three reports.

We make Tracy AI Interviewer.

Two interviewers meet the same candidate. One writes "strong, 4". The other writes "fine, 3". In the debrief they argue about the candidate when they should be arguing about the scale, because neither wrote down what a 4 means.

Structured interview scoring fixes that by deciding, before anyone is interviewed, what each score looks like and what evidence earns it. This guide covers the scale, the anchors, when to score, how to combine scores and where scoring goes wrong. Tracy AI Interviewer scores each candidate on your skills and weights, and the method below works on any sheet. Our guide to screening candidates covers the step before it.

Why structured interview scoring beats a gut call

The reason to bother is predictive. A 2022 re-analysis in the Journal of Applied Psychology by Sackett and colleagues, covering studies of selection procedures, estimated a mean validity of .42 for structured interviews and .19 for unstructured ones. That is a correlation with job performance ratings, about twice as strong a link, not a hiring success rate. The structured figure rests on 105 studies and 7,864 people.

Structure helps because it narrows how much one person's mood decides the outcome. It does not make the criteria correct. Scoring only works if the criteria came from the job.

Write the anchors before the first interview

Use a 1 to 5 scale with three anchors written out: what a 1, a 3 and a 5 sound like for each criterion. Interviewers fill in the 2 and the 4. Five points give enough room to separate strong from good without pretending to be exact. Some teams use four points to remove the safe middle. Pick one and keep it.

Write anchors as behaviours in an answer, not adjectives. "Confident" is an adjective. "Names the steps in order and says what each ruled out" is a behaviour.

The table shows three written anchors for one criterion.

The anchors are an example. Write yours with the people who do the job, and test them on two past hires before you use them.

Score each answer, alone, with a line of evidence

Score each answer straight after you hear it, not at the end. By the end, the last answer and the first impression have taken over. Score alone before the debrief, so the loudest voice cannot set everyone's number. Then compare.

Write one line of evidence for every score: a quote, a number or a moment. "Strong" is not evidence. "Ruled out DNS first because the error was intermittent" is. A score with no evidence line is an opinion.

A fictional worked example, one answer, scored against the anchors above:

Question: Tell me about the last hard fault you diagnosed.

Candidate (fictional): A customer's sync failed twice a week. I checked logs first, then ruled out the network because it only failed at night, then found a scheduled job holding a lock.

Score: 4. Steps in order, one thing ruled out with a reason. Did not say what they would check next, which a 5 needs.

Each criterion gets a score against written anchors, and the weights produce one comparable number.

Combine scores with weights and floors, not averages

Weights set what matters. If troubleshooting counts for 35 percent and written clarity for 20, a candidate strong at the first and weak at the second can still lead. Set weights before the interviews and do not change them after you have met the candidates.

Do not average away a must-have. If a skill is a hard requirement, set a floor, for example no score under 3 on troubleshooting, and apply it before the weighted total.

When two interviewers differ by two points or more on a criterion, do not split the difference. Read the evidence lines and ask which one the anchor supports. If the anchor cannot settle it, the anchor was vague. Rewrite it.

Know the five errors that bend a score

These errors recur, and each has a cheap fix.

• Halo. One strong answer lifts every score. Score each criterion on its own evidence.
• Leniency. Everyone gets 4. Compare your average with a colleague's on the same candidates.
• Recency. The last answer counts most. Score as you go.
• Similarity. You rate people like you higher. Keep the evidence line and read it again.
• Drift. By candidate 40 your 3 is a different 3. Re-read the anchors every ten interviews.

The last one is why scoring a few interviews by hand is easy and scoring hundreds is not.

Score on your skills and weights with Tracy

You set the skills and weights for a role or role family. Tracy interviews each candidate on them and follows up on what each person says, and every result carries a written assessment per question with its reasoning, a recording and a timestamped transcript. Candidates are ranked against the others who took the interview, and strong ones are flagged to the recruiter.

A person keeps the last word. They can override a rejection, move a candidate forward or invite them to interview again, and the history stays on the record. Tracy does not publish a bias audit, so use the evidence to check a score, not to excuse it. See AI interview reports for what the reviewer reads.

The routine is the same at any volume: anchors first, evidence per score, a person decides.

Pilot your scale on one role

Pilot your scale on one role.

1. Write four criteria, weights and three anchors each.
2. Load the criteria and weights into Tracy and interview everyone who applies for a fortnight.
3. Have the hiring manager read ten reports and score each by hand against the same anchors.
4. Stop if the manager's scores and the assessments disagree on more than a few candidates and the anchors cannot explain why.
5. Measure the share of scores with an evidence line and the hours the manager spent.

Tracy costs $1 per credit, one credit per started 20 minutes, and the first 3 results are free with no seats. Try one on the Connect page, and see pricing.

Which path fits you:

• Many applicants for the same role: volume hiring.
• Technical roles: technical interviews.

Good looks like this: two reviewers can read the same evidence and land within a point of each other.

Common questions

Sources

• Sackett, Zhang, Berry and Lievens (2022), Revisiting meta-analytic estimates of validity in personnel selection, Journal of Applied Psychology 107(11), retrieved 9 Nov 2026.
• Tracy AI Interviewer pricing, retrieved 5 Nov 2026.