Skip to content

Write the job scorecard before the first interview

In short

A job scorecard fixes what the role needs before anyone is interviewed: outcomes, weighted competencies and which of them are must-haves. In Tracy AI Interviewer you set the skills Tracy covers for the role, and every applicant is interviewed on them, with a recording and an assessment per question behind each result. Your team applies the weights and decides.

What a job scorecard does well

A hiring manager writes a job description, the team posts it, and three weeks later five interviewers judge candidates on five different things. A job scorecard is written before all of that. It says what the job must achieve, which skills it takes and how much each one weighs, so the posting, the interview and the decision all measure the same thing. Copy the job scorecard template below and see it filled in for a fictional platform engineer.

A scorecard only governs the candidates who were interviewed on its skills. If half the pool is never interviewed, half the pool is never measured. Tracy AI Interviewer closes that gap, and the section after the example shows how.

What is a job scorecard?

A job scorecard is a short document, written before hiring starts, that says what a good hire looks like: the outcomes the job must deliver and the competencies it takes, with weights. It is for the hiring team, and it keeps every interviewer judging the same job.

Three documents get confused, and they are written in this order:

  • The job scorecard comes first.
  • The job description tells applicants about the job. It is built from the scorecard.
  • The interview scorecard is the form an interviewer fills in for each candidate. Its criteria are the scorecard's competencies.

Write the job scorecard first and the others stay consistent. Skip it and each interviewer brings their own idea of the job.

The job scorecard template

Copy the tables into a document or a sheet. Fill in Table 2 before you write the job description, and agree the weights in Table 3 with the hiring manager before the first interview.

Table 1: the role

FieldEntry
Role
Hiring manager
Team and level
Why this role exists, in one sentence

Table 2: outcomes

Outcome in the first 12 monthsHow we will know
Outcome 1
Outcome 2
Outcome 3

Table 3: competencies

CompetencyWhat good looks likeWeightMust-have?How it is assessed
Competency 130
Competency 225
Competency 320
Competency 415
Competency 510

Scroll the table sideways to see all 5 columns.

Table 4: hiring plan

StageWhat it checksWho runs it
Screen
Interview
Work sample or final

Use three outcomes and four or five competencies, with weights that add up to 100. Mark the must-haves, because those are pass or fail and the weights only rank candidates who clear them.

Three numbered documents in order: job scorecard, job description, interview scorecard.
The job scorecard comes first and the other two documents are built from it. Illustration with sample content.

Three documents, written in one order: scorecard, job description, interview scorecard.

From the scorecard to a job description

The scorecard turns into a job description in five parts. Copy this skeleton into a Word or Google document.

  • Title and mission: the role name and the sentence from Table 1.
  • What you will deliver: the three outcomes from Table 2, in plain words.
  • What we are looking for: the must-haves from Table 3, five at most, with the weights left out.
  • How we hire: the stages in Table 4, so applicants know what is coming.
  • Practical details: location, hours and the pay range where you publish one.

A description built this way stays short, and every line on it can be tested at interview.

A filled example: platform engineer

A fictional platform engineer role. The content is written for this page.

Table 5: filled example, outcomes

Outcome in the first 12 monthsHow we will know
Deploys reach production in under 30 minutes.Median deploy time from the pipeline dashboard.
On-call pages drop by a third.Pages per week, compared with the quarter before.
Two teams run on the shared service template.Number of teams using it.

Table 6: filled example, competencies

CompetencyWhat good looks likeWeightMust-have?How it is assessed
Infrastructure as codeBuilds and reviews environments from code and explains the trade-offs.30YesInterview
Debugging productionFinds a cause from logs and metrics and says how the fix was checked.25YesInterview
Automation and pipelinesDesigns a deploy pipeline with rollbacks.20NoWork sample
Working with developersMoves a team to a shared template without a mandate.15NoInterview
Writing and documentationWrites a runbook another engineer can follow.10NoWork sample

Scroll the table sideways to see all 5 columns.

Two competencies are must-haves, so a candidate who does not show infrastructure as code or production debugging stops there, whatever they score on the rest. The weights then rank the candidates who clear both.

What the filled scorecard shows that blank fields do not

The weights in Table 6 add up to 100, and the two must-haves, infrastructure as code (30) and debugging production (25), carry 55 of them. Weights alone would let a candidate weak on both still collect 45 points from the other three. The must-have column stops that: if a candidate does not show either one, your reviewer stops there, whatever they show on the rest, and the weights rank only those who clear both.

Interviewed on the same competencies, every applicant arrives with comparable evidence for the reviewer to weigh.

Using one scorecard when you scale up

Once the scorecard is written, the same scorecard serves every hire for the role, and a role family can share most of it. Keep the competencies and change the weights. A platform engineer and a site reliability engineer might share four of five competencies and weigh them differently.

  • Version it. Date the scorecard and note what changed, so earlier hires can be judged against the one in force then.
  • Do not change weights mid-search. Changing them after you have met candidates defeats the point.
  • Review after the first hires. Compare what the scorecard asked for with what the new hires needed in their first months, and adjust the next version.
  • Reuse the interview, not only the document. If every applicant gets the same interview, the scorecard scales with volume.

What Tracy adds to the job scorecard template

Three things change when Tracy does the interviewing.

01 Set the skills Tracy covers

The competencies in Table 3 are the place to start. Describe the role and Tracy drafts the questions, and you edit them or bring your own.

02 Reach every applicant

Each applicant starts on click, at any hour, with no scheduling, so the scorecard covers everyone and not only the shortlist.

03 Keep the evidence

The result holds what each person said to each question, with a recording of both sides and a timestamped transcript.

Tracy does not weigh, rate or rank. The weights in Table 3 are applied by your reviewer, and a person decides.

To see what a result holds per question, read what a Tracy result shows for each question. The interview guide template and the candidate evaluation template pick up from there. To try it, set the competencies from Table 3 as the skills for one role and run three real interviews.

Four steps: write the scorecard, set the skills for the role in Tracy, applicants interview, the reviewer applies the weights.
Tracy interviews every applicant on the scorecard's skills, and the team applies its own weights. Illustration of the workflow.

Interviewing on the scorecard's skills turns its competencies into evidence your team can weigh.

Three lines: start on click, follow-up on vague answers, evidence with every result.
Tracy covers the skills you set for every applicant, and your team applies the weights. Illustration with sample content.

Every applicant can start on click, every vague answer gets a follow-up, every result comes with evidence.

Which path fits you

Common questions

What is a job scorecard?
A short document written before hiring that sets the outcomes a role must deliver and the competencies it takes, with a weight for each. Interviewers score against it so the team judges the same things.
What should a job scorecard include?
The role and its mission in one sentence, three outcomes for the first year, four or five weighted competencies with what good looks like, which are must-haves and how each is assessed.
How is a job scorecard different from a job description?
A job description speaks to applicants about the job. A job scorecard speaks to the hiring team about what a good hire looks like, with weights. Write the scorecard first and build the description from it.
Is there a job description template for Word?
Use the five-part skeleton above in Word or Google Docs: title and mission, what you will deliver, what we are looking for, how we hire and practical details.
How do you use a job scorecard when you are scaling up?
Keep one scorecard per role family, vary the weights per role, date each version and review it after the first hires. Interview every applicant on the same skills so the scorecard applies to all of them.

Run this template as a Tracy interview

Copy the template, set its skills for one role and let Tracy interview every applicant. The first 3 results are free.

Try it free (opens in a new tab) Book a demo (opens in a new tab)

Your first 3 results are free. No card needed.

Sources