TLDR AI trainer screening breaks when applicants outnumber the reviewers who can talk to them. Tracy AI Interviewer interviews every applicant in their own field, follows up and waits for the answer.
She tests what a CV hides: judging a model's answer, writing criteria, explaining a score. A reviewer decides from a recording, transcript and assessment per question.
Roy Junyoung Jang, Director, Residency & Investment at Antler Korea, says Tracy "sometimes brings up exactly those questions I might have forgotten to ask."
Next: create an interview for one role.
We make Tracy AI Interviewer.
AI trainer screening breaks at the same point on every project. A project opens, applicants arrive faster than your reviewers can speak to them, and every CV says the right thing: a degree, a specialty, years in the field. None of that shows whether a person can say why a model's answer is wrong, write the criteria for scoring it, or explain a score so that a second grader reaches the same one.
Tracy AI Interviewer runs that conversation for every applicant. She asks questions in the applicant's own field, follows up on what they just said and waits for the end of the thought. Your reviewers then read a recording, a timestamped transcript and a written assessment for each question, and decide. For a question bank to copy, see our guide to AI trainer interview questions. For the wider process, see how we handle expert vetting.
What does AI trainer screening have to find out?
AI trainers read a model's answer and judge it. Depending on the project they write a hard prompt with the correct answer, write the criteria for grading, rate and rank several answers, or flag one that is wrong or unsafe. Four abilities carry most of that work, and a CV shows none of them.
| Ability | Ask | Follow up with | A strong answer |
|---|---|---|---|
| Judging a model's answer | A model's answer is correct but skips a step. How do you score it? | Two graders disagree on that mark. What in your rationale settles it? | Writes the missing step into the rationale and gives a rule another grader can apply. |
| Writing criteria | Write one grading criterion for an answer that explains a concept in your field. | Give me an answer that meets it and one that only looks as if it does. | The criterion can be checked with a yes or no, and the two examples differ on one point. |
| Explaining a score | You gave this answer 2 out of 5. Tell me why in two sentences. | What would move it to a 4? | Names the gap and what would change the score. A feeling is not a reason. |
| Knowing when to stop | When do you send a task back instead of scoring it? | Tell me about the last time that happened. | Names a real case, says what was unclear and what they did next. |
Give every applicant the same opening questions so you can compare them.
How does an AI trainer job description set up the interview?
An interview built from a vague AI trainer job description gets vague answers. The description should say what the person will do (write prompts, grade, rank, review), the field, the weekly hours, the quality bar and what experience counts.
Tracy drafts the questions from that brief, and you edit them or bring your own. Interview style runs from quick to deep and from technical to behavioural, and you set the skills the role must show and their weights. With CV check turned on, Tracy can also ask about what the CV claims. The product page lists each setting.
AI trainer test or interview: which should you use?
An AI trainer test scores the output of a task. It shows what someone produced on the day, not how they got there or whether they can explain it. An interview shows the reasoning. A common setup is to interview everyone who applies, then give a short calibration task to the people who pass and score it against your own answer key. For why a conversation finds more than a recorded form, read one-way versus two-way video interviews.
A worked example: one scoring question and its follow-up
This is an illustration with sample data, not a real interview. The candidate is fictional.
Tracy: A model's answer is correct but skips a step. How do you score it?
Candidate (fictional): I mark the missing step in my rationale and score it down one level.
Tracy: Two graders disagree on that mark. What in your rationale settles it?
Candidate: A rule. A skipped step that changes the reader's conclusion costs a level, and one that does not costs nothing. I would add an example to the rubric.
Tracy: Where would that rule fail?
A follow-up on the applicant's own answer shows how they reason about a score.
What do your reviewers read after the interview?
Each finished interview leaves a record a person can check.
- A recording and a timestamped transcript behind every result.
- A written assessment for each question, with its reasoning, scored on the skills and weights you set for the role.
- A ranking against the others who took the same interview, with the strongest flagged to you automatically.
- Integrity signals with their evidence, covered below.
- One record you can send on, with the resume match, interview score, strengths and gaps and the recording, shared by link or PDF with the lab team that needs it.
A reviewer can override a rejection or invite an applicant to interview again. See AI interview reports for how to read one.
The report ties each point to its question, the time it was said and the words behind it.
Can you screen hundreds of AI trainer applicants without a queue?
Each applicant opens a link in a browser, with no account and no scheduling, and starts when they click, at any hour. The interview is ready in under 10 seconds. A dropped applicant can rejoin within 10 minutes (customers can configure that), a reminder goes out after a day, and interviews that failed are chased automatically. Applicants are told on the join screen that the conversation is recorded and transcribed. See what applicants see on the participant experience page.
Tracy also captures tab changes, abnormal eye tracking, live face detection, delay before answers, fullscreen exits and external screens, and shows the evidence to a person. A signal is a prompt to look, never proof, and nothing is decided in the interview. Our guide to AI interview cheating detection covers what a reviewer should do with one.
What stays with your team
Tracy runs the first conversation. Three things stay with people.
- Credentials. Not stated for Tracy, so check a degree, licence or work history separately where the project requires it.
- A reviewer of your own. Have one of your graders read five or six transcripts and score them blind. If they disagree with the written assessments, change the questions before you invite the next hundred.
- The decision. Nothing is decided in the interview. A person decides from the evidence, and Tracy publishes no bias audit.
If you onboard a handful of trainers a month, a call with your own reviewer is cheaper. The case for Tracy starts when applicants outnumber reviewer hours; see AI interview versus human interview.
How to start: one role, one link
Start with one role.
- Write the job description for one role, with the task type, the field and the quality bar.
- Open the Connect page, describe the role and let Tracy draft the questions, or paste your own.
- Share the link with the first applicants. The first 3 results are free, with no card.
- Compare the reports with the people you would have approved from the CV alone.
After the free results, Tracy costs $1 per credit, and one credit covers each started 20 minutes of an interview that produced a result. There are no seats and no upfront cost, a session with no answers is never charged, and 100 interviews of 20 minutes cost $100. The pricing page has the current plan, and our guide to AI interview cost sets prices side by side.
Which path fits you:
- Subject experts or AI trainers to vet: expert vetting.
- Hundreds of applicants for one project: volume hiring.
- Engineers for the same projects: technical interviews.
- Programme or fellowship applications: applications.
Questions
How do you screen AI trainers?
What should an AI trainer job description include?
What is an AI trainer test?
Create an interview for this exact role. Send us one role and we will build the interview plan in days, with a recording and report you can check. Try it free · Book a demo
Sources
- Tracy AI Interviewer pricing, retrieved 5 Nov 2026.
- Tracy AI Interviewer Connect page, retrieved 5 Nov 2026.
- Tracy AI Interviewer product page, retrieved 5 Nov 2026.
- Tracy AI Interviewer home page, customer testimonials (Antler Korea), retrieved 9 Nov 2026.