TLDR Tracy AI Interviewer asks each legal expert questions in their own practice area, follows up on what they said and waits for the answer, so you see how they check a source and not just what they remember.
The 19 questions below test how an expert applies a rule, verifies authority, handles jurisdiction and dates, reads drafting closely, and stays inside the limits of the job.
Every result comes with a recording and a timestamped transcript, plus integrity signals and their evidence, for a person to review. Bar admission and other credentials are checked separately.
Three weeks into a legal-evaluation project, a reviewer finds that an approved expert cited a case that does not say what the expert said it says. The expert had a good CV and the right practice area. Nobody had asked how they check a source. Legal expert interview questions for AI training projects have to ask exactly that.
The job is to read a model's answer to a legal question, find what is wrong, unsupported or out of date, and explain why so a second reviewer would agree. Job seekers can use the questions to prepare, and should answer the follow-up too.
Three limits. The questions test legal reasoning for review work, are not legal advice and state no law of any place. And they do not confirm admission to practise: Tracy does not verify a licence or a credential, so check bar admission with the body that holds the record. For the generalist version of this job, see our guide to AI trainer interview questions. For the rest of the process, see expert vetting.
Legal review work depends on checking sources, so ask about the check
The failure that matters most has a name in the research. Stanford researchers who tested AI legal research tools on a pre-registered set of over 200 open-ended legal queries separated two kinds of error. A response can be incorrect, or it can be "misgrounded": the law is described correctly but the source it cites does not support the claim (source, 23 May 2024). A reviewer who only checks whether the statement of law is right will pass a misgrounded answer, which is why the second group below is the longest.
The questions test applying a rule to facts, authority and citations, jurisdiction and time, reading and drafting, the limits of the job, and working method. A lawyer on your team should adapt them to the project's practice areas.
Nineteen legal expert interview questions, grouped by what they test
Choose one or two from each group. Ask the follow-up after the first answer, decide what a strong answer contains before the first expert joins, and give every expert the same opening questions.
Group 1: applying a rule to facts
| Ask | Follow up with | Listen for |
|---|---|---|
| 1. A tenant stops paying rent after the landlord ignores a repair request. Pick a jurisdiction and talk me through how you would think about it. | Which single fact, if it changed, would change your answer? | Names the jurisdiction first. Separates facts given from facts missing. Says what they would confirm in the source. |
| 2. Describe a fact pattern in your practice area where the obvious answer is wrong. | What did a less experienced colleague get wrong about it? | A specific trap. Says what the colleague skipped, not just that they erred. |
| 3. How do you explain to a non-lawyer why "it depends" is the honest answer? | Where does "it depends" stop being an answer? | Names what it depends on. Knows when a hedge avoids the question. |
Group 2: authority and citations
| Ask | Follow up with | Listen for |
|---|---|---|
| 4. A model cites a case to support its answer. Walk me through verifying it. | The case exists but does not say that. What do you do? | Reads the passage, not the headnote. Treats a real case with an unsupported proposition as an error. |
| 5. How do you tell binding from persuasive authority in your jurisdiction? | Tell me about a time that distinction changed an answer. | Explains it in local terms. Gives a case where a model leaned on the wrong kind. |
| 6. How do you check that a statute or regulation is the current version? | What do you do when you cannot reach the source? | Names where they look and checks the effective date. Marks the answer unverified, not guessed. |
| 7. What makes you distrust a citation at first glance? | Which sign has misled you? | Concrete tells: a court that does not hear that case type, a year that does not fit, a quote that reads too neatly. Admits one that fooled them. |
Group 3: jurisdiction and time
| Ask | Follow up with | Listen for |
|---|---|---|
| 8. A question never says where the dispute is. What do you do? | When is it fine to answer in general terms? | Asks, or states an assumption aloud. Answers generally only when the point holds everywhere. |
| 9. Tell me about an area of law that changed recently in your practice. How would an earlier-trained model go wrong? | How would you score an answer that was right last year? | A real change and its date. Marks the answer out of date, with a note for the next reviewer. |
| 10. A question assumes a rule that does not exist in your jurisdiction. How do you respond? | Which scores higher, a model that corrects the premise or one that plays along? | Corrects the premise first. Scores pushback higher and can explain the risk of playing along. |
Group 4: reading and drafting
| Ask | Follow up with | Listen for |
|---|---|---|
| 11. Take this phrase: "the supplier will use reasonable efforts to deliver on time." How would you read it? | How would you redraft it for the supplier, then for the buyer? | Reads it in context and says what each side might argue. Offers two drafts for two clients. |
| 12. Summarise a contract clause in two sentences for a client who is not a lawyer. | What did you leave out, and why? | Plain words that keep the obligation and the trigger. Names the omission and why it was safe. |
| 13. You find two clauses in the same agreement that conflict. What do you do? | What if both parties already acted on one of them? | Looks for an order-of-precedence clause first. Raises conduct as a factor. |
Group 5: confidentiality and limits
| Ask | Follow up with | Listen for |
|---|---|---|
| 14. What would you never share from a client matter in a task, a note or a chat with us? | How do you build a realistic scenario, then? | Builds an invented scenario from the shape of a matter. Removes names and details unasked. |
| 15. Where is the line between legal information and legal advice, and why does it matter when you judge a model? | Give me an answer you would mark down for crossing it. | A practical line, such as telling one person what to do about their own dispute. A concrete example. |
| 16. When should a model decline to answer or refer the person to a lawyer? | When would that be unhelpful? | Sees both failures. Refers when facts or stakes call for it, answers when it can help. |
Group 6: working method
| Ask | Follow up with | Listen for |
|---|---|---|
| 17. You have ten minutes to grade a long answer. What do you do first? | What do you skip, and what makes you spend longer? | A fixed order: claim, authority, jurisdiction, date. Says which findings stop the clock. |
| 18. You and another reviewer disagree about whether an answer is correct. How do you resolve it? | When do you hold your position? | Goes to the source together. Holds on text they can point to, moves on taste. |
| 19. How many hours a week, in which time zone and for how long can you commit? What gets in the way? | What do you do when a client deadline lands in your project week? | Specific numbers and a plan for the bad week. Says early if it will not fit. |
For a short call, take 1, 4, 6, 10, 15 and 18, and ask question 19 of everyone. For other practice areas, keep the groups and change the facts: an employment lawyer and a tax specialist should meet the same structure with different cases.
A strong answer shows the work, and a weak one repeats the rule
Weak answers are often accurate summaries of the law with no way to check them. These pairs are written by us as examples, not quotes from real experts.
| Question | Weak answer | Strong answer |
|---|---|---|
| 1. The unpaid rent | "The tenant can withhold rent if the landlord doesn't fix things." | "It depends on where the property is. I would ask that first, then whether the tenant gave written notice, and read the statute for that place before I said anything firm." |
| 11. "Reasonable efforts" | "It means they have to try hard." | "It is ambiguous. A buyer will say a careful supplier's standard, a supplier will say it stops short of unreasonable cost. I'd check what the agreement says about cost and deadlines first." |
Notice who owns the claim. "The law says" is a quotation. "I read it, and here is the date" is a person who did the work.
The follow-up asks what the expert does when the citation exists but does not support the claim, which a prepared expert cannot recite.
Ask every applicant the same questions without booking a single call
A lawyer on your own team cannot interview every applicant when a project lands with a deadline. Tracy runs the interview for each expert, and each setting below is on the product page.
- Brief or your own questions. Describe the role and Tracy drafts questions you can edit, or paste the 19 above. Style runs from quick to deep.
- CV check, as a setting. With it on, Tracy can read the CV and ask about what it claims.
- Follow-ups that wait. Tracy asks for the missing detail in the expert's field and waits until the thought is finished, which matters for questions 4 and 13. A director at an accelerator residency programme said: "Tracy sometimes brings up exactly those questions I might have forgotten to ask, which has been a pleasant surprise."
- Start on click. Experts open a browser link with no account and no calendar, at any hour. The join screen says the conversation is recorded and transcribed. Candidates rate Tracy 92 on NPS.
- A result a reviewer can check. A recording, a timestamped transcript and a written assessment for each question sit behind every result.
- Integrity signals, with evidence. Tracy captures tab changes, gaze, face detection, delay before answers, fullscreen exits and external screens, and shows the evidence to a human reviewer. A signal is not proof, and your team decides. See AI interview cheating detection.
Pricing is $1 per credit, one credit per started 20 minutes of an interview that produced a result, so a 25-minute interview takes two credits. The first 3 results are free with no card, there are no seats and an empty session is never charged. See the pricing page, or the Connect page for the API and MCP server.
Each point in the result links to the moment the expert said it, so a lab client can see why the expert was approved.
Verify admission yourself, and keep a lawyer in the loop
- Admission and standing. Check whether someone is admitted to practise, and in good standing, with the body that keeps the record. Tracy does not.
- A lawyer on your side. Have one of your lawyers read five or six transcripts and score them blind. If they disagree with the written assessments on citation checks, change the questions before inviting more experts.
- The decision. A person approves or declines from the evidence.
If you approve a handful of attorneys a month, one conversation with your own lawyer is likely the better use of time (AI interview vs human interview). The case for Tracy starts when applicants outnumber your reviewers' hours. Pilot on one practice area in one jurisdiction with 20 applicants, and read AI interview reports first.
Questions
What should you ask a legal expert in an AI training interview?
How do you test legal reasoning without giving legal advice?
Can an AI interviewer interview legal experts?
Does Tracy check bar admission or credentials?
How long should a legal expert interview be?
Run these questions on your own applicants. The first 3 results are free, with no card. Try it free · Book a demo
Sources
- Stanford HAI, 23 May 2024 (updated 30 May 2024): definition of a misgrounded legal answer, retrieved 9 Nov 2026.
- Tracy AI Interviewer pricing, retrieved 5 Nov 2026.
- Tracy AI Interviewer Connect page, retrieved 5 Nov 2026.
- Tracy AI Interviewer home page, customer proof, retrieved 9 Nov 2026.