Skip to content

How to Run a Coding Interview With Follow-Ups on Trade-Offs

In short

Tracy AI Interviewer runs a coding task inside the interview, then follows up on each candidate's approach and trade-offs, so engineers review reasoning and not only output.

Candidates start on click at any hour. Your team reads a recording, a timestamped transcript and a written assessment for each question, and a person decides.

Next step: run one coding task as a Tracy interview and read the follow-ups.

We make Tracy AI Interviewer.

Picture a candidate who writes a rate limiter that passes every test. The interviewer says "great" and moves on to the next problem. Nobody asks why the candidate chose a counter over a list of timestamps, or what happens under load. The first answer showed they had seen the problem. Coding interview follow-up questions show whether they can reason about it.

This page gives four kinds of follow-up, example questions with what to listen for, and how Tracy AI Interviewer runs them for every candidate: a coding task in the same interview, then follow-ups on approach and trade-offs, with the recording and transcript for your engineers to review.

How do you follow up on a coding answer?

A passing solution shows the candidate has seen the problem. Tests check an output, not why the candidate chose a hash map or a queue, or what changes when the input grows a hundred times. In our view, ten minutes of follow-ups on one problem tell you more than a second problem does. Four kinds of coding interview follow-up questions reach the reasoning behind the code:

  • The rejected option. "You used a list of timestamps. Why not a counter that resets each minute?" It makes the candidate name an alternative and say what it costs.
  • The changed constraint. "Now there are 10,000 requests a minute from one user. What changes?" It shows whether they see where the design bends.
  • The failure mode. "Two servers handle the same user. What goes wrong?" It tests whether they think beyond one machine.
  • The cost. "What does this use in memory and time, and who pays when it is slow?" It tests whether they can put a rough size on it.

A generic "what is the time complexity?" gets a recalled answer. A follow-up that quotes the candidate's own choice cannot be recalled, because that choice did not exist before they wrote it.

Coding interview follow-up questions for four common problems

Pick one problem and two follow-ups from the table, and write down what a strong answer contains before the first interview.

ProblemFollow up withListen for
Rate limiter for an APIWhy a list of timestamps and not a counter that resets each minute? Then: what happens at 10,000 requests a minute from one user?Names the burst at the window edge. Gives a rough memory cost and a cheaper option such as a sliding counter, and says what accuracy it gives up.
Cache with expiryWhat does a reader see if an invalidation arrives late?Separates stale reads from lost writes. Says which the product can tolerate and why.
Remove duplicates from a sorted listWhat if the list does not fit in memory?Moves from in-place to streaming and says what order guarantee that needs.
Slow SQL reportThe query took two seconds yesterday and a minute today. What do you check first?Starts with the plan and the data, not with a rewrite. Names an index and what it costs on writes.

These questions are our own examples. An engineer on your team should adjust each to the stack and the level.

How many follow-ups should you ask per problem?

How you ask matters as much as what you ask:

  • Let the candidate finish. The better part of an answer often comes after the first sentence.
  • Quote their words. "You said you would drop old timestamps. When?" shows you listened.
  • Go two levels. The first follow-up tests the choice and the second tests the cost.
  • Ask every candidate the same opening problem. The follow-ups differ, because they follow the answers. The starting point does not.

Doing this well takes an experienced engineer in the room. The limit is time: your senior engineers have a few hours a week, and a hiring round can bring far more candidates than those hours cover.

What makes a trade-off answer strong?

Whatever the problem, a strong answer to a trade-off follow-up usually holds four things:

  • It names a cost. Memory, latency, complexity, a failure mode. "It is faster" is not a trade-off.
  • It names the condition that would flip the choice. "At this size a list is fine. Past about this many requests I would use a counter."
  • It puts a rough size on it. Thousands or millions, milliseconds or seconds.
  • It says what the candidate does not know. "I would measure that" is a better answer than a confident guess.

A weak answer restates the first answer in more words, or names a technique with no cost attached. Both are easy to see in a recording and hard to recall in a debrief a week later.

How Tracy runs the follow-up for every candidate

Tracy runs a coding task inside the interview itself: live coding, SQL, debugging or multiple choice. The candidate talks through the approach in a coding room. She asks about what they said and wrote, waits for the end of the thought and follows up on the approach and trade-offs. Your engineers choose the problem, or describe the role and edit the questions she drafts, and set the skills and their weights. A Director at a top-tier early-stage VC says of founder interviews that Tracy "sometimes brings up exactly those questions I might have forgotten to ask, which has been a pleasant surprise."

Candidates take the interview from a browser link, with no engineer on the call. Afterwards your team reads a recording of both sides, a timestamped transcript and a written assessment for each question with its reasoning. Candidates are ranked against others who took the same interview. Tracy also shows integrity signals such as tab changes and delay before answers, with their evidence, to the reviewer, as signals for a person to interpret and never as proof.

Card: Tracy asks what to check on a slow report; the candidate says the query plan; Tracy asks what an index costs on writes.
Tracy follows up on the cost of a candidate's fix to a slow query, so engineers see whether they reason about trade-offs. Illustration with sample data

Tracy follows up on the cost of a candidate's fix.

Transcript card: Tracy asks why a list of timestamps, the candidate names the burst, Tracy asks what changes at scale.
Tracy follows up on a candidate's own design choice, so your engineers see whether they can reason about cost. Illustration with sample data

A follow-up built on the candidate's own answer shows whether they can reason about cost.

Keep these calls with your engineers

A follow-up shows reasoning. It does not replace your engineers' judgement about the person:

  • Reading the code. Style, clarity and how the solution would live in your codebase. Review the code itself, not only the discussion.
  • Large system design. It needs a longer conversation with a person.
  • The decision. A recruiter can override a rejection, move a candidate forward or invite them to interview again, and the history stays on the record.

For a first round that covers every applicant, see how an AI screening interview works. For a bank of problems to use inside it, see software engineer interview questions with follow-ups. Our guide to AI technical interviews covers the formats, the technical interviews use case shows the flow, and what Tracy does and AI interview reports cover the product and what your team reads afterwards.

Pilot it on one role family

Start with one role family, one owner and a stop condition.

  1. Pick the role and the owner. One backend role, one engineering manager.
  2. Choose one problem and two follow-ups from the table. Write the strong answer for each first.
  3. Run 20 candidates through it. Have two engineers read the same five results and say who they would advance.
  4. Compare with your current screen. Note who advanced here that the old screen would have dropped.

Stop condition. If two engineers agree on who advances no more often than they did from the old screen, rewrite the follow-ups and rerun on the next 20 before widening. At $1 per credit and one credit per started 20 minutes, 20 candidates at 20 minutes cost $20, and the first three results are free (see pricing).

Want it set up for you? Send us one role and we will build the interview plan in days.

Common questions

What are good coding interview follow-up questions?
Ask about the candidate's own choice: why this structure and not another, what changes if the input grows a hundred times, what breaks with two servers, and what it costs in memory or time. Each should refer to what they said or wrote.
How many follow-up questions should you ask per problem?
Two levels per decision: one on the choice, one on its cost. More turns the interview into an interrogation.
Can an AI interviewer ask coding follow-up questions?
Yes. Tracy AI Interviewer runs a coding task inside the interview and follows up on approach and trade-offs. Your engineers read a recording, a timestamped transcript and an assessment for each question, and a person decides.
What if the candidate uses an AI assistant?
State your rule in the invitation. Tracy shows tab changes and delay before answers to the reviewer as evidence, never as proof, and a follow-up on the candidate's own choice shows whether they can explain the code.

Try Tracy on your next role

Tracy interviews every applicant, asks follow-up questions and sends you a ranked shortlist.

Try it free (opens in a new tab) Book a demo (opens in a new tab)

Your first 3 results are free. No card needed.

Sources