Skip to content

Collect interviews in ChatGPT and keep every code checkable

In short

ChatGPT for qualitative research speeds up the first pass: summaries, candidate codes and groupings. A 2024 viewpoint that tried ChatGPT-3.5 on one transcript found it may give different answers to the same question, so every code needs a quote. Tracy AI Interviewer collects the interviews from the chat, with a recording and timestamped transcript behind each one. ChatGPT codes, and your team verifies and decides.

What ChatGPT for qualitative research does well

A 2024 viewpoint in the Journal of Medical Internet Research tried ChatGPT-3.5 on one podcast transcript and concluded it can make thematic analysis more efficient and add insight, with human researchers staying involved. That is the repetitive first pass: a three-line summary per participant, candidate codes, groupings and the sentences that support each one. The example on this page is a study of how front-desk staff at small clinics handle no-shows.

The gaps sit on either side of the coding. Thin answers give thin codes, and a second run can code differently.

Where does ChatGPT need checking in qualitative analysis?

The JMIR authors, who used ChatGPT-3.5, list three limits, and each one has a cost in your study. ChatGPT "may give different answers to the same questions, leading to nonreproducible results", so a second run can propose different codes. It "may not capture every nuance that a human analyst would pick up", so a softly worded concern can vanish. And the paper says "given ChatGPT's limited context window, ChatGPT may not remember the full text of an interview transcript or the codes it previously generated". That model handled a request and its response within a combined 4,096 tokens, about 3,000 words in English. Current models may differ, so check your own. The authors do not count the variability as a drawback: "we do not see this variability as a drawback, as humans would also generate different results when doing thematic analysis". They suggest the different responses may bring new insights. Read a difference between two runs the same way, as something to examine, not an error to hide. The paper is a viewpoint on one transcript with no participants, so it shows what to watch for, not how often it happens.

Where do the interviews come from?

Anywhere, but the data sets the ceiling. Thin answers give thin codes.

A code is only as good as the answer it came from, so the interviews and the checks matter more than the coding step.

A clinic receptionist says they just call patients; Tracy asks who they call first and what they say.
Tracy asks for the missing detail while the participant is still talking, so there is something to code. Illustration with sample content. Sample content, not a real participant.

A vague answer becomes codeable detail once Tracy asks for the last example.

With Tracy added to ChatGPT, you ask for the interview in the chat, check the preview and approve it, and each participant joins from a link in a browser with no account. Tracy asks for the missing detail: "we just call them" becomes "who do you call first, and what do they say?" The join screen says the conversation is recorded and transcribed. The setup walkthrough covers the install, and a first study from prompt to readout follows one example.

What Tracy adds to ChatGPT qualitative research

  1. Reach every participant you invite. Each person starts on click, at any hour, with no scheduling and no researcher on the call. Your sample stops depending on calendars.
  2. Ask the next question. Tracy follows the answer in the participant's own field and waits for it, so the data has detail worth coding.
  3. Keep the evidence. Each result has a recording, a timestamped transcript and a written assessment per question with its reasoning, so a theme can point to a quote and the quote to a moment.

What is a workflow that holds up?

Five checks sit around the coding step, and each one answers a limit listed above. The flow shows the loop they sit in.

Four steps with icons: collect with Tracy, code in ChatGPT, re-run and compare, verify quotes in the recording.
Tracy collects the interviews, ChatGPT codes them, and you re-run the coding and verify each quote in the recording. Illustration of the workflow. Source: workflow on this page.

Tracy collects, ChatGPT codes, you re-run and verify: the loop the five checks sit in.

What are the five checks?

  1. Re-run it. Ask for codes twice, in separate chats, and compare. Codes that appear both times are steadier. Codes that appear once need a look.
  2. Restate your code list. Paste the list again when you add transcripts, so early choices do not drift in a long chat.
  3. Demand quotes with timestamps. "For each code, give an exact quote, the participant and the timestamp." A code with no quote is dropped.
  4. Code a sample by hand. Read three transcripts yourself before you look at ChatGPT's codes, then compare. If its codes miss what you saw, find out why.
  5. Ask for the counter-case. "Which participants contradict this theme?" Then read those answers.

Tracy gives what each person said to each question, with the words and timestamps, which gives checks 3 and 4 something to land on. It reports what people said and does not score or rank them. Grouping answers into themes is done by ChatGPT or by you.

What does an audit trail look like?

A theme with no quote cannot be checked. Every theme should point to a quote, and every quote to a place in the recording. If your workflow cannot say that, the analysis cannot be checked, whatever tool wrote it. Here is a sample trace we wrote, with an invented participant and timestamp, from the front-desk study:

  • Theme: staff phone the patient first.
  • Quote: P2, "I call them before anything else."
  • Timestamp: 05:18 in P2's transcript.
  • Check: open the recording at 05:18 and read what came before and after.

A theme without the last three lines goes back for another pass.

Gathering that evidence by hand is the slow part. Five 20-minute interviews are 100 minutes of live calls before any reading starts. Collected through Tracy, those calls happen without you, and the time goes into the checks.

Four steps: theme, quote from P2, timestamp 05:18, open the recording at that moment.
One theme traced to a quote, a timestamp and the recording. Sample we wrote, with invented content. Sample we wrote; invented content.

A theme you can check points to a quote, and the quote to a moment in the recording.

Can ChatGPT handle a large set of transcripts?

Not in one go. OpenAI's pricing page shows file uploads limited on Free and included on Go, Plus and Pro, and the 2024 JMIR authors, testing ChatGPT-3.5 with a 4,096-token limit, found it may not remember a whole transcript or its earlier codes. Current models may differ, so check your own. Work in batches of a few transcripts, keep the code list in view and merge the batches yourself. A step-by-step reading method covers the analysis, and the same workflow in Claude is on its own page.

What does a first run look like?

Start small. Collect five interviews on one question, such as what front-desk staff do when a patient does not turn up. Ask ChatGPT to summarise each participant in three lines, without interpreting. Then ask for codes with quotes and timestamps, and run the same request again in a new chat. Say one run gives you "call the patient" and "offer the slot to the waitlist", and the other gives you only the first. Go to the transcripts, find the waitlist remark and decide whether it belongs in your codebook. Repeat until two runs agree on most codes, and keep the disagreements in a note, since they show where the data is thin.

What stays with the researcher?

The question, the sampling, the interpretation and the claim you publish. If your study needs ethics review or must follow a named method, ChatGPT does not change that. Pricing for the interviews is $1 per credit, one credit per started 20 minutes, the first 3 results free (pricing).

Which path fits you?

Common questions

Can ChatGPT do qualitative research?
It can do a first pass: summaries, candidate codes and groupings. It may give different answers to the same question and may miss nuance, so check each code against the transcript.
How do I use ChatGPT for qualitative research?
Ask in small steps: summarise each participant, propose codes with quotes and timestamps, group them, then ask for contradicting cases. Re-run the coding and compare, and verify every quote.
Is ChatGPT reliable for thematic analysis?
Not on its own. A 2024 JMIR viewpoint that tested ChatGPT-3.5 on one transcript concluded researchers must stay involved, and its authors did not count variable answers as a drawback, since humans also differ. The test was small, so treat it as a list of risks, not a rate.
Can ChatGPT analyse interviews I have already recorded?
If you have transcripts, you can paste or upload them, subject to your plan. Work in small batches and check quotes against the recordings.
Where do I get interview transcripts?
From interviews you run, or from Tracy: each has a recording and timestamped transcript. You send participants a link and they join in a browser.
Does Tracy code or score the interviews?
No. Tracy gives what each person said to each question, with timestamps. It does not score or rank participants. The coding is yours, with or without ChatGPT.

Collect one small study from ChatGPT

Add Tracy, describe the study and approve the preview. You read what each person said. The first 3 results are free.

Try it free (opens in a new tab) Book a demo (opens in a new tab)

Your first 3 results are free. No card needed.

Sources