AI Mock Interview Practice: What It Grades Well and Where It Misleads

You get better at pleasing the grader, not at passing the interview. An AI mock interview gives instant, patient feedback, but part of that feedback depends on how you ask, how long you talk and chance.

Picture a common setup. A final-year student in Pune has a system design round in three weeks, and the practice question is a URL shortener, a service that turns long links into short ones. The setup is made up, but the grading problems behind it are documented in the studies cited below.

Every night the student pastes an answer into a chat assistant and asks how it went. The scores climb from 6 to 8 out of 10 (made up). Then a senior engineer runs a human mock, and the answer falls apart in ten minutes.

The student never asked how many links a day the service must handle. The answer was polished but ran well past the time limit. When asked why not number the short codes with a plain running counter, the student had no trade-off ready.

A grader with a rubric could have flagged two of those misses: the missing scale question and the missing trade-off. The way the student asked hid them. This guide fixes the asking first, then adds the human checks a model cannot replace.

You need six words. Each one comes back in the prompts and the weekly plan.

• AI mock interview: a practice round where a chat assistant plays the interviewer, asks follow-up questions and gives feedback. Many assistants also offer a voice mode, where you speak and it listens, as of 27 October 2026.
• Rubric: a fixed list of what a good answer must show, such as stated requirements or a capacity estimate. You grade against it one line at a time.
• Model grader: the assistant used as a judge of an answer. Researchers call this LLM-as-a-judge, where LLM means large language model, the kind of program behind chat assistants.
• Sycophancy: a model's habit of agreeing with you or praising you because of how you framed the request, not because the answer earned it.
• Transcript: the written record of the round. In voice mode, the transcript you save and paste for grading is made by speech recognition, so its mistakes become your mistakes.
• Human mock: a practice interview run by a person, ideally an engineer who has interviewed candidates before.

An AI mock interview is a loop: the assistant asks, you answer, it asks a follow-up, and at the end it grades the transcript. Its strength is cheap, patient repetition with instant feedback on what you covered.

The grading step is what researchers study as LLM-as-a-judge. A 2023 study found strong model judges agreed with human preferences over 80 percent of the time on open chat questions. That matched how often humans agreed with each other.

That result has a scope. It compared answers to general chat questions by preference. It did not test interview hiring decisions, so read it as a sign the tool can help, not as a hiring verdict.

An everyday picture helps. A model grader is like a practice examiner marking your mock board exam paper. The rubric is the marking scheme, and quoted evidence is the examiner underlining the line that earned each mark.

Grading three times is asking three examiners to mark the same paper. The human mock is the real exam hall. Where the picture breaks: a strict examiner does not add marks because you say you are proud of the answer, and a model grader can.

Checks a model grades well

A model grades best when the answer to the check can be found in the transcript. These five kinds of check fit that test.

• Coverage: did you state requirements, estimate load, sketch an API (the list of requests the service accepts), a data model and a trade-off. Each is a yes-or-no line on your rubric.
• Structure of a story: whether a behavioural answer has a situation, your own action and a result. The common frame is STAR, short for situation, task, action and result.
• Vague wording: phrases like it just scales, terms you never defined, and claims with no number behind them.
• Follow-up volume: it never tires of asking why, so you can rehearse ten follow-ups on one design in one sitting.
• Facts with a reference: for example, the time complexity of a standard algorithm, meaning how its running time grows with input size. This works when the grader is given or first writes the correct answer.

Research supports the yes-or-no point. A 2024 study repeated each judgement 100 times. Simple correct-or-incorrect checks with the right answer supplied were almost perfectly consistent. The authors linked lower reliability on another task to several answers being correct.

Another 2024 study turned each task into a checklist of yes-or-no questions before judging. Exact agreement with human preferences rose from 46.4 to 52.2 percent. That is a real gain, and also a reminder that even checklists leave room for disagreement.

It misleads in six ways, five of them measured in studies. It flatters, it can reward length and it changes its mind between runs. It tilts towards its own writing and one answer position, and can accept a confident wrong solution. And it only sees words.

It tells you what you want to hear

Sycophancy is well studied. A 2023 study gave five assistants the same maths solutions, arguments and poems. Feedback became more positive when the user said they liked the text or wrote it.

The same study then pushed back on correct answers, saying the answer seemed wrong and asking if the assistant was sure. Assistants often abandoned correct answers. A separate 2023 study, on classification tasks, found models changed answers about 46 percent of the time when challenged this way.

For you this cuts both ways. Saying you wrote the answer inflates the praise. Arguing with the grader can make it agree with a design that is actually wrong.

It can reward longer answers

Verbosity bias means a grader prefers a longer answer even when the extra words add nothing. The 2023 judge study padded 23 answers with a reworded copy of their own list.

Pause and predict: how often would a grader prefer the padded copy that says nothing new? Two weaker judges preferred it in 91.3 percent of the 23 cases.

The strongest judge in that test fell for it in 8.7 percent of cases, so the size of the bias depends on the grader. A timed round punishes padding either way. So put a time or word limit in the prompt and in your rubric.

It changes its score between runs

Grading the same answer twice can give two different results. A 2024 study re-ran judges 100 times per question, changing only the random seed, the number that drives a model's randomness. It warned that a single judgement can mislead even with fixed settings.

A 2026 study of several graders found substantial variability on identical inputs even at temperature 0. Temperature is the setting meant to control how random the output is, and 0 is its least random value.

It tilts towards its own writing and one answer position

If you ask for a model answer and then ask which is better, yours or its own, expect a tilt. A 2024 study found model graders rated their own outputs higher than other models' outputs, even when human raters judged them similar.

Order matters too, which researchers call position bias. In the 2023 judge study, even the strongest judge, using the study's default prompt, gave the same verdict after two answers were swapped in only 65 percent of cases.

It is easily swayed on technical facts

A grader shown a confident wrong solution can accept it. In the 2023 judge study, maths grading failed in 14 of 20 judgements. Giving the judge its own separately written answer as a reference cut failures to 3 of 20.

That study tested maths, not code, but the lesson carries over. A grader reading your code is not running it, so run your code against your own test cases before asking for any review.

It only sees words

A transcript loses what an interviewer notices: the diagram you drew, a long silence, whether you checked in before going deep. A grader judges only the words it received.

Voice mode often adds a step: speech recognition turns your voice into the text you later grade. Word error rate is the share of words a recogniser gets wrong. A 2020 study of five commercial recognisers found an average of 0.35 for Black speakers of African American English and 0.19 for white Californian speakers.

That study measured speech in the United States, not Indian accents, so do not read its numbers onto yourself. It does show that recognisers can work better for some voices than others, and systems have changed since 2020. Read your own transcript before you trust any feedback on clarity.

Fix the rubric before you grade, keep who wrote the answer out of the prompt, demand quoted evidence and cap length. The seven prompts below use capital-letter placeholders, such as PASTE TRANSCRIPT HERE; replace them before sending.

Run the interview in one chat and the grading in a fresh one. That keeps cues such as how you felt about your answer, which changed feedback in the sycophancy study, out of the grading.

Prompt 1: run the round

Act as a strict interviewer for a 45 minute system design round.
The question is: design a URL shortener.
Ask one question at a time and wait for my reply.
Do not give hints, praise or a model answer during the round.
If I start drawing components before asking about scale, interrupt me.
When I type END, stop. Do not grade yet.

Prompt 2: grade against the rubric

Below is a transcript of a practice system design round.
Grade it as a strict reviewer with no stake in the result.
Use this rubric, one line at a time:
PASTE RUBRIC HERE
For each line, give PASS, PARTIAL or MISSING.
Quote the exact words from the transcript that prove the grade.
If you cannot quote evidence, the grade is MISSING.
Do not reward length. Extra words that add nothing count against the answer.
Transcript:
PASTE TRANSCRIPT HERE

Notice what the prompt leaves out: who wrote the answer and whether you like it. The 2023 sycophancy study changed feedback with exactly those cues. Leaving them out removes those two triggers, though not every bias.

Prompt 3: hold your ground

Before you change any grade, restate the quoted evidence for your first grade.
Change it only if I point to words in the transcript that you missed.
If I just disagree without new evidence, keep the grade and say why.

Prompt 4: drill follow-ups

Here is a summary of my design:
PASTE SUMMARY HERE
Ask me ten follow-up questions a senior interviewer would ask, hardest last.
Ask one at a time and wait for my answer.
After each answer, name only the rubric line that question tested.

Prompt 5: check a technical answer with a reference

Solve this problem yourself first and show your full answer:
PASTE PROBLEM HERE
Then compare my solution below with yours, step by step.
List every difference, and mark any point you are not certain about as UNSURE.
My solution:
PASTE SOLUTION HERE

This copies the reference-guided fix from the 2023 judge study, where the grader's own separate answer cut maths grading failures. For code, still run your tests; the comparison only tells you where to look.

Prompt 6: compare two versions fairly

Here are two answers to the same question, labelled FIRST and SECOND.
Using this rubric, say which one meets more lines, quoting both:
PASTE RUBRIC HERE
FIRST:
PASTE ONE ANSWER HERE
SECOND:
PASTE THE OTHER ANSWER HERE

Send it twice, in two fresh chats, with the answers swapped. Count a win only if the same answer wins both times; otherwise call it a tie. This is the swap check the 2023 judge study used against position bias.

Prompt 7: check a behavioural story

Here is a behavioural interview story:
PASTE STORY HERE
In one line each, using my own words, list the situation, my action and the result.
If the action describes what the team did rather than what I did, say so.
If the result has no number or visible outcome, say so.

Use one short rubric per round type, with lines that can be checked in the transcript. Write it before you practise, so neither you nor the grader can move the goalposts.

Here is the rubric the student uses for the URL shortener round. The SevyDevy guide on common system design interview mistakes is a good source of extra lines.

Grade each line PASS, PARTIAL or MISSING instead of one score out of 10. A single number hides which line moved, and it invites the run-to-run swings described above.

Beginners can stop here: you have a rubric, the reasons to distrust a bare score, and the prompts that make a grader show evidence. The rest covers stability checks, a weekly plan and the interview frame.

Grade the same transcript three times in fresh chats, and act only on lines that got the same grade every time. Lines that disagree go onto the list for your human mock.

Three runs is our rule of thumb, not a number from a study. The research only says a single judgement can mislead. Three is small enough to take a few minutes and large enough to show wobble, and our rule asks all three to agree.

Copy each run's grades into this short script. It needs only Python 3 and prints one verdict per rubric line.

# Paste the grades from three fresh-chat runs of the same transcript.
runs = [
    {"Requirements first": "PASS", "Capacity estimate": "PARTIAL",
     "Trade-off named": "PASS", "Failure handling": "MISSING"},
    {"Requirements first": "PASS", "Capacity estimate": "MISSING",
     "Trade-off named": "PASS", "Failure handling": "MISSING"},
    {"Requirements first": "PASS", "Capacity estimate": "PARTIAL",
     "Trade-off named": "PARTIAL", "Failure handling": "MISSING"},
]

for line in runs[0]:
    grades = [run[line] for run in runs]
    if len(set(grades)) == 1:
        verdict = "stable: act on it"
    else:
        verdict = "unstable: ask a human"
    print(f"{line:20} {' / '.join(grades):28} {verdict}")

We ran it on Python 3.13 with the sample grades above. It printed this:

Requirements first   PASS / PASS / PASS           stable: act on it
Capacity estimate    PARTIAL / MISSING / PARTIAL  unstable: ask a human
Trade-off named      PASS / PASS / PARTIAL        unstable: ask a human
Failure handling     MISSING / MISSING / MISSING  stable: act on it

What just happened? Requirements and failure handling got the same grade three times, so the student keeps the first and fixes the second. The estimate and the trade-off wobbled, so those two go to the human mock on Saturday.

Run four short AI practice rounds, one grading session and one human mock each week, and let the human mock choose next week's focus. The AI sessions give you volume; the human mock calibrates them, meaning it tells you which AI grades match a real interviewer.

The split is our suggestion, not a research finding. Shift it to fit your timetable, but keep one human mock a week and the same rubric for both.

1. 1

   Monday: timed AI round

   Run Prompt 1 on the week's question with a phone timer. Save the transcript. Do not grade yet.

2. 2

   Tuesday: grade and pick one line

   Run Prompt 2 three times in fresh chats and the stability check. Pick one stable MISSING or PARTIAL line to fix this week.

3. 3

   Wednesday: coding round

   Solve one problem against a timer, then run it on your own test cases, including empty and very large inputs. Only then use Prompt 5.

4. 4

   Thursday: behavioural round in voice mode

   Tell two stories aloud. Read the transcript for speech-to-text errors first, then run Prompt 7 on each story.

5. 5

   Friday: follow-up drill

   Run Prompt 4 on Monday's design, aiming at the line you picked on Tuesday. Answer aloud within two minutes each.

6. 6

   Saturday: human mock

   Ask a classmate, senior or working engineer to run the same question for 45 minutes, using your rubric. Give them the unstable lines to watch.

7. 7

   Sunday: compare and set focus

   List where the human and the AI disagreed. The top two disagreements become next week's rubric changes and focus.

Finding human mocks is easier as a swap. You interview a friend one week, and they interview you the next. A peer who has never hired anyone still notices timing, silences and unclear diagrams.

Save senior engineers for the question only they can answer: would this pass at the level you are aiming for. Give them your rubric so their notes line up with the AI's.

Use it to practise, not to cheat, and keep private material out of it. Three limits matter most.

• Do not paste questions from a real interview you were asked to keep confidential, or your employer's internal documents. Practise on public questions instead.
• Check the data settings of the tool you use, such as whether chats are saved or used for training. Settings differ between tools and change over time.
• In a real interview, use AI only when the company says you may. The SevyDevy guide on AI-assisted coding interviews covers rounds where it is allowed.

Rehearse one fixed answer frame until it is automatic, and use the AI to drill each step and the human mock to test the whole. The SevyDevy system design interview framework guide explains each step in depth.

1. 1

   Clarify requirements

   Ask about scale, reads versus writes and expiry in the first five minutes. This line is a yes-or-no check, so AI grades it well.

2. 2

   Estimate capacity

   Say the numbers aloud with units, and redo the maths later yourself. Do not let a grader approve arithmetic you have not checked.

3. 3

   Define the API and data

   Name the create and redirect endpoints and the table that maps short codes to long URLs.

4. 4

   Draw the request path

   Walk one redirect from browser to cache to database. Ask the interviewer which part to go deeper on.

5. 5

   Name trade-offs and failures

   Give one sharp trade-off, such as random codes vs counter codes, and say what happens when a node fails.

6. 6

   Wrap up on time

   Summarise the design in under a minute and say what you would build next. Only a timer and a person can grade this step.