How do you decide what to use versus reject from the AI's output?

In this article

1. Before you start
2. Why interviewers ask this
3. Deciding what to keep from AI output, from first principles
4. The Staff-level answer
5. The rubric
6. Three variants of this question
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

Interviewers set the rules for AI use stage by stage, so it helps to know how the interview stages and rounds fit together.

Why interviewers ask this

An interviewer who allows AI is no longer scoring who types the cleanest code; the interviewer is scoring what you do with code you did not type. Accepting everything shows only the assistant's skill. Rejecting everything shows you cannot use a tool your future team will use.

Canva, in its engineering post, says it asks whether candidates can identify and fix issues in AI-generated code and whether they can make solutions meet production standards. The post also says the most successful candidates "didn't just prompt AI and accept whatever it generated." Candidates with little AI experience struggled, it adds, not because they could not code but because they lacked the judgment to spot suboptimal suggestions.

Rules differ by company. Anthropic's candidate guidance asks for live interviews to be done without AI unless Anthropic says otherwise. A weak answer says "I check that it works." A strong answer names what it checks, in what order, and what it throws away.

Deciding what to keep from AI output, from first principles

Edge case

An edge case is an input at the boundary of what a function must handle, such as an empty list, a single item, a duplicate or the largest allowed size, where code that works on typical inputs often fails.

Verification

Verification is the act of checking that code does what the requirement says, by running it on inputs you chose and by reading it, instead of trusting that it looks right or that its author sounded sure.

In plain words

Treat the assistant like a fast new colleague. The colleague writes fluent drafts and never says "I am not sure". You would read a draft like that before putting your name on it.

The analogy breaks in one place. A human colleague usually sounds less certain when they are guessing. An assistant often sounds equally certain whether it is right or wrong. The National Institute of Standards and Technology (NIST) names this risk, and its generative AI profile, NIST AI 600-1 (July 2024), §2, defines confabulation as "confidently stated but erroneous or false content."

In one precise sentence: you keep a suggestion only after you have checked it against the requirement, run it on inputs the assistant did not choose, read it for faults tests miss, and can explain it line by line.

The four steps below follow that order. The example is illustrative: invented code and invented test cases, not output from a real assistant.

1 · Rules and requirement2 · Unseen test3 · Read for faults4 · DecideReplay

Step-by-step description

1. Know what the round allows and what must hold.
2. Run inputs the assistant did not choose.
3. Look for what tests cannot show.
4. Use, rewrite or reject, and say why.

• Check the rules of the round and the requirement before you use any suggestion.
• Test the suggestion on a case the AI did not choose.
• Read the code for what tests cannot show.
• Decide to use, rewrite or reject the suggestion, and say why out loud.

Step 1: Check the rules of the round and the requirement

Check the rules of the round and the requirement before you use any suggestion. The first use-or-reject decision is not about code at all. Anthropic's guidance says live interviews are "all you–no AI assistance unless we indicate otherwise", and that it will say when AI is allowed for a coding challenge, while it encourages Claude for preparing and for sharpening a draft you wrote yourself. Canva, by contrast, expects AI use in its coding round. If the rules are unclear, ask the interviewer before you open the assistant.

Next, restate the requirement in one sentence out loud and list its constraints. Suppose the task is a function that returns the median of a list, with None for an empty list. That sentence is your yardstick: every suggestion is measured against it, not against how plausible it looks. You now have something to test against, which is what the next step needs.

Step 2: Test the suggestion on a case the AI did not choose

Test the suggestion on a case the AI did not choose. An assistant's own demo covers the inputs it thought of, which are usually the easy ones. Suppose it offers this function, with a demo on an odd-length list and a one-item list:

def median(xs):
    s = sorted(xs)
    return s[len(s) // 2]

The AI version passes every check in its demo but only half of the full tests. Choose your own inputs: an even-length list, an empty list, a pair of equal values. The table shows an illustrative run.¹

The code is fluent and looks right, yet it picks the upper middle item instead of averaging the two middle ones, and it crashes on an empty list. The fixed version passes all six tests. Say what failed and why out loud; that sentence is the signal. The fixed version returns a float whenever the length is even (2.0 for the equal pair) and the original item for odd lengths; the values compare equal in Python, but it is worth stating if asked.¹

Text description of the diagram

• The AI version passes every check in its demo but only half of the full tests.
• The fixed version passes all six tests.

Step 3: Read the code for what tests cannot show

Read the code for what tests cannot show. Tests only cover the inputs you thought of, so a clean run is not proof. Reading finds a different class of fault: a call to a function or library that does not exist, which is confabulation; an outdated call to a library you know has changed; a database query built by joining strings, which is open to injection; a speed claim the loops do not support; and code that runs but breaks a stated constraint, such as changing the input list when the requirement said not to.

These are typical failure types, not measured frequencies. Practise one habit: when the assistant states a complexity, check it against the loops in the code you can see. In the median example the function sorts the whole list, so its cost grows with the length of the list times its logarithm. That is fine for a list but would not fit a requirement to track the median of a stream. Reading ties the code back to the requirement from step one.

Step 4: Decide to use, rewrite or reject the suggestion, and say why out loud

Decide to use, rewrite or reject the suggestion, and say why out loud. Use it as it stands when it meets the requirement, passes your tests and you can explain every line. Rewrite it when the idea is right and a detail is wrong, as with the median: keep the sorting, replace the last line.

def median(xs):
    if not xs:
        return None
    s = sorted(xs)
    mid = len(s) // 2
    return s[mid] if len(s) % 2 else (s[mid - 1] + s[mid]) / 2

The standard library's statistics.median is another rewrite option, but it raises an error on an empty list, so the None requirement still needs a guard.

Reject it when the approach is wrong or you cannot explain it, and name a second viable design and the trade-off that made you drop the first. Ownership is the closing check. Canva says it expects engineers "to take full ownership of any code they produce, whether they wrote it themselves or with AI assistance." In a round, that means whatever you submit is yours to defend.

The Staff-level answer

It is Staff-level because it orders the checks, ties every decision to a reason and ends on ownership instead of on the code. The spoken structure: rules, unseen tests, reading, then a stated decision.

The rubric

This table describes what a strong answer shows; it is a guide for preparing, not any company's scoring sheet.

Three variants of this question

Walk me through how you would review this AI-generated function.

Restate the requirement, run your own tests on empty, single and even-length inputs, then read for faults tests miss, such as invented calls and unsupported complexity claims. Finish with a stated decision: use, rewrite or reject.

When would you not use the AI at all for this task?

When the rules of the round say not to, or when the task is the skill being tested, such as writing the core loop from memory. Anthropic's guidance, for example, marks live interviews as "all you" with no AI assistance unless it says otherwise.

The tests pass, but the AI's approach is slower than the one you had in mind. Which do you submit?

Say which requirement decides it. If the input size or latency requirement makes the slower approach a problem, rewrite or reject; if it does not, keep the clearer one and state the trade-off you accepted.

Common mistakes

• "It runs on the example, so I will keep it." (wrong) The example holds the cases the assistant picked. (right) "It passes the assistant's example; I will test an empty and an even-length input before I keep it."
• "The assistant sounds sure, so the logic must be right." (wrong) Fluent, confident text can be false. (right) "Confidence is not evidence; I will check it against the requirement."
• "I will fix the bug silently and move on." (wrong) The interviewer sees only the final code, not the judgment. (right) "The even-length case failed because it takes the upper middle item; I am averaging the two middle items instead."
• "I rejected it because I prefer my own style." (wrong) A style preference is not a reason an interviewer can score. (right) "I rejected it because it fails the empty-list requirement and the fix is smaller than the rest of the function."
• "The test failed, so I will ask the assistant to fix its own failing test." (wrong) Without reading why it failed, you cannot tell a real bug from a wrong test. (right) "I read the failure first: the even-length case returned the upper middle item, so I am fixing that line."

Follow-up questions

Is it allowed to use AI in a coding interview?

It depends on the company and the stage. Canva expects AI use in its AI-assisted round, while Anthropic asks for live interviews without AI unless it says otherwise. Ask before you start; this overview of how interview rounds differ helps, and the Anthropic interview process page covers one company in detail.

What if the AI's code works but I do not understand it?

Do not submit it as it stands. Ask the assistant to explain it, trace it by hand on a small input, or replace it with something simpler you can defend, because you are expected to own the code you hand in.

How do you spot a made-up function in AI-written code?

Check every call against documentation or a quick run of that single call in isolation. A made-up function is a form of confabulation, so a confident name and a plausible signature prove nothing.

If the interviewer allows AI, how much of it should I use?

Use it for well-defined subtasks and keep control of the overall solution, which is the pattern Canva's post describes in its strongest candidates. Hand over the boring, checkable pieces; keep the design decisions.

What do you say out loud when you reject a suggestion?

Name the failure, the requirement it breaks and what you will do instead, in one or two sentences. Practising that phrasing is part of interview skills, and a take-home or online assessment may ask for the same reasoning in writing, as in a take-home assignment example. The AI engineer question list has more questions in this style.

Sources

1. Yes, You Can Use AI in Our Interviews. In fact, we insist — Canva Engineering Blog (Simon Newton), 11 Jun 2025
2. How to collaborate with Claude during our hiring process (Guidance on Candidates' AI Usage) — Anthropic, 10 Jul 2025
3. NIST AI 600-1, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile — NIST, 26 Jul 2024