In this article
Before you start
Why interviewers ask this
The question is open on purpose, because the first sentence of your reply shows whether you treat prompting as guesswork or as engineering. Candidates report it in interviews for applied LLM roles. It often opens a round on prompt engineering before a task such as classification or extraction.
A weak answer lists wording tips: "be specific, add examples, say please." A strong answer names the loop: criteria, a first draft, a test set, small controlled changes. A Staff-level answer adds what happens in production: the model version changes, outputs vary from run to run, and the prompt is one part of everything the model reads.
Prompt engineering from first principles
- Success criteria
- Success criteria are the written, measurable statements of what a correct output is for a task, such as label accuracy or valid format, set before the first prompt is written so that every version can be scored against them.
In plain words
Think of writing a brief for a capable new colleague who has never seen your project. You say what you want, what a good result looks like, and show an example or two. Then you check their first few results against a checklist, not against your mood.
The analogy breaks in one place. The model can give different output for the same brief on different runs, and it remembers nothing from earlier requests unless you include them.
In short, you write, measure and revise the text the model receives. OpenAI's guide defines the practice as writing effective instructions so the model consistently meets your requirements, and notes that output is non-deterministic, so one good result proves little.
Step 1: Define success before you write anything
Writing down success criteria first is the foundation of the loop. Anthropic's guidance starts an application with exactly this: define success criteria, then design evaluations against them. Its example replaces "good performance" with "accurate sentiment classification."
Success criteria are specific and measurable. For sorting support tickets, two criteria are enough: the label matches a human label, and the output follows the required format so a program can read it.
Input: the task and what a good result means for the business. Output: a short list of numbers you can compute. You now know what the prompt will be judged on, which is what makes the next steps testable.
Step 2: Write a clear first prompt
Start with a plain prompt. The task is the instruction; the details are the audience, the allowed labels, the format of the answer and what to do when unsure. OpenAI's prompting guide advises putting overall tone or role guidance in the system message and task details and examples in user messages. A system message is the instruction slot, set by the developer, that frames the model's role and tone for the whole conversation.
Few-shot prompting is giving the model a few worked demonstrations inside the prompt at inference time, with no weight updates. Brown et al. (2020) use that definition, which is why it is called in-context learning rather than training. Keep the examples in one compact block so they are easy to scan and update, as OpenAI's guide suggests. If you need machine-readable output, see getting reliable structured output from an LLM.
Input: criteria and a few labeled examples. Output: prompt version one.
Step 3: Test on a labeled set, not on one example
An evaluation set is a collection of realistic inputs, each with the correct answer written down, that you run every prompt version against. OpenAI's guide recommends building tests and evaluation suites that measure prompt behavior, so you can monitor performance as you iterate or change model versions.
The reason is arithmetic. Take 200 invented support tickets (illustrative). Prompt v1 has no examples and labels 150 correctly, 75.0%. Prompt v2 adds three examples and labels 168 correctly, 84.0%.1 The standard error of an accuracy is the square root of p(1-p)/n, so it is about 3.1 points for v1 and 2.6 for v2.1
The 9-point gap has an approximate 95% interval of 1.1 to 16.9 points, so v2 probably helps, though the size is uncertain.1 Both prompts ran on the same tickets, so a paired comparison on the tickets where they disagree is the sharper test; the interval here treats the runs as independent, which is conservative.
If the same rates came from only 20 tickets (illustrative rates; 84% of 20 tickets is not a whole number), the interval would run from -15.9 to 33.9 points, and the test would say almost nothing.1 On one ticket it says nothing at all. Count format too: valid output rose from 85.5% to 98.0% in the toy run.1 Include adversarial inputs in the set, such as text that tries to override the instructions, and keep sampling settings such as temperature fixed while you compare versions.
Step 4: Change one thing at a time, then keep it stable
Iterate by changing one element per run, such as the instruction wording, the examples or the format, and re-running the evaluation set. A labeled test set scores every prompt version, and if two things change at once you cannot tell which one moved the score.
Production adds two habits. OpenAI's documentation advises pinning applications to specific model snapshots for consistent behavior, and its prompting guide says to run prompt tests every time you publish, because catching problems early is cheaper than fixing them in production. Re-run the same set whenever the model version changes.
Step-by-step description
- Write measurable criteria first.
- Role, task and a few examples.
- Score it on a labeled set.
- Then re-run the same tests.
- Success criteria are specific and measurable.
- A labeled test set scores every prompt version.
- Change one thing at a time and re-run the tests.
Text description of the diagram
- Prompt v2 scored 84% accuracy against 75% for v1 on 200 invented tickets.
- Valid format rose from 85.5% to 98%.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →The Staff-level answer
What's your approach to prompt engineering?
I treat it as a measured loop, not wordsmithing. First I write down what success means as numbers: for ticket triage, label accuracy and the share of outputs a program can parse. Then I write a plain first prompt, with the role in the system message and the task plus a few labeled examples in the user message.
I score it on a held-out set of realistic cases, not on one example I liked, and I look at the interval, because a nine-point gain on twenty cases could be noise. I change one thing per run, pin the model version, and re-run the set whenever the model changes. If prompting plateaus, I look at the wider context or at fine-tuning.
What makes it Staff-level: it names a measurement, admits that a small test set is noisy, and covers what happens after launch, not only the first draft.
The rubric
| Dimension | Weak answer | Strong answer | Staff answer |
|---|---|---|---|
| Correctness | Lists wording tips and magic phrases | Describes a loop of draft, test and revise | Starts from written success criteria and defines the metrics |
| Depth | "Add examples" with no reason | Explains few-shot as demonstrations in the prompt | Explains when examples or reasoning steps help and when they do not |
| Trade-offs | Never mentions cost | Notes extra examples cost tokens | Weighs prompting against retrieval or fine-tuning for the task |
| Communication | Rambles through techniques | Gives the loop in order | Opens with the loop, then one concrete example with numbers |
| Follow-up readiness | Surprised by "how do you know it works?" | Has a test set ready | Can state the noise on a small test set and the plan for model upgrades |
Three variants of this question
How do you decide between prompting, retrieval and fine-tuning?
Start with prompting because it is the cheapest to change, and measure the failures. If the model lacks facts, add retrieval of the right documents; if it needs a consistent behavior that prompts cannot hold, consider fine-tuning. Brown et al. (2020) contrast few-shot prompting with fine-tuning that needs thousands or tens of thousands of examples.
How do you know a prompt change actually helped?
Run both versions on the same labeled evaluation set and compare the metrics with their uncertainty. A gap smaller than the standard error of the difference is not evidence, and a single better-looking example is not either.
What do you do when the model ignores an instruction?
Check the failing cases first, then make the instruction more direct, move it to a clearer place, or add an example of the right behavior. Change one thing, re-run the evaluation set, and confirm the fix did not break other cases.
Common mistakes
- "I tried it on an example and it looked great." (wrong) One output says little about all inputs. (right) "I scored it on a labeled set of realistic cases and report the interval."
- "Few-shot prompting trains the model." (wrong) No weights change. (right) "The examples are context at inference time."
- "I rewrite the whole prompt each time." (wrong) You cannot tell what helped. (right) "I change one element per run and re-run the tests."
- "The prompt is done once it works." (wrong) Output varies and models change. (right) "I pin the model version and re-run the tests on every upgrade."
Follow-up questions
What is the difference between prompt engineering and context engineering?
Prompt engineering is writing and organizing the instructions a model receives. Context engineering is curating the whole set of tokens present at inference, including information outside the prompt. Anthropic calls it the natural progression of prompt engineering.
Does few-shot prompting train the model?
No. Few-shot prompting gives demonstrations at inference time with no weight updates. The generative pre-trained transformer (GPT)-3 (generative pre-trained transformer 3) paper by Brown et al. (2020) also reports datasets where few-shot learning still struggles, so it is not a universal fix.
Does chain-of-thought prompting always help?
No. Chain-of-thought prompting is few-shot prompting whose examples include intermediate reasoning steps, in the form input, chain of thought, output. Wei et al. (2022) report larger gains on more complicated problems, and that the ability emerges in sufficiently large models, so measure it before assuming it helps.
How do you keep a prompt stable when the model is updated?
Pin the model version in production, keep the evaluation set under version control, and re-run it before switching. OpenAI's guide recommends pinning model snapshots and building evaluation suites you re-run when you change model versions. More questions of this kind are in prompt engineering interview questions and answers.
How do you evaluate a prompt whose output has no single right answer?
Grade with code checks where you can, such as length or required fields, and otherwise with a rubric applied by humans or by a language model given that rubric. Anthropic's guidance covers code-graded, human-graded and model-graded evaluations. Check any model grader against human labels first; the judge prompt page shows how to write one.
- Answer with a loop: define success, draft, test, change one thing, repeat.
- Success criteria are specific and measurable, such as label accuracy and valid format.
- Judge a prompt on a labeled set with its uncertainty, never on one example.
- Few-shot examples are context at inference time; no weights change.
- Pin the model version and re-run the tests on every upgrade.
Sources
- Define success criteria and build evaluations — Anthropic (Claude Platform Docs), 26 Oct 2026
- Prompt engineering (guide) — OpenAI developer docs, 26 Oct 2026
- Prompting (Refine your prompt) — OpenAI developer docs, 26 Oct 2026
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (arXiv 2201.11903) — arXiv (Wei et al.), 28 Jan 2022
- Language Models are Few-Shot Learners (arXiv 2005.14165) — arXiv (Brown et al.), 28 May 2020
- Effective context engineering for AI agents — Anthropic Engineering, 29 Sep 2025
