Prompt engineering: few-shot, chain-of-thought and the interview questions

In this article

1. Start here: the learning path
2. Why prompt engineering works
3. The anatomy of a prompt
4. Few-shot prompting: what examples buy and what they cost
5. Chain-of-thought prompting: the evidence and its limits
6. Prompting is one lever, not the only one
7. Treat the prompt as versioned code
8. The topics in this path
9. Questions interviewers ask on prompt engineering
10. All 5 guides in Prompt engineering
11. Follow-up questions
12. Sources

Start here: the learning path

Few-shot prompting

Few-shot prompting is a technique that places a handful of worked input and output examples inside the prompt so the model imitates the pattern without any change to its weights.

Chain-of-thought prompting

Chain-of-thought prompting is a technique that gets a model to write out intermediate reasoning steps before its final answer, either by showing worked examples or by asking it to think step by step.

Context engineering

Context engineering is the wider practice of deciding everything that enters a model's context window for a request, including instructions, examples, retrieved documents and conversation history, of which the wording of the prompt is one part.

The path runs from measurement to technique: what a good answer is, how a prompt is built, what examples and reasoning prompts buy, how to get machine-readable output, and how to grade a model with another model. The figure shows the working loop that the rest of the page fills in.

Text description of the diagram

• Define what a good answer is before writing the prompt.
• An eval set scores each prompt version on the same cases.
• The version that scores best is committed with its score.

If you want a skill order rather than a page order, start with the measurement and add levers after it: define what a good answer is and build a small eval set first, then learn the anatomy of a prompt, examples, reasoning prompts, structured output and last judge prompts. Anthropic's guidance puts the same two things first, a clear definition of success and a way to test against it.

1. How do you get reliable JSON from an LLM?
2. What's your approach to prompt engineering?
3. How do you write a good judge prompt?

Why prompt engineering works

Prompt engineering works because a language model continues the text it is given, so the text shapes the continuation. The Generative Pre-trained Transformer (GPT) paper on GPT-3 (Brown et al., 2020) showed that a 175-billion-parameter model could perform tasks given only a text description and a few examples, "without any gradient updates or fine-tuning". That result is why the examples live in the prompt and not in training. How the model produces tokens one at a time is explained in the Transformers guide, and how those tokens are served in the LLM inference guide.

The discipline has an engineering half that interviews increasingly test. Anthropic's documentation (read October 2026) assumes you already have a clear definition of success, a way to test against it empirically, and a first draft prompt. Without the first two, you cannot tell whether a rewrite helped.

The anatomy of a prompt

A prompt has four parts: the standing instruction, the examples, the input for this request, and the output format. The OpenAI documentation (read October 2026) separates the instruction from the input by message role, with developer messages carrying the application's rules and taking priority over user messages that supply inputs.

A good instruction states the role, the task, the audience and the failure to avoid. A good format section names the exact fields and gives one valid example. Mark the input off with clear delimiters, such as Extensible Markup Language (XML) tags, so the model can tell your instructions from the data. Anthropic says tagging each type of content "reduces misinterpretation". It reduces confusion but does not stop instructions hidden in the input from being followed, which is called prompt injection, so untrusted input needs its own checks.

Where the input goes differs by vendor. The OpenAI guide says context is "usually best positioned near the end of your prompt". Anthropic's best practices (read October 2026) say to place long documents and inputs near the top, above the query, and report that queries at the end can improve response quality by up to 30 percent in their tests. Check the guidance for the model you use, and measure.

Few-shot prompting: what examples buy and what they cost

Few-shot prompting buys consistency of format and style, and it costs input tokens on every request. The OpenAI documentation says to include a handful of input and output examples that cover a diverse range of inputs.

The cost is simple arithmetic. With illustrative values of eight examples of 150 tokens each and one million requests a day, the examples add 1,200 input tokens to each request and 1.2 billion input tokens a day ¹. The numbers are assumptions, not measurements, but the shape is real: examples are a recurring cost, so a staff-level answer asks whether the same behaviour can come from a shorter instruction. A fixed example block is also reusable content, and the same OpenAI guide says to keep content you expect to reuse "at the beginning of your prompt" so that prompt caching can apply.

Examples also bias the model, and their order is a hidden input. Zhao et al. (2021) found that the order of training examples can move accuracy "from near chance to near state-of-the-art", and traced the instability to a bias toward answers common in pre-training data or placed near the end of the prompt. Lu et al. (2022) found that some orderings are far better than others, that the effect appears across model sizes, and that a good ordering for one model may not carry over to another. A label bias like this is one documented reason a few-shot answer changes when the examples are reordered, so test several orderings.

Chain-of-thought prompting: the evidence and its limits

Chain-of-thought prompting raised accuracy sharply on arithmetic benchmarks in the original studies. Wei et al. (2022) showed that eight chain-of-thought examples let a 540-billion-parameter model reach state-of-the-art accuracy at the time on the Grade School Math 8K (GSM8K) maths word-problem benchmark. Kojima et al. (2022) then showed that no worked examples were needed (the phrase alone, in a two-stage zero-shot prompt): adding the phrase "Let's think step by step" raised accuracy of text-davinci-002 on MultiArith from 17.7% to 78.7% and on GSM8K from 10.4% to 40.7%.

Text description of the diagram

• MultiArith accuracy rose from 17.7% to 78.7%, a gain of 4.45 times.
• GSM8K accuracy rose from 10.4% to 40.7%, a gain of 3.91 times.
• Both results are for text-davinci-002 in 2022.

Those are gains of 4.45 times (61.0 points) and 3.91 times (30.3 points) ¹. The boundary is as important as the gain: both results are for 2022 models, and the OpenAI documentation now distinguishes reasoning models, which plan across steps from a high-level goal, from GPT models, which are faster and cheaper but benefit from more explicit instructions. Whether an explicit "think step by step" still helps depends on the model, so test it.

Prompting is one lever, not the only one

Prompting is often the cheapest lever to try, but not every failing evaluation is a prompting problem. Anthropic's overview notes that you can sometimes improve latency and cost more easily by choosing a different model, so the right next step depends on which measure is failing. A sensible order is to define the success measure, try a clearer prompt, and add examples or retrieved context. Where the failing measure is latency or cost, try a different model early, and consider training a model only when those levers have run out.

This is also where the term context engineering enters. Anthropic's post on context engineering for agents (September 2025) frames it as the wider set of strategies for curating everything the model sees, of which the wording of the prompt is one part. See the AI agents guide for where prompts sit inside larger systems.

Treat the prompt as versioned code

A production prompt is a versioned artifact with tests. The OpenAI documentation recommends pinning production applications to specific model snapshots, building evals that track performance as prompts and model versions change, and keeping prompts in code with representative fixtures and tests. An eval is a repeatable test set with a scoring rule, so a change either raises the score on the same cases or it does not.

In practice this means a prompt change is reviewed like a code change: the old and new versions sit side by side, both run against the same eval set, and the winner is committed with the score that justified it. Pinning a snapshot matters for the same reason, since an unpinned model can change under a prompt that was tested against the old one. LLM observability covers the monitoring side, which catches the failures an eval set did not.

The topics in this path

Getting reliable JSON from an LLM

Reliable structured output is layered. First, state the schema in the request, so the model knows the fields and types expected. Second, use the provider's structured-output mode or constrained decoding, which forces the syntax to be valid. Third, validate every reply against the schema and retry or repair on failure, because a constrained reply can still be a refusal or be cut off before it ends.

Each layer catches a different failure: the schema steers meaning, the constraint guards syntax, and validation guards everything the first two miss. A program that parses JavaScript Object Notation (JSON) from a model without the third layer will eventually crash on a bad reply. The dedicated page on getting reliable JSON from an LLM walks through each layer with the failure it prevents. The interview point is that "ask nicely" is not a plan.

Chain-of-thought: the follow-up questions

Interviewers push on two points once you have named the technique: whether the written reasoning is the real reason for the answer, and what it costs. The reasoning text is extra output the user pays for in time and tokens, so it adds latency to every request and is only worth buying where accuracy improves. On the first point, Turpin et al. (2023) found that chain-of-thought explanations "can systematically misrepresent the true reason for a model's prediction", describing them as plausible yet misleading.

That means the visible steps are useful to read and to grade, but they are not a proof of how the model reached its answer. State both points in one breath, and add that you would measure the technique on your own task and model before keeping it. The figures earlier on this page come from 2022 models, so the honest answer includes a test on the model in use.

Designing a prompt process and a judge prompt

An interviewer who asks for your approach to prompt engineering wants a process, not a trick: define success, build a small eval set, write the simplest prompt, change one thing at a time, and version the winner.

A judge prompt asks a model to grade another model's output against a rubric. Its core rule is a rubric with explicit levels, plus a check of the judge's scores against human labels so that you know whether the judge can be trusted.

Questions interviewers ask on prompt engineering

The full answered list is the prompt engineering interview questions page. As a rough way to sort them, which is a judgement and not a measurement, they tend to be about explaining a technique, designing a prompt for a business task, making the output machine-readable, and evaluating or versioning a prompt. A longer answered list is in the AI engineer question list, and agent-specific questions are in agentic AI interview questions.

Follow-up questions

How do large language models follow a prompt?

A model does not run a prompt like a program; it predicts the next token from all the text so far, and instruction-tuned models have learned to treat that text as a request. The mechanism is covered in the Transformers guide.

What is context engineering, and how is it different from prompt engineering?

Context engineering decides everything that enters the model's context window, including retrieved documents and history, while prompt engineering concerns the wording of the instructions and examples within it. Usage of the two terms overlaps, so define yours when you use one, as in the AI agents guide.

How do I prepare for a prompt engineering interview?

Practise the process answer, build one small eval set for a task you know, and be ready to explain few-shot and chain-of-thought with their costs. The prompt engineering interview questions page is the practice list.

How do I become a prompt engineer?

Build evidence of the engineering half: a task, an eval set, a prompt history that shows measured improvement, and a note on what you tried that did not work. A prompt without a test proves little, and the AI engineer question list shows what else is asked.

Why does a few-shot prompt give different answers when the examples are reordered?

Example order and label balance act as hidden inputs, and published studies found large swings from order alone. Fix the order, balance the labels, and measure variation across several orderings, as set out in the prompt engineering interview questions.

Sources

1. Large Language Models are Zero-Shot Reasoners (arXiv 2205.11916 v4) — arXiv (Kojima et al.), 29 Jan 2023
2. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (arXiv 2201.11903 v6) — arXiv (Wei et al.), 10 Jan 2023
3. Language Models are Few-Shot Learners (arXiv 2005.14165 v4) — arXiv (Brown et al.), 22 Jul 2020
4. Prompt engineering overview — Anthropic documentation (page undated; read 2026-10-24), 24 Oct 2026
5. Prompt engineering guide — OpenAI documentation (page undated; read 2026-10-24), 24 Oct 2026
6. Prompting best practices — Anthropic documentation (page undated; read 2026-10-24), 24 Oct 2026
7. Calibrate Before Use: Improving Few-Shot Performance of Language Models (arXiv 2102.09690 v2) — arXiv (Zhao et al.), 10 Jun 2021
8. Fantastically Ordered Prompts and Where to Find Them (arXiv 2104.08786 v2) — arXiv (Lu et al.), 3 Mar 2022
9. Language Models Don't Always Say What They Think (arXiv 2305.04388 v2) — arXiv (Turpin et al.), 9 Dec 2023
10. Effective context engineering for AI agents — Anthropic, 29 Sep 2025