Few shot vs zero shot prompting: when does each win?

In this article

1. Before you start
2. Why interviewers ask this
3. What each one is
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This page is part of the prompt engineering learning path.

Why interviewers ask this

A wrong prompting choice shows up in production as a drifting classifier, a bloated bill or a broken output format, and interviewers ask about few-shot against zero-shot to see whether a candidate can prevent that. The question separates people who treat a prompt as a tested component from people who paste examples until the output looks right.

A weak answer says "few-shot is better because examples help." A strong answer defines both, says that examples cost input tokens on every call, names the sensitivity to example choice and order, and describes how to test the choice on held-out inputs. A Staff-level answer adds where examples stop helping, such as multi-step reasoning, and how to keep the examples under version control. Candidates report prompting questions like this in applied-AI interviews; the wider set is in prompt engineering interview questions and answers.

What each one is

Zero-shot prompting

Zero-shot prompting is a way of using a language model in which the prompt contains an instruction in natural language and the new input but no worked examples, so the model must rely on what it learned in training to work out both the task and the output format.

In-context learning

In-context learning is the ability of a language model to perform a new task from instructions and examples placed in its prompt alone, without any gradient update, which is why few-shot prompting changes the output but does not change the model.

Zero-shot prompting in plain words

A demonstration is a worked input-and-output example placed in a prompt to show the model the task. Zero-shot prompting hands the model a task description and the new input and nothing else. The paper on Generative Pre-trained Transformer 3 (generative pre-trained transformer (GPT)-3) defines the condition as one where "no demonstrations are allowed and only an instruction in natural language is given to the model". One-shot is the same with exactly one demonstration.

Think of giving a capable new colleague a written task and asking for the result with no sample to copy. The analogy breaks in one place: the model cannot ask a clarifying question, so every ambiguity in the instruction, such as the exact output format, is a place where it guesses.

The strength of zero-shot is a short prompt that is cheap to send and easy to maintain. The weakness is that the model decides the format and the boundaries of the labels for itself.

Few-shot prompting in plain words

Few-shot prompting adds worked examples to the same prompt. In the GPT-3 paper, few-shot means allowing as many demonstrations as will fit into the context window, typically 10 to 100 (Brown et al., 2020). In a chat application programming interface (API), the examples are written as example user and assistant messages (Microsoft Learn, accessed October 2026).

Think of showing the colleague three finished forms before asking for the fourth. The analogy breaks in one place: the colleague keeps the skill, while the model forgets everything when the call ends. The same paper applies GPT-3 "without any gradient updates or fine-tuning", so the weights never change.

That is why few-shot is a prompting technique and not a training technique: the examples are just more text in the context window.

Why can few-shot swing from near chance to near state-of-the-art?

Few-shot results can be unstable, with accuracy moving from near chance to near state-of-the-art when only the examples change. Zhao et al. (2021) show that the prompt format, the choice of training examples and even their order can cause this.

They trace it to bias: language models lean toward certain answers, such as those placed near the end of the prompt or common in the pre-training data. To measure the bias, they ask for a prediction on a content-free input such as "N/A", then adjust the scores so that prediction is uniform. In the arXiv version, this contextual calibration improves average accuracy by up to 30.0% absolute and reduces variance across prompt choices.

The practical lesson is that a few-shot prompt is a setting you must test, not a fixed good. The follow-up questions below list ways to stabilize it.

Do the labels in the examples have to be correct?

The labels in few-shot examples matter less than the format they show, on many classification tasks. Min et al. (2022) report that randomly replacing the labels in the demonstrations barely hurts performance on a range of classification and multiple-choice tasks, consistently over 12 different models including GPT-3.

What the examples do supply, they say, is the label space, the distribution of the input text and the overall format of the sequence. An example therefore mostly tells the model what kind of answer to give and how to write it.

Do not stretch the result. It covers classification and multiple-choice tasks, not tasks where the answer must be worked out, and later work found that larger models follow flipped labels in the examples more closely (Wei et al., 2023). Keep labels correct anyway; wrong labels in a reviewed prompt are a maintenance trap.

How the two differ side by side

Zero-shot and few-shot prompting differ in what the prompt holds, what it costs per call and how stable the output is; neither changes the weights.

Both columns leave the weights alone because both are in-context learning, as defined above.

Worked example

Two quantities show what the choice does: one measured accuracy gain from the literature and one computed token cost.

Brown et al. (2020) report that GPT-3 reached 64.3% accuracy on the TriviaQA benchmark zero-shot, 68.0% one-shot and 71.2% few-shot. One example adds 3.7 points and the few-shot setting adds 6.9 points over zero-shot.¹ This is one benchmark and one model, so read it as an example of a modest gain, not a rule.

Text description of the diagram

• GPT-3 reached 64.3% accuracy on TriviaQA zero-shot, 68.0% one-shot and 71.2% few-shot.
• The few-shot setting adds 6.9 points over zero-shot.

Now a prompt you could write yourself, a three-example sentiment classifier (illustrative):

Classify the review as Positive, Neutral or Negative.

Review: The battery died in two days.
Label: Negative

Review: It works, nothing special.
Label: Neutral

Review: Setup took a minute and the sound is great.
Label: Positive

Review: {new_review}
Label:

The examples cover all three labels, show the exact output format, and end with "Label:" so the model completes one word. Because the last example is Positive, the recency bias described above could nudge answers toward that label, so you would shuffle the example order and compare results.

Suppose the instruction is 60 tokens, the new input 30 and each example 40 (all illustrative). Zero-shot sends 90 input tokens per call; the prompt above with three examples sends 210, which is 2.33 times as many.¹

At five examples, a million calls send 200,000,000 extra input tokens.¹ Whether that is worth the format control depends on the task, which is the decision below.

Trade-offs and edge cases

When should you pick each one?

Zero-shot wins when the task is common, the instruction is clear and token cost matters; few-shot wins when the output format is unusual, the label set is specific or a house style must be matched. Start with zero-shot and check the output on real inputs. Add a few examples only when the format or the labels come out wrong. Shuffle the example order and re-test before you trust the result.

The same logic applies to a judge prompt, where a few graded examples anchor the scale; see how to write a good judge prompt.

Text description of the diagram

• Start with zero-shot and check the output on real inputs.
• Add a few examples only when the format or the labels come out wrong.
• Shuffle the example order and re-test before you trust the result.

Does few-shot fix multi-step reasoning?

Standard few-shot prompting is not enough for multi-step reasoning. Wei et al. (2022) define a chain of thought as "a series of intermediate natural language reasoning steps that lead to the final output", and show that it can be elicited in sufficiently large models by including such steps in the few-shot exemplars. The gain appears at large scale, not in small models.

The same paper stresses sensitivity to exemplars, which echoes the instability above. A zero-shot variant exists: Kojima et al. (2022) reported that appending a phrase such as "Let's think step by step" can trigger step-by-step reasoning without examples.

Do few-shot examples still help with newer models?

Few-shot examples help less predictably with newer models, and the direction depends on the model type. The DeepSeek-R1 report (2025) says few-shot prompting degraded the performance of that reasoning model and recommends a zero-shot prompt that states the problem and the output format. Treat that as one model family's finding and test your own model.

In the other direction, long context windows allow many-shot prompting. Agarwal et al. (2024) study in-context learning with hundreds of examples and report gains over a handful. The practical rule is unchanged: measure on a held-out set, because the best number of examples, from none to many, depends on the model and the task.

Other edge cases

Examples compete for the context window with the input, so a long document leaves little room for them. Examples also go stale: when the task changes, the examples must change too, which is why teams keep them in one reviewed block. The model can copy surface features of the examples, such as their length, so vary them on purpose.

How to answer it in an interview

The spoken structure: define both, say weights do not change, give the cost, then the test you run before shipping. Your own approach to prompting ties these together; see what's your approach to prompt engineering.

Common mistakes

• "Few-shot trains the model on my examples." (wrong) No gradient update happens and the weights stay the same. (right) "Few-shot is in-context learning: the examples are text in the prompt."
• "Every label in my examples has to be perfect for the model to work." (wrong) On classification and multiple-choice tasks, random labels barely hurt in one study. (right) "Format, label space and input distribution matter most there, but I still keep labels correct."
• "More examples are always better." (wrong) Results depend on which examples you choose and their order, and some reasoning models do worse with examples. (right) "I add examples while a held-out set improves and stop when it does not, and I re-test when the model changes."
• "I put all the Positive examples last." (wrong) Models lean toward answers near the end of the prompt. (right) "I mix and shuffle the label order, then re-test."
• "Few-shot will fix my math errors." (wrong) Standard examples do not fix multi-step reasoning. (right) "I show the reasoning steps in the examples, on a model large enough to use them."

Follow-up questions

Few-shot prompting gives different answers on near-identical inputs. How do you stabilize it?

Fix the example order and test a few shuffles, balance the labels, and avoid long runs of the same label: Zhao et al. (2021) report that majority-label and recency biases push GPT-3 toward certain answers and help explain the high variance across examples and orderings. Contextual calibration with a content-free input such as "N/A" improved accuracy by up to 30.0% absolute and reduced variance in Zhao et al.. Then measure on a held-out set, not on a few hand-picked inputs.

What is the difference between few-shot learning and few-shot prompting?

Few-shot learning in classic machine learning means training a model from a few labeled instances. Few-shot prompting puts examples in the prompt at inference time and trains nothing, as in Brown et al. (2020).

How many examples should a few-shot prompt contain?

No source gives a best number. The GPT-3 paper allowed as many as fit in the context window, typically 10 to 100 (Brown et al., 2020), but a practical start is a small handful, increased only while a held-out set improves.

Can you get step-by-step reasoning without examples?

Yes, in the zero-shot variant reported by Kojima et al. (2022), a phrase such as "Let's think step by step" is appended to the prompt. Few-shot chain-of-thought instead shows worked reasoning in the examples (Wei et al., 2022).

How do you give few-shot examples to a chat model?

Write them as example user and assistant message pairs after the system message (Microsoft Learn, accessed October 2026). When the output must be machine-readable, for example JavaScript Object Notation (JSON), pair examples with the checks in how to get reliable JSON from an LLM.

Sources

1. Language Models are Few-Shot Learners (arXiv 2005.14165) — arXiv (Brown et al.), 28 May 2020
2. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? (arXiv 2202.12837) — arXiv (Min et al.), 25 Feb 2022
3. Calibrate Before Use: Improving Few-Shot Performance of Language Models — PMLR, ICML 2021 (Zhao et al.), 18 Jul 2021
4. Calibrate Before Use: Improving Few-Shot Performance of Language Models (arXiv 2102.09690) — arXiv (Zhao et al.), 19 Feb 2021
5. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (arXiv 2201.11903) — arXiv (Wei et al.), 28 Jan 2022
6. Prompt engineering techniques — Microsoft Learn, 26 Oct 2026
7. Large Language Models are Zero-Shot Reasoners (arXiv 2205.11916) — arXiv (Kojima et al.), 24 May 2022
8. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (arXiv 2501.12948) — arXiv (DeepSeek-AI), 22 Jan 2025
9. Many-Shot In-Context Learning (arXiv 2404.11018) — arXiv (Agarwal et al.), 17 Apr 2024
10. Larger language models do in-context learning differently (arXiv 2303.03846) — arXiv (Wei et al.), 7 Mar 2023