How many responses per group (G) does GRPO need?

In this article

1. Before you start
2. Why interviewers ask this
3. Group size (G) from first principles
4. The Staff-level answer
5. The rubric
6. Three variants of this question
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This page belongs to the fine-tuning and alignment learning path and builds on the transformer language models that GRPO trains.

Why interviewers ask this

A candidate who answers "sixty-four, like DeepSeek" or "eight, the default" has memorized a setting, not an idea. Interviewers use this question, reported by candidates in interviews on reinforcement learning for language models, to see whether you know what G controls: the quality of the baseline that replaces a critic, and the generation cost of every prompt.

A weak answer quotes one number. A strong answer gives the range and says G trades a steadier baseline against more sampling. A Staff-level answer adds the failure that decides the choice: a group whose rewards are all identical contributes nothing, so the right G depends on how often the model passes. The rubric below shows the difference line by line.

Group size (G) from first principles

Group size

Group size, written G, is the number of responses that GRPO samples from the current model for each prompt before scoring them, so every response is judged against its own prompt's group rather than against a separate learned critic network.

Dead group

A dead group is a set of G responses whose rewards are all identical, for example all wrong, so the group has no spread, every advantage is zero and the prompt adds no reward signal to that update (only the Kullback-Leibler (KL) penalty, if one is used, still acts).

In plain words

Picture a class of G students answering one question. Each is graded against the class average, not against a fixed answer key. Students above the average get a push up. Students below it get a push down.

The analogy breaks in two places. The average comes from only G students, so a tiny class gives a shaky average. And if every student gets the same mark, the curve says nothing at all.

In one precise sentence, GRPO scores each sampled response by how far its reward sits above or below the mean reward of the group sampled for the same prompt.

Step 1: Sample a group of G responses per prompt

GRPO samples G responses for each prompt and scores each with a reward function.

1 · Sample a group2 · Score and compare3 · Gains flatten4 · Choose GReplay

Step-by-step description

1. Each prompt gets G sampled responses.
2. The group mean is the baseline.
3. A larger G helps less and costs more.
4. Pass rate and budget decide.

• Each prompt gets a group of G sampled responses scored by a reward function.
• The group mean reward replaces a learned critic as the baseline.
• A group with identical rewards gives every response an advantage of zero.
• Gains flatten as G grows while generation cost keeps rising.

GRPO samples a group of G outputs from the old policy for each question, as DeepSeek-AI (2025) describe it. A reward function then scores each output. A critic is a second network, usually as large as the policy, that estimates how good a situation is. Shao et al. (2024) write that GRPO "foregoes the critic model, instead estimating the baseline from group scores."

Input: one prompt. Output: G scored responses. The method is a policy gradient method; see whether GRPO is a policy gradient algorithm. G is not the batch size. The batch holds many prompts, each with its own group, and the third variant below returns to that difference. You now know where the baseline comes from: the group itself.

Step 2: Group-relative advantages, and the group that teaches nothing

The advantage of a response is its reward minus the group's mean reward, divided by the group's standard deviation, as the Hugging Face large language model (LLM) course writes it. Take an illustrative group of four with one correct response, rewarded 1, and three wrong ones, rewarded 0. The mean is 0.25 and the population standard deviation is 0.4330. The correct response gets an advantage of +1.7321 and each wrong one gets −0.5774.¹ A sample standard deviation would rescale these to +1.5 and −0.5 but keep the signs, so check which one your library uses.¹

Now make all four wrong. The mean is 0 and the spread is 0, so every numerator is 0 and every advantage comes out 0, because there is nothing to compare.¹ That is a dead group. Implementations add a small epsilon to the spread so the division stays defined. Only the KL penalty, if one is used, still acts.

Spread also changes the scale. With one correct response among 64, the correct one gets +7.9373 and each wrong one −0.1260, so a rare success is amplified.¹

Step 3: Published settings and diminishing returns

Published group sizes differ, and returns diminish as G grows. Each setting below is the choice of one run, not a constant of the method.

The v2 version of the R1 paper states that DeepSeek-R1-Zero training samples 16 outputs for each question (DeepSeek-AI, 2025, v2). The same paper also samples 16 responses per question to measure accuracy on the American Invitational Mathematics Examination (AIME) benchmark, so 16 appears both as a training group and as an evaluation setting. In a 2025 study comparing G = 2, 4 and 8, a larger G gave better accuracy and lower KL divergence at higher computation cost.

In a group-size ablation of a 2026 study, accuracy was 64.1 percent at n=8, 65.9 at 16, 66.0 at 32 and 66.2 at 64, so going from 16 to 64 adds 0.3 points.¹ The paper's own wording is that marginal returns diminish significantly as n increases from 16 to 64. Gains flatten as G grows while generation cost keeps rising.

Step 4: Match G to the pass rate and the sampling budget

The pass rate is the share of a prompt's sampled responses that earn full reward. At an illustrative pass rate of 0.06, a group of 4 has every response wrong 78.07% of the time, 60.96% at 8, 37.16% at 16, and a group of 64 is all wrong only 1.91% of the time.¹ All-correct groups are dead too, but at this pass rate their chance is negligible.

Text description of the diagram

• At a pass rate of 0.06, a group of 4 has every response wrong 78.07% of the time.
• At that pass rate, a group of 64 is all wrong only 1.91% of the time.

Bigger groups are not free. With an illustrative budget of 1,024 completions per step, G=4 allows 256 prompts and G=64 allows 16.¹ At pass rate 0.06 the expected prompts with any signal are about 56, 50, 40 and 16 for G of 4, 8, 16 and 64.¹ That count ignores how noisy a four-sample baseline is, so smaller is not simply better. On a competition-mathematics benchmark with Qwen2.5-Math-7B and a fixed sampling budget, the best G was 64 at budgets of 1,024 and 2,048 and 128 at 4,096. In each case the winner beat the next-best G by under one point, and every result is from a single run (arXiv 2603.01162). Measure on your task.

The Staff-level answer

What makes it Staff-level: it names the dead-group rate as the measurement that drives the choice and weighs it against the fixed budget, instead of quoting one paper's number.

The rubric

Three variants of this question

What is the minimum group size for GRPO?

A group of one has a mean equal to its own reward, so every advantage is zero; this follows from the formula, not from a paper. A group of two is the smallest with a comparison, but its mean and spread come from only two samples, so they are noisy by construction. In a 2025 study comparing G = 2, 4 and 8, a larger G gave better accuracy and lower KL divergence at higher computation cost.

What group size should you use on a small GPU budget?

Start from the undefined default of eight. A larger group costs more generation per prompt (Comparative study, 2025), so lower G only if you measure that the dead-group rate stays low; at a low pass rate a small G produces many dead groups.

Is the group size the same as the batch size?

No. G is responses per prompt; the batch holds many prompts' groups, so completions per step equal prompts times G. DeepSeekMath reports 64 outputs per question and a batch size of 1,024 (Shao et al., 2024); if that batch counts completions, it holds 16 prompts.¹ TRL ties the effective batch size to num_generations, so confirm on its documentation that your batch divides evenly by G.

Common mistakes

• "G has to be sixty-four." (wrong) That was one paper's setting. (right) "DeepSeekMath used sixty-four, other recipes use sixteen or eight, and I choose by pass rate and budget."
• "G is the batch size." (wrong) G is only the responses per prompt. (right) "The batch holds many prompts, each with its own group of G."
• "A bigger group is always better." (wrong) Gains flatten while cost rises. (right) "Bigger helps with diminishing returns, and under a fixed budget it means fewer prompts."
• "I ignore groups where everything fails." (wrong) They quietly waste generation. (right) "I track the share of dead groups and change G or prompt difficulty if it is high."

Follow-up questions

How do you handle prompts where every response fails?

Measure the share of dead groups first. If it is high, raise G, make the training prompts easier, or add partial credit to the reward function, because an all-fail group contributes zero advantage.

Does GRPO still need a critic?

No. DeepSeekMath replaces the learned critic with the average reward of the sampled group, which saves training resources.

Did the DeepSeek-R1 paper train with 16 responses per group?

Yes: 16 per question in training (DeepSeek-AI, 2025, v2). The paper also samples 16 responses per question for its AIME accuracy plot, so the same number serves as a training group and as an evaluation setting.

How does GRPO compare to DPO?

GRPO samples fresh groups and scores them with a reward, while direct preference optimization (DPO) learns from fixed preference pairs; the GRPO versus DPO comparison covers the trade.

How does GRPO differ from RLHF?

GRPO changes how the baseline is estimated, using a group mean instead of a critic. The comparison of DPO and reinforcement learning from human feedback (RLHF) and the guide to fine-tuning an LLM place that choice inside the full training pipeline.

Sources

1. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (arXiv 2402.03300) — arXiv (Shao et al.), 5 Feb 2024
2. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (arXiv 2501.12948v2) — arXiv (DeepSeek-AI), 4 Jan 2026
3. TRL GRPO Trainer and GRPOConfig reference — Hugging Face documentation, 26 Oct 2026
4. TRL Paper Index — Hugging Face documentation, 26 Oct 2026
5. Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statistic (arXiv 2603.01162) — arXiv, 1 Mar 2026
6. Comparative Analysis and Parametric Tuning of PPO, GRPO, and DAPO for LLM Reasoning Enhancement (arXiv 2512.07611) — arXiv, 1 Dec 2025
7. Deep Dense Exploration for LLM RL via Pivot-Driven Resampling (arXiv 2602.14169) — arXiv, 1 Feb 2026
8. Understanding the DeepSeek R1 Paper (LLM Course, chapter 12) — Hugging Face LLM Course, 26 Oct 2026