Fine-Tuning & LLM Training

How do reasoning models work?

By Interview Clozo · Published 27 October 2026 · Last verified 26 October 2026 · 11 min read

A reasoning model is a language model trained with reinforcement learning (RL) to write a long chain of reasoning tokens before its visible answer, which is how reasoning models work: extra thinking at answer time buys accuracy. The follow-up interviewers ask next is the bill: those tokens are charged as output and can run out before any answer appears, so thinking longer is not free and does not always pay.

In this article

Before you start
Why interviewers ask this
Reasoning models from first principles
The Staff-level answer
The rubric
Three variants of this question
Common mistakes
Follow-up questions
Sources

Before you start

This page is one step of the how large language models are trained learning path.

Why interviewers ask this

Reasoning models change two things a system designer must budget: the quality of hard answers and the number of tokens paid for each one. Candidates report questions on this topic such as when to use supervised fine-tuning against reinforcement fine-tuning, and what to consider about the reasoning token budget, so the topic sits where training and serving meet.

A weak answer says "the model thinks step by step" and stops. A strong answer says where the thinking comes from (a training signal, not a prompt), who checks it, and what it costs at serving time. A Staff-level answer adds the limit: extra thinking helps only when the base model already solves the problem some of the time, and a cap on tokens can cut thinking off before the answer. The rubric below shows the difference line by line.

Reasoning models from first principles

Reasoning model: A reasoning model is a language model trained to generate a long sequence of intermediate steps, called reasoning tokens, before it writes the visible answer, so that harder problems get more computation than easy ones.
Reasoning token: A reasoning token is a token the model writes while working through a prompt, before the visible answer; some APIs (OpenAI's) hide these tokens and others (DeepSeek-R1) return them, but they fill the context window and are billed as output.
Verifiable reward: A verifiable reward is a training signal computed by a program that checks the final answer, such as a unit test for code or an answer match for maths, so no human has to grade each attempt.
Test-time compute: Test-time compute is the computation a model spends while answering one prompt, such as writing a longer reasoning trace or sampling several attempts, as opposed to computation spent during training.

In plain words

Think of an exam with scratch paper. A student who must write only the final answer guesses on hard questions. A student allowed to fill the paper with working checks each step and catches errors. The reasoning tokens are the scratch paper. The analogy breaks in two places: the model's paper is billed by the token, and a training process, not a teacher, taught it what good working looks like. Longer working on a hard question also means more tokens to pay for, which is why the cost step below matters.

In one precise sentence, a reasoning model is rewarded in training for correct final answers, and that pressure makes it write longer, self-checking working before it answers.

Step 1: Chain-of-thought prompting showed that writing steps first improves the answer

Writing out intermediate steps is called chain of thought. Wei et al. (2022), in "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models", showed it with prompts: eight worked examples in the prompt were enough for a 540-billion-parameter model to reach state-of-the-art accuracy on a grade-school maths benchmark. In the paper's Figure 1, the prompt's worked exemplar notes that two cans of three balls make six, then writes 5 + 6 = 11; the model copies that style on a new question.

Input: a question. Output: the steps, then the answer. The prompt supplied the habit. A reasoning model moves the habit from the prompt into the model's training; that framing is this page's own synthesis, not a claim by Wei et al. You now know what the model writes; the next step is how it learns to write it.

Step 2: Reinforcement learning with a checkable reward teaches the habit

DeepSeek reports that reasoning abilities can be incentivized through pure reinforcement learning, without human-labelled reasoning trajectories, on verifiable tasks such as mathematics, coding competitions and science. The model samples attempts, a program checks each final answer (the verifiable reward), and attempts that pass are made more likely. Patterns such as self-reflection, verification and strategy adaptation emerged in that setting.

Input: a prompt and a checker. Output: a policy that writes more useful working. Group-based update rules do this sampling efficiently; see group relative policy optimization (GRPO). Training with only human preference scores is a different route, compared with direct preference optimization in direct preference optimization (DPO) vs reinforcement learning from human feedback (RLHF).

Reasoning tokens are hidden in OpenAI's API but still occupy the context window and are billed as output tokens. OpenAI's guide says a request may use anywhere from a few hundred to tens of thousands of them. A Microsoft documentation example usage object shows 75 input tokens and 1,186 output tokens, of which 1,024 are reasoning. The visible answer is then 162 tokens, reasoning is 86.3% of the output, and billed output is 7.32 times what the user reads in that one example.¹

The cap matters too: max_output_tokens covers reasoning, visible and non-visible formatting tokens, so a low cap can end a response before any visible text while input and reasoning tokens are still charged. Illustratively, with a cap of 1,000 and a problem that needs 4,000 reasoning tokens, the visible answer is empty.¹ OpenAI recommends reserving at least 25,000 tokens when you start. An effort setting trades thinking for cost; for example, generative pre-trained transformer (GPT)-5 supports minimal, low, medium and high.

Step 4: More samples and longer thinking help until the base model is too weak

Test-time compute can also be spent on sampling several attempts and keeping the best, called best-of-N, with a checker to pick. If one attempt is right with probability 0.4 (illustrative) and the checker is perfect, the chance that at least one of eight is right is 0.9832.¹ Real checkers err, so this is an upper bound.

Snell et al. (2024) study searching with a process reward model (PRM), a verifier that scores each step, and adaptive revision, on a PaLM 2 base model. They report a compute-optimal strategy that improves efficiency by more than 4x over best-of-N, and that a smaller model with non-trivial success can beat a 14x larger one at matched compute. The boundary, in the paper's body (Section 7): on the hardest questions or at high inference load, pretraining a larger model was the more effective use of compute. These findings come from that setting, not from reinforcement-learning-trained reasoning models.

Reasoning models spend billed tokens thinking first, and a checker's reward during training is what teaches them to do it.CC BY 4.0 — source: How do reasoning models work?

Step-by-step description

Reasoning tokens come before the answer.
A checker rewards correct answers.
Thinking is billed and capped.
Extra attempts raise the odds.

Reasoning tokens come before the visible answer.
A program checks each final answer, the verifiable reward.
Reasoning tokens are billed as output.
Sampling several attempts and keeping the best is called best-of-N.

In this illustrative case more samples lift the chance of one correct answer fast, but only a checker can pick it.CC BY 4.0 — source: How do reasoning models work?

Text description of the diagram

With a 0.4 chance per sample, eight samples give 0.9832.
A perfect checker is assumed, so this is an upper bound.

Knowing it is half the job. Saying it is the other half.

Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.