In this article
Foundations
What is a large language model?
A large language model is a neural network trained on large amounts of text to predict the next token, which lets it generate text and perform many language tasks. A token is a piece of text, such as a word or part of a word, that the model reads and writes. Brown et al. describe a generative pre-trained transformer (GPT) called GPT-3 as an autoregressive language model with 175 billion parameters (Brown et al., 2020).
Why it is asked: it checks that you can define the term without hype. Read the full explanation: the transformers hub.
Why is an LLM important?
One pre-trained model can be pointed at many tasks using only text, where older systems needed a separate model per task. Brown et al. (2020) report that GPT-3 performs strongly on translation, question answering and cloze tasks with tasks and examples given purely as text, with no gradient updates. The cost is serving and checking a very large model, so importance has a price.
Why it is asked: it tests whether you can name the benefit and its cost. Read the full explanation: how transformers generate text.
What is a token, and why does tokenization matter?
A token is the unit a model reads and writes, produced by a tokenizer that splits text using a vocabulary learned from data. Byte pair encoding builds that vocabulary by repeatedly merging the most frequent pair of symbols. Tokenization matters because context limits and cost are counted in tokens, and rare words or other languages can split into many tokens, which makes them costlier.
Why it is asked: it separates people who know the input is not words. Read the full explanation: byte pair encoding.
What is a transformer, and why did it replace recurrent networks?
A transformer is a neural network built around attention instead of recurrence. Vaswani et al. introduced it as a model based solely on attention, which trains faster and parallelises better than earlier encoder-decoder models; it reached 28.4 on the bilingual evaluation understudy (BLEU) translation score on the Workshop on Machine Translation (WMT) 2014 English-to-German task (Vaswani et al., 2017). Parallel training over a whole sequence is what let models scale.
Why it is asked: it is the standard opener for the architecture. Read the full explanation: transformer architecture.
What is self-attention?
Self-attention lets each token compute a weighted mix of the value vectors of the tokens it can see. Each token produces a query, a key and a value; the weights are a softmax of the query-key dot products divided by the square root of the key dimension, so scores are scaled and normalised (Vaswani et al., 2017, §3.2.1). A decoder-only LLM applies a causal mask, so a token attends only to itself and earlier tokens. It is how a token like "it" can draw on the noun it refers to.
Why it is asked: it tests whether you can describe the mechanism, not only the name. Read the full explanation: self-attention.
What is a context window, and what limits it?
A context window is the maximum number of tokens a model can attend to in one request, including both the prompt and the generated text. Standard self-attention compares every token with every other, so work grows with the square of the length, and the memory that stores earlier tokens grows with length. Quality can also fall before the limit is reached.
Why it is asked: it links the architecture to cost. Read the full explanation: context rot.
What is an embedding?
An embedding is a list of numbers that represents a token, a sentence or a document so that similar meanings sit close together. Models turn tokens into embeddings at the input, and separate embedding models turn whole texts into vectors for search. Closeness is usually measured by cosine similarity or a dot product.
Why it is asked: embeddings underlie search and retrieval. Read the full explanation: what an embedding is.
Training and alignment
What is the difference between pre-training and fine-tuning?
Pre-training is the first, expensive stage in which a model learns to predict the next token on a very large text corpus. Fine-tuning continues training on a smaller, targeted data set to change behaviour or style. In the InstructGPT recipe the authors trained GPT-3 further, first with supervised learning on labeler-written demonstrations and then with reinforcement learning from human feedback (Ouyang et al., 2022).
Why it is asked: it tests whether you know where knowledge and behaviour come from. Read the full explanation: fine-tuning versus prompt engineering.
What are scaling laws?
Scaling laws are empirical equations that predict a language model's loss from its size, data and compute. Kaplan et al. (2020) found that cross-entropy loss follows a power law in model size, dataset size and training compute, with some trends spanning more than seven orders of magnitude, with architecture details such as width or depth mattering little within a wide range. They let teams forecast a run before paying for it.
Why it is asked: it shows whether you reason about training cost. Read the full explanation: the transformers hub.
What does compute-optimal training mean (Chinchilla)?
Compute-optimal training spends a fixed compute budget on the model size and data amount that give the lowest loss. Hoffmann et al. (2022) trained over 400 models and concluded that size and training tokens should scale in equal proportion: for every doubling of model size, double the tokens. Their 70-billion-parameter Chinchilla used about four times the data of the 280-billion-parameter Gopher on the same compute and outperformed it.
Why it is asked: it separates "bigger is better" from a budgeted view. Read the full explanation: LLM inference hub.
What is reinforcement learning from human feedback, and why use it?
Reinforcement learning from human feedback (RLHF) is a training step that uses human rankings of model outputs to train a reward model, then uses reinforcement learning to tune the language model to score highly on it while staying close to the original. Ouyang et al. (2022) report that larger models do not automatically follow user intent, and that human evaluators preferred outputs from the 1.3-billion-parameter InstructGPT over the 175-billion-parameter GPT-3. They also report improved truthfulness and less toxic output, with simple mistakes remaining.
Why it is asked: it tests your grasp of alignment. Read the full explanation: direct preference optimization versus RLHF.
How does direct preference optimization differ from RLHF?
Direct preference optimization (DPO) trains the language model straight from preference pairs with a simple classification loss, without a separate reward model or reinforcement-learning loop. Rafailov et al. (2023) call standard RLHF complex and often unstable, and report that DPO matches or beats existing methods on sentiment control, summarization and single-turn dialogue. Which is better in practice is a tuning and data question.
Why it is asked: it is the common follow-up to RLHF. Read the full explanation: DPO versus RLHF.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →Adapting and prompting a model
What is LoRA, and why use it?
Low-rank adaptation (LoRA) freezes the pretrained weights and trains small low-rank matrices added to each layer. Hu et al. (2021) report that, against full fine-tuning of GPT-3 175B with Adam, it cut trainable parameters by 10,000 times and graphics processing unit (GPU) memory by 3 times, adds no inference latency, and matched or beat full fine-tuning on the models they tested. It makes per-task copies cheap to store.
Why it is asked: it tests practical adaptation. Read the full explanation: LoRA fine-tuning.
What is few-shot prompting?
Few-shot prompting puts a few worked examples of the task in the prompt so the model continues the pattern, with no weight update. Brown et al. (2020) evaluated GPT-3 this way, specifying tasks and demonstrations only through text, and note datasets where few-shot learning still struggles. Examples should match the real input format, because the model copies the pattern it sees.
Why it is asked: it is the cheapest adaptation tool. Read the full explanation: prompt engineering interview questions.
What do temperature and top-p do?
Temperature rescales the model's scores before the softmax: a low value sharpens the distribution toward the likeliest token, a high value flattens it. Top-p, or nucleus, sampling samples only from the smallest set of tokens whose probability adds up to p. Holtzman et al. (2019) report that decoding for likelihood gives "bland and strangely repetitive" text and that nucleus sampling keeps diversity without losing coherence.
Why it is asked: it checks that you know generation is sampling. Read the full explanation: beam search.
What is a hallucination, and how do you reduce it?
A hallucination is generated text that is unintended or not supported by the input or by fact. Ji et al. (2022) survey the problem across summarization, dialogue, question answering and translation, and cover measurement and mitigation. Reduce it by grounding answers in retrieved sources, asking for citations, constraining outputs, and measuring it on your own test set.
Why it is asked: it is the first reliability question. Read the full explanation: LLM observability.
What is retrieval-augmented generation?
Retrieval-augmented generation (RAG) fetches relevant passages from an index and gives them to the model along with the question. Lewis et al. (2020) combined a pretrained sequence-to-sequence model, the parametric memory, with a dense vector index of Wikipedia as non-parametric memory, and report output that is more specific, diverse and factual than a model without retrieval.
Why it is asked: it is the standard way to add knowledge. Read the full explanation: retrieval-augmented generation architecture.
Should you use prompting, RAG or fine-tuning?
Choose by what is missing. Start with a clear prompt and examples. Add retrieval when the model lacks facts that change or must be cited. Use fine-tuning when you need a consistent style, format or behaviour that prompts do not hold. They combine: a model after fine-tuning can still use retrieval. Compare cost, latency and the effort to keep each current.
Why it is asked: it tests judgment over habit. Read the full explanation: fine-tuning versus prompt engineering.
Serving and choosing a model
What is a KV cache?
A KV cache is a store of the attention keys and values of tokens already processed, so the model does not recompute them for each new token. Each generation step then runs the model on only the newest token instead of re-processing the whole prefix, but the cache uses memory that grows with context length and with the number of concurrent requests, so memory often limits how many users a GPU can serve.
Why it is asked: it links architecture to serving cost. Read the full explanation: KV cache.
What is FlashAttention?
FlashAttention is an exact attention algorithm that uses tiling to cut reads and writes between a GPU's high-bandwidth memory and its on-chip memory. Dao et al. (2022) report training that ran about 15% faster end to end on a Bidirectional Encoder Representations from Transformers (BERT) model called BERT-large at sequence length 512 than the MLPerf 1.1 training speed record, and about 3 times faster on GPT-2 at sequence length 1K. It does not approximate attention: the output is the same, only the data movement changes.
Why it is asked: it shows whether you know memory traffic is the bottleneck. Read the full explanation: FlashAttention.
What is quantization, and what does it cost?
Quantization stores a model's weights, and sometimes activations, in fewer bits to cut memory and speed up serving. The cost is lost precision, which can lower output quality, so you measure the quantized model on your own tasks before shipping it. Lower precision pays off most when memory or bandwidth limits serving.
Why it is asked: it tests serving trade-offs. Read the full explanation: LLM quantization.
Which LLM is best for interview preparation?
No single model is best; it depends on the question type. Use a model to generate practice questions, to critique your structure and to explain concepts, then check each factual claim against a primary source because models can state errors confidently. Rehearse the answer out loud, which a model cannot do for you. Compare two or three models on questions you already know the answers to.
Why it is asked: it reveals how you use AI tools in your own work. Read the full explanation: AI engineer interview questions.
Which LLM is the best for a product?
There is no best model in general: choose by the task, quality on your own evaluation set, cost per request, latency, context length and data-handling needs. Public leaderboards change quickly, so they are a shortlist, not a decision. Build a small test set from real inputs, score candidates with the same rubric, and re-run it when a model changes.
Why it is asked: it tests evaluation discipline. Read the full explanation: small language models.
Text description of the diagram
- Pre-training predicts the next token on a very large text corpus.
- Preference tuning uses RLHF or DPO.
- Serving uses retrieval and a KV cache.
For more rounds like this one, see agentic AI interview questions, and the deep learning hub.
- Define the terms precisely: token, context window, attention, pre-training, fine-tuning.
- Attention made training parallel; that is why transformers replaced recurrence and why models scaled.
- Training is stages: pre-train, tune to instructions, tune to preferences with RLHF or DPO.
- Choose between prompting, retrieval and fine-tuning by what is missing, and measure on your own data.
- Serving cost is memory: the KV cache, attention data movement and numeric precision.
Sources
- Attention Is All You Need (arXiv 1706.03762 v7) — arXiv (Vaswani et al.), 2 Aug 2023
- Language Models are Few-Shot Learners (arXiv 2005.14165 v4) — arXiv (Brown et al.), 22 Jul 2020
- Scaling Laws for Neural Language Models (arXiv 2001.08361) — arXiv (Kaplan et al.), 23 Jan 2020
- Training Compute-Optimal Large Language Models (arXiv 2203.15556) — arXiv (Hoffmann et al.), 29 Mar 2022
- Training language models to follow instructions with human feedback (arXiv 2203.02155) — arXiv (Ouyang et al.), 4 Mar 2022
- Direct Preference Optimization (arXiv 2305.18290 v3) — arXiv (Rafailov et al.), 29 Jul 2024
- LoRA: Low-Rank Adaptation of Large Language Models (arXiv 2106.09685 v2) — arXiv (Hu et al.), 16 Oct 2021
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv 2005.11401 v4) — arXiv (Lewis et al.), 12 Apr 2021
- The Curious Case of Neural Text Degeneration (arXiv 1904.09751 v2) — arXiv (Holtzman et al.), 14 Feb 2020
- Survey of Hallucination in Natural Language Generation (arXiv 2202.03629 v7) — arXiv (Ji et al.), 14 Jul 2024
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness (arXiv 2205.14135 v2) — arXiv (Dao et al.), 23 Jun 2022
