In this article
Before you start
This page builds on deep learning and a little probability.
Why interviewers ask this
A list of product names shows no mechanism, and the follow-up questions probe the mechanism. The question is short on purpose. Interviewers use it to see whether you can define generative AI in one breath, separate it from the older kind of machine learning that attaches a label to an input, and describe the mechanism in plain words.
A weak answer says generative AI is AI that makes things, and stops. A strong answer says it learns what its training examples look like and produces new ones from a prompt, and adds one limit. A Staff-level answer explains the next-token loop, the role of sampling, and when classical machine learning is the better tool. The sections below build that answer from zero.
Generative AI from first principles
- Generative AI
- Generative AI is a kind of machine-learning system that is trained on a large set of examples so that it can create new content, such as text, images or other data, in response to a prompt, rather than attach a label to an input. It learns what its training examples look like and produces new ones that resemble them.
- Predictive model
- A predictive model is a machine-learning model that is trained on many labeled examples to make a prediction about an input, such as whether an X-ray shows signs of a tumor or whether a borrower is likely to default on a loan, and that outputs a label or a number instead of new content.
In plain words
Think of a student who reads thousands of recipes and then writes a new one. The student does not copy any single recipe. The student has absorbed what recipes look like: which ingredients go together and how the steps run. The analogy breaks here. A model has no taste buds and no kitchen, so it cannot check that its recipe works.
The dictionary wording adds the trigger. Merriam-Webster describes generative AI as AI that generates new content, such as images or text, in response to a submitted prompt, by learning from a large reference database of examples. A prompt is the input, such as a question, that a user submits to ask for the content.
In one precise sentence, generative AI learns the pattern of its training examples well enough to produce new examples that resemble them, one small piece at a time.
Step 1: A generative model learns what the data looks like, not a label
A generative model learns what its training data looks like, whereas a predictive model learns to attach a label to an input. MIT News (Zewe, 2023) describes the older use of AI: models trained on millions of examples to say whether an X-ray shows a tumor or whether a borrower will default. The same article describes generative AI as a machine-learning model trained to create new data, one that learns to generate more objects that look like the data it was trained on.
The machinery overlaps. The MIT researcher quoted there says the same algorithms can often be used for both. The difference is the job.
Input: many examples. What changes: the model's internal numbers are adjusted until its output resembles those examples. Output: a model that can produce more examples like them. You now know what the model is for. The next step shows what it reads.
Step 2: Text, images and other data become tokens
Generative AI converts every input into tokens, so one method can handle text, images and other data. MIT News (Zewe, 2023) says the common thread across the main approaches is that they convert inputs into tokens, the numerical representations of chunks of data. For text, a word is the simplest example of a chunk.
Input: raw data. What changes: the data becomes a list of numbers. Output: a sequence of tokens. The model never sees letters or pixels, only these numbers, and its output is converted back at the end. Models that read text and images together, called multimodal models, rely on the same idea.
A transformer is a model architecture that relates every token to every other token through attention. Vaswani et al. (2017) proposed it as a network based solely on attention mechanisms, without recurrence or convolutions. MIT News notes that its attention map captures each token's relationships with all other tokens, which helps the model use context.
Step 3: The model gives every possible next token a probability
A generative language model scores every possible next token with a probability, and those probabilities add to one. A tiny counting model shows the idea. Take this illustrative corpus: "the cat sat on the mat . the cat ate the fish . the dog sat on the mat ." It has 20 tokens and 9 distinct ones.1
The word "the" appears six times. It is followed by "cat" twice, "mat" twice, "dog" once and "fish" once. So the probabilities of the next word are 0.3333 for "cat", 0.3333 for "mat", 0.1667 for "dog" and 0.1667 for "fish".1 After "cat", the words "ate" and "sat" each get 0.5.1
| Quantity | Value (illustrative corpus) |
|---|---|
| Tokens in the corpus | 20 |
| Distinct tokens | 9 |
| Probability of "cat" after "the" | 0.3333 |
| Probability of "dog" after "the" | 0.1667 |
| Probability of "sat" after "cat" | 0.5 |
| Probability of "cat" then "sat" after "the" | 0.1667 |
This counting model is a Markov chain, an early example of generative AI in MIT News (Zewe, 2023), introduced by Andrey Markov in 1906. It looks back only one word. The base models behind today's chatbots work in much the same way, according to the same article, but are far larger, with billions of parameters, and are trained on much of the public text on the internet.
Step 4: Sampling picks one token, and repeating builds new content
Sampling draws one token at random in proportion to its probability, appends it to the text and repeats the loop. Input: the text so far. What changes: one token is added. Output: a longer text. Because the draw is random, the same prompt can give different results.
Temperature is a setting that reshapes the next-token probabilities before the draw: below one it sharpens them, above one it flattens them. In the toy model, a temperature of 0.5 raises "cat" and "mat" to 0.4 each and cuts "dog" and "fish" to 0.1 each. A temperature of 2 moves them to 0.2929 and 0.2071.1
Run with a fixed seed at temperature 1, the toy model wrote "the cat ate the cat sat on the fish" and "the fish ." Of 200 runs, 171 differed from every training sentence, counting unfinished sequences.1 New does not mean good: the output follows learned patterns, and nothing in the loop checks that it makes sense or is true.
Step-by-step description
- The model learns what its training data looks like.
- Inputs become numbers for chunks of data.
- Every possible next token gets a probability.
- One token is drawn and appended, then the loop repeats.
- A generative model learns what its training data looks like.
- Text, images and other data are converted into tokens.
- The model gives every possible next token a probability.
- Sampling picks one token, which is appended before the loop repeats.
Text description of the diagram
- At temperature 0.5 the words cat and mat each get 0.4 after the.
- At temperature 2 the words cat and mat each get 0.2929.
What are examples of generative AI models?
Generative AI examples include next-word autocomplete, image generators, text chatbots and tools that design molecules. Named examples are the chatbot ChatGPT, which MIT News discusses, an image generator named in Merriam-Webster's example sentence, and Stable Diffusion. MIT News (Zewe, 2023) lists several model families. Markov models were an early example for next-word prediction. In 2014 researchers proposed the generative adversarial network (GAN), where a generator tries to fool a discriminator. Diffusion models followed in 2015 and refine their output step by step; one sits at the heart of Stable Diffusion. In 2017 the transformer arrived and led to the large language models behind chatbots.
The same article describes research uses beyond chat, such as synthetic images for training vision systems and new protein or crystal structures. For a closer look at one family, read how diffusion models work. Training any of these networks adjusts their numbers by backpropagation.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →The Staff-level answer
What is generative AI?
Generative AI is machine learning that creates new content, such as text or images, from a prompt, by learning what its training examples look like. In a language model, the most common kind, it learns probabilities over the next token, a numeric chunk of data: it scores the possible next tokens, samples one, appends it and repeats. Image models such as diffusion models instead refine random noise step by step. Temperature controls how adventurous the sampling is. The trade-off is that fluent output is not verified output, so I check anything that matters. For structured prediction, classical machine learning is often better. I choose by the output I need.
The spoken structure: give the definition, name the next-token loop, then state the trade-off and the decision rule.
What makes it Staff-level is that it names the mechanism and the sampling knob, admits the limit, and ends with a rule for choosing between generative and classical methods. MIT News (Zewe, 2023) describes diffusion models as models that generate new data by iteratively refining their output.
The rubric
| Dimension | Weak answer | Strong answer | Staff answer |
|---|---|---|---|
| Correctness | "AI that makes things." | Creates new content from a prompt after learning from many examples. | Adds that the model learns a distribution over its data (for text, over the next token) and samples from it. |
| Depth | No mechanism. | Names tokens and next-token scoring. | Explains the repeat loop and the role of temperature. |
| Trade-offs | None. | Says output can be wrong or biased. | Says when classical machine learning wins and how to verify output. |
| Communication | Buzzwords and product names. | One definition sentence, then one example. | Definition, mechanism, limit, decision rule, in a short answer. |
| Follow-up readiness | Stuck on "how is it different?" | Contrasts with a model that predicts a label. | Names the architecture families and when each fits. |
Three variants of this question
What is generative AI in short answer?
Generative AI is machine learning that creates new content, such as text or images, from a prompt after learning from many examples. Say that sentence first, then add one example.
What is generative AI good at compared to core ML models, and when would you use ML models instead?
Generative AI is good at producing new content and at acting as a human-friendly interface to machines; MIT News (Zewe, 2023) quotes a professor saying so. The same article quotes him saying that for predictions on structured data, such as a spreadsheet's tabular data, generative models tend to be outperformed by traditional machine-learning methods. Use a predictive model when you need a label or a number from structured data.
How can I prepare for a generative AI interview?
Practice saying the definition, the next-token loop and the trade-off out loud until each takes a few sentences. Be able to name the model families and what each is for, and work through questions like those in deep learning interview questions and answers.
Common mistakes
- "Generative AI is the same as all AI." (wrong) Before the recent boom, AI usually meant predictive models, and generative AI is a model trained to create new data (MIT News, 2023). (right) "Generative AI is one kind of machine learning, trained to create new content."
- "The model looks up a stored answer." (wrong) It builds output one token at a time from learned probabilities, which is why the toy model produced sentences that were not in its corpus. (right) "It samples new text from learned patterns."
- "Fluent output means true output." (wrong) Nothing in the sampling loop checks facts, and the MIT article notes these models can amplify false statements. (right) "I verify output where accuracy matters."
- "Generative AI is always the better, newer choice." (wrong) For tabular prediction, traditional methods tend to do better, per the same article. (right) "I pick the method by the output I need."
Follow-up questions
What are the top 3 generative AI?
Three families MIT News (Zewe, 2023) explains are generative adversarial networks, diffusion models and transformers. Each fits a different job: GANs and diffusion models are used for images, and transformers underlie large language models.
What is the transformer architecture in generative AI?
The transformer is a network built on attention instead of recurrence or convolutions, as Vaswani et al. (2017) propose. It lets each token weigh its relationship to every other token. Answer the next-level question by naming the attention map.
What is retrieval-augmented generation (RAG) in generative AI?
Retrieval-augmented generation (RAG) is a pattern where a system first retrieves relevant documents and adds them to the prompt, so the model answers with that text in front of it. Lewis et al. (2020) proposed it by combining a pretrained generator with a retrieved document memory. It is a way to supply facts the model did not learn.
Is generative AI good or bad?
It depends on the use. MIT News (Zewe, 2023) lists benefits, such as new tools for artists and scientists, and red flags, such as worker displacement, inherited bias and copyright questions. A balanced answer names one of each.
Is a large language model the same thing as generative AI?
No. A large language model (LLM) is one kind of generative AI, a transformer-based system that generates text, while generative AI also covers image and other generators.
- Generative AI creates new content from a prompt after learning what its training examples look like.
- It differs from a predictive model, which outputs a label or a number for an input.
- In a language model, inputs become tokens and the model scores every possible next token with a probability.
- Sampling picks one token, appends it and repeats; temperature sharpens or flattens the choice.
- Fluent output is not verified output, and classical methods often win on structured data.
Sources
- Generative AI (definition) — Merriam-Webster, 26 Oct 2026
- Explained: Generative AI — MIT News (Adam Zewe), 9 Nov 2023
- Attention Is All You Need (arXiv 1706.03762, v7) — arXiv (Vaswani et al.), 12 Jun 2017
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv 2005.11401) — arXiv (Lewis et al.), 22 May 2020
