In this article
Before you start
Why interviewers ask this
A model that learns from three examples and then gives the chance of tails on the fourth flip as zero is the kind of failure this question is built to expose. Interviewers ask "derive the MLE for a biased coin", "what is the difference between MLE and MAP?" and "why is ridge regression a Bayesian method?".
All three test one idea: an estimate is a choice about how much to trust the data against what you believed before it. A weak answer recites "MAP has a prior". A strong answer writes the log-likelihood, shows that MAP adds the log of the prior, names the prior that turns it into L2 regularisation, and knows that the two agree as data grows. It is reported in interviews at large technology companies.
What each one is
- MLE
- Maximum likelihood estimation (MLE) is a method that estimates a model's parameter as the value under which the observed data would have been most probable, using only the data and no prior belief.
- MAP
- Maximum a posteriori (MAP) estimation is a method that estimates a model's parameter as the value that is most probable after seeing the data, found by maximising the likelihood multiplied by a prior distribution over the parameter.
MLE in plain words
Picture a juror who judges only on the evidence shown in court. She asks one question: which explanation makes this evidence least surprising? MLE works the same way. It looks at the data and nothing else.
The likelihood is the probability of the observed data, viewed as a function of the parameter. The maximum likelihood estimate is the peak of that function. NIST's engineering handbook (read Oct 2026) calls it "the probability of obtaining that particular set of data".
In practice we maximise the log-likelihood, which is the logarithm of the likelihood. A product of many probabilities becomes a sum, and a sum is easier to differentiate. The same logarithm underlies entropy and Kullback-Leibler (KL) divergence.
The analogy has a limit. A juror cannot write down what she believes about the world as a probability distribution. MLE has no place for such a belief at all.
MAP in plain words
Now picture a second juror. She also weighs the evidence. But she starts with a sense of what is usually true, and that sense nudges her verdict. MAP works this way. It adds your prior belief to the data.
The prior is a probability distribution over the parameter. It encodes what you believed before the data. The posterior is the updated distribution after the data. MAP returns the peak of the posterior.
The log of a product is a sum. So MAP maximises the log-likelihood plus the log of the prior. The prior is an extra term. It pulls the estimate toward values you thought likely.
A uniform prior adds a constant and changes nothing. So MLE is MAP with a flat prior.
Here the analogy also breaks. A juror's hunch is vague. A prior is a precise distribution you can write, test and change. The same idea runs through probability and statistics, and cross-entropy loss is the negative log-likelihood of a classifier.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →Worked example
A coin with unknown chance of heads p, and three flips that all land heads. Every number below is computed, not typed.1
The likelihood of h heads in n flips is p to the power h times (1 − p) to the power (n − h), up to a constant factor that does not depend on p. Its log is h log p + (n − h) log(1 − p), up to the same constant.
Set the derivative to zero and the peak is p = h/n (when h is 0 or n the derivative never reaches zero inside the interval, and the peak sits at the boundary p = 0 or p = 1). For three heads in three flips, the MLE is 1.0: the coin never lands tails, which is a poor belief to hold after three flips.1 The coin estimate h/n is unbiased at every sample size; with three flips its problem is variance, an extreme value, not bias.
For MAP, choose a Beta prior. The Beta distribution is a family of probability distributions on the interval from 0 to 1, with two shape parameters a and b. A conjugate prior is a prior whose posterior stays in the same family. The Beta is conjugate to coin flips, so the posterior is again a Beta, here with shape parameters h + a and n − h + b. NIST's beta distribution page, which writes the shapes as p and q, gives the mode as (p − 1)/(p + q − 2) when both shapes are above 1. Here the posterior shapes are h + a and n − h + b, so the MAP estimate is:
This needs both posterior shapes above 1. Under a uniform prior (a = b = 1), three heads in three flips gives a Beta(4,1) posterior, whose peak sits at the boundary p = 1, so MAP equals MLE there.
| Data | MLE | MAP, Beta(2,2) | MAP, Beta(5,5) |
|---|---|---|---|
| 3 heads in 3 flips | 1.0 | 0.8 | 0.6364 |
| 7 heads in 10 flips | 0.7 | 0.6667 | 0.6111 |
A grid search over 2,000,001 values of p reproduces each closed-form value to four decimals.1 Beta(2,2) acts like two imaginary extra flips, one head and one tail; Beta(5,5) like eight.
Text description of the diagram
- Three heads in three flips gives an MLE of 1.0.
- MAP maximises the log-likelihood plus the log-prior, which is the likelihood times the prior in log space.
The prior fades as data arrives. Keep 70% heads and grow the sample: with Beta(2,2) the MAP estimate is 0.6667 at 10 flips, 0.6961 at 100 and 0.6996 at 1,000, against an MLE of 0.7 throughout.1
Text description of the diagram
- With Beta(2,2) the MAP estimate is 0.6667 at 10 flips, 0.6961 at 100 and 0.6996 at 1,000, against an MLE of 0.7 throughout.
- A uniform prior adds a constant, so MLE is MAP with a flat prior.
Now the link interviewers love. Linear regression fits a straight-line model with one weight per input; ridge regression adds a penalty on the squared size of the weights, and a regulariser is any such penalty added to a loss to keep the model simple. Give the weights a zero-mean Gaussian prior with standard deviation tau (τ, a measure of spread), and assume the noise is Gaussian with standard deviation sigma (σ). Up to a constant, the negative log-posterior is:
Multiply through by 2σ² and it is the squared error plus lambda (λ) times the squared weight norm, where λ is σ² over τ² (the same geometry as in linear algebra for machine learning). The prior must be centred at zero: a prior centred at some value m penalises the distance from m, which is not ridge.
The scikit-learn documentation (version 1.9.1, read Oct 2026) states that ridge's L2 regularisation is "equivalent to finding a maximum a posteriori estimation under a Gaussian prior over the coefficients". We checked it on 20 simulated points with 5 weights, sigma and tau both 1, so lambda is 1: the MAP solution has a weight norm of 3.0023 against 3.396 for MLE, and the gradient of the negative log-posterior at the ridge solution is below 1e-9.1
The reading: a prior is a regulariser, and the stronger the prior the more the estimate shrinks.
Trade-offs and edge cases
MLE and MAP differ in whether they use a prior, and that choice has four consequences worth knowing.
| Property | MLE | MAP |
|---|---|---|
| Uses a prior | No | Yes |
| Equals the other when | The prior is flat | The prior is flat |
| 3 heads in 3 flips | 1.0 | 0.8 with Beta(2,2) |
| Matches ridge | No, with Gaussian noise that is ordinary least squares | Yes, with Gaussian noise and a Gaussian prior |
| Changes if you reparametrise | No | Yes |
MAP is not invariant to reparametrisation. The peak of a density moves when you change variables, which is a mathematical property of modes, so MAP for p and MAP for the log-odds of p (the logarithm of p divided by 1 − p) are different estimates, while the MLE transforms consistently. The posterior mean is not invariant either, because the mean of a transformed variable is not the transform of the mean. That is one reason some statisticians report the posterior median, which moves consistently under a monotone change of variable, or the whole posterior.
A wrong prior hurts. Beta(5,5) pulled three heads in three flips down to 0.636;1 if the coin is in fact biased, that prior needs many flips to wash out. Choose priors you can defend, and check sensitivity.
MLE can be biased with little data. An estimator is biased when its average over many samples misses the true value, and its variance is how much it moves from sample to sample. NIST notes MLE estimates become unbiased with minimum variance as the sample grows, though with very few observations they can be heavily biased. The textbook case is the Gaussian variance, whose MLE divides by n and so runs low.
Both give a single number. Neither reports uncertainty. NIST notes MLE variances can be approximated for large samples, and a full Bayesian analysis keeps the whole posterior.
Text description of the diagram
- With plenty of data and no real prior, MLE is enough and simpler.
- With scarce data, or whenever you would add L2 regularisation anyway, MAP states that choice as a prior.
The flow is our reading of the results above. With plenty of data and no real prior, MLE is enough and simpler. With scarce data, or whenever you would add L2 regularisation anyway, MAP states that choice as a prior.
How to answer it in an interview
What is the difference between MLE and MAP?
MLE picks the parameter that maximises the likelihood of the data, so it uses only the data. MAP maximises likelihood times a prior, so in log space it adds the log-prior as a penalty. With a flat prior they are identical. For linear regression with Gaussian noise, a zero-mean Gaussian prior on the weights makes MAP exactly ridge regression, with lambda equal to noise variance over prior variance. For a coin with three heads in three flips, MLE says probability one, while a mild Beta prior gives 0.8. As data grows the prior washes out and the two converge. MAP still returns a single point, not uncertainty.
The spoken structure is: data only versus data plus prior, the flat-prior case, the ridge link, then the limit.
Common mistakes
"MAP is Bayesian, MLE is not, so MAP gives the posterior." (wrong) MAP returns only the posterior's peak. (right) "MAP is a point estimate from a Bayesian model."
"MLE and MAP always differ." (wrong) With a flat prior they coincide, and with plenty of data they converge. (right) "They agree when the prior is flat or the data is large."
"Regularisation and priors are unrelated." (wrong) For linear regression with Gaussian noise, L2 is MAP under a Gaussian prior, as scikit-learn states. (right) "A penalty is a negative log-prior, up to a constant."
"The MLE of a coin after three heads is a fair estimate." (wrong) It says tails is impossible. (right) "With little data the coin MLE is extreme, which is a variance problem; a prior tempers it."
Follow-up questions
How do you derive the maximum likelihood estimate for a biased coin?
Write the log-likelihood h log p + (n − h) log(1 − p), differentiate with respect to p, and set the result to zero: h/p − (n − h)/(1 − p) = 0, so p = h/n. The second derivative is negative, so this is a maximum.
Why is MAP with a Gaussian prior the same as L2 regularisation?
With Gaussian noise the negative log-likelihood is the squared error over 2σ², and a zero-mean Gaussian prior adds, up to a constant, the squared weight norm over 2τ²; multiplying through by 2σ² gives the ridge objective with λ equal to σ² over τ². The prior must have mean zero, and other noise models give other objectives: logistic regression gives an L2-penalised log-loss, not ridge. A zero-mean Laplace prior gives an L1 penalty in the same way.
Why is the sample standard deviation of the mean sigma over root n?
For independent draws with variance σ², the variance of the sample mean is σ²/n, because variances of independent terms add and dividing by n scales the variance by 1/n². Its standard deviation is therefore σ/√n. The MLE of a Gaussian mean is the sample mean, so this is also the spread of that MLE from sample to sample.
Why does L1 regularisation create sparsity?
The L1 penalty has a corner at zero, so the best weight often sits exactly at zero instead of just near it. A zero-mean Laplace prior gives that penalty as its negative log, so MAP under a Laplace prior can set weights to exactly zero, while a Gaussian prior only shrinks them.
Why is a point estimate not the whole story?
MLE and MAP discard how sure you are. Three flips and three thousand flips can give the same estimate, but not the same confidence; the posterior distribution, or a confidence interval, carries that difference. See the Gaussian distribution page for how spread is described, and the probability and statistics hub for the wider map.
Are MLE and MAP good estimators as the sample grows?
Yes, in the sense NIST describes: as the sample grows, MLE estimates become unbiased (their average over many samples hits the true value) with minimum variance (the least movement from sample to sample). With very few observations they can be heavily biased. MAP shares the large-sample behaviour because the prior's weight shrinks as data arrives; in our coin sweep the MAP-MLE gap fell from 0.033 at 10 flips to 0.0004 at 1,000.
- MLE maximises the likelihood; MAP maximises likelihood times prior, which adds the log-prior as a penalty.
- A flat prior makes MAP equal MLE; a large sample makes them converge.
- Three heads in three flips: MLE 1.0, MAP with Beta(2,2) 0.8, with Beta(5,5) 0.636.
- With Gaussian noise, a zero-mean Gaussian prior on the weights is ridge regression, with lambda equal to sigma squared over tau squared.
- Both are point estimates; MAP changes under reparametrisation, MLE does not.
Sources
- 8.4.1.2. Maximum likelihood estimation — NIST/SEMATECH e-Handbook of Statistical Methods (undated page, read 2026-10-24), 24 Oct 2026
- 1.3.6.6.17. Beta distribution — NIST/SEMATECH e-Handbook of Statistical Methods (undated page, read 2026-10-24), 24 Oct 2026
- 1.1. Linear Models, scikit-learn 1.9.1 documentation — scikit-learn documentation (read 2026-10-24), 24 Oct 2026
