What are the assumptions of linear regression?

In this article

1. Before you start
2. Why interviewers ask this
3. Linear regression assumptions from first principles
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

Why interviewers ask this

A candidate who recites "linearity, independence, homoscedasticity, normality" without saying what breaks has memorized a list, and the interviewer's next question exposes that. The assumptions come up in classical machine learning screens as "what are the assumptions", "how do you check them", "which ones matter for prediction and which for inference" and "what happens if you duplicate the data". The last of these has a crisp, computable answer, which makes it easy to ask.

A weak answer names four or five words. A strong answer ties each assumption to a consequence: which estimate becomes biased, which interval becomes wrong, and which plot shows it.

Linear regression assumptions from first principles

In plain words

Ordinary least squares

Ordinary least squares (OLS) is a method that fits a linear model by choosing the coefficients that minimize the sum of the squared residuals. The scikit-learn guide writes the problem as minimizing the squared length of Xw minus y, where X is the table of feature values, w the coefficients and y the observed targets.

Residual

A residual is the gap between an observed value of y and the value the fitted model gives for the same x. Residuals are what the fit leaves unexplained, and every assumption check looks at them.

Heteroscedasticity

Heteroscedasticity is a condition in which the spread of the errors changes with x, so the scatter fans out like a funnel. Its opposite, equal spread, is called homoscedasticity.

Multicollinearity

Multicollinearity is a condition in which two or more features carry almost the same information, so the model cannot tell their effects apart. The guide warns that the estimate then becomes highly sensitive to random errors.

Linear regression is a model that predicts y as an intercept plus a weighted sum of the features. Picture a ruler laid through a cloud of dots. The ruler is your line, and each dot's distance from it is a residual. The assumptions say what the cloud must look like for the ruler's position, and your confidence in it, to mean something. If the dots bend away from the ruler, the position itself is off. If they bunch up in time or fan out, the position may still look fine while the confidence is not.

The analogy breaks in one place. A ruler is just placed; a regression also reports how uncertain the placement is, and that uncertainty is the part the assumptions protect.

In one precise sentence, the assumptions describe the errors around a mean that is linear in the coefficients, and most of them guard the standard errors rather than the coefficients.

Step 1: The average of y is linear in the coefficients

Linearity means the expected value of y given x is linear in the coefficients. "Linear" refers to the coefficients, not to a straight line in x, so x squared still counts and a linear regression can fit a curve. Break it and the coefficients are biased, because the assumed shape does not match the data. The errors must also average zero at every x (exogeneity): leave out a variable that moves with x and the slope absorbs its effect.

Anscombe's 1973 paper Graphs in Statistical Analysis uses four small data sets. All four give the same fitted line, an intercept of 3.00 and a slope of 0.500, and a correlation of 0.816 or 0.817, yet one is a clean cloud, one a curve, one has an outlier and one hangs on a single point¹. The numbers cannot tell them apart; a residual plot can. A curved band of residuals means the relationship is not linear.

Step 2: Errors are independent of each other

Independence means one observation's error tells you nothing about the next one's; it often fails for time-ordered or grouped data. When it fails, the coefficients stay unbiased, but the standard errors can be far too small, typically when x also trends with the order (time series) or is shared within groups.

In the simulation, 4,000 data sets of 60 points were drawn with a true slope of 3, and a nominal 95 percent interval was built for each. With independent errors it caught the truth 0.952 of the time. With errors that carried over from one point to the next at a rate of 0.8, and x rising with the order of the data, it caught it only 0.489 of the time¹. With x in random order, coverage was 0.951¹. The Durbin-Watson statistic, near 2 when neighboring residuals are unrelated, was 2.07 for an independent sample and 0.37 for a correlated one, both from a separate 200-point sample¹.

Step 3: The spread of the errors is equal everywhere

Equal variance means every error is drawn with the same spread, whatever the x value or the fitted value. Break it and the coefficients stay unbiased, but the usual standard errors are wrong, and the plot of residuals against fitted values fans out like a funnel.

Even strong unequal spread did less damage here. When the error spread grew with x in the same simulation, from a standard deviation of 0.4 to 12.6, the nominal 95 percent interval still covered the true slope 0.911 of the time¹. The usual remedies are to model the logarithm of y when errors scale with its size, or to use heteroscedasticity-consistent standard errors, which statsmodels offers as a family of covariance options named in its documentation. On a synthetic multiplicative relationship, the residual spread in the upper half of the fitted values was 1.96 times the lower half; after taking the log of y the ratio was 0.95¹.

Step 4: No feature is an exact copy of the others

The formal assumption is no perfect collinearity: no feature is an exact linear combination of the others, so X-transpose-X (the feature table multiplied by its own flipped copy, a square table of how the features overlap) can be inverted. Near-copies break no assumption and leave OLS unbiased and best linear unbiased, but they inflate the variance, because OLS cannot separate their effects. Predictions can stay fine, but individual coefficients swing wildly from sample to sample and may flip sign.

The variance inflation factor (VIF) measures it: for one feature it equals one divided by one minus the squared correlation with the others. Two features with a correlation of 0.6994 give a VIF of 2.0; at 0.9586 it is 12.3; at 0.9952 it is 104.4¹. Fixes are to drop or merge the duplicates, to use ridge regression, which adds a penalty on the size of the coefficients, or to collect data that varies the features independently.

Step 5: Normal errors matter mostly for small-sample tests

Normality of errors means the errors follow a normal distribution. For coefficient tests and intervals it is needed only for exact small-sample results; a prediction interval for a single new point leans on it at any size, because the central limit theorem averages the coefficients, not one new error. The Gauss-Markov theorem, summarized on Wikipedia, says OLS is the best linear unbiased estimator when errors have mean zero, equal variance and no correlation; it does not require normality.

With a large sample, the central limit theorem, which says averages of many independent values look normal, makes coefficient intervals approximately right even for skewed errors. A quantile-quantile (Q-Q) plot, which compares residual quantiles with normal quantiles, shows departures; heavy tails matter more than mild skew. If the plot bends only in the far tails and the sample has hundreds of rows, the intervals are usually still close to right; if the sample is small and the tails are heavy, a bootstrap, which re-fits the model on many resampled copies of the data, gives intervals that do not lean on normality.

1 · Linearity2 · Independence3 · Equal variance4 · No exact copies5 · Normal errorsReplay

Step-by-step description

1. The average of y is linear in the coefficients.
2. One error tells you nothing about the next.
3. The scatter does not fan out.
4. No feature is an exact combination of others; near-copies inflate variance.
5. Needed for small-sample tests, and for a single-point prediction interval at any size.

• A residual plot shows whether the assumed shape was right.
• The Durbin-Watson statistic was 2.07 for an independent sample and 0.37 for the correlated one.
• On a synthetic multiplicative example, the residual spread in the upper half of the fitted values was 1.96 times the lower half before taking the log.

Worked example

The values come from fitting a line to each of the 11-point sets of Anscombe's paper¹. R-squared is the share of the variation in y that the line explains. Any of the sets could pass a check of R-squared alone, which is 0.667 for the first and 0.666 for the next two¹.

Text description of the diagram

• With independent errors the interval caught the truth 0.952 of the time.
• With correlated errors and x rising with the data order the interval caught the truth only 0.489 of the time; with x shuffled, 0.951.

Each row is 4,000 simulated data sets of 60 points with a true slope of 3, and the last two rows differ only in whether x is sorted along the order of the data¹. These are illustrative inputs, not measurements of real data.

The standard error ratio is 0.698, which equals the square root of 38 divided by 78, that is, the square root of the residual degrees of freedom before over the degrees of freedom after¹. The t-statistic is the coefficient divided by its standard error. It therefore grows by a factor of 1.433, although the duplicated data holds no new information. That is the independence assumption broken on purpose.

Text description of the diagram

• A curved band of residuals means the relationship is not linear.
• The plot of residuals against fitted values fans out like a funnel when the variance is not equal.

Trade-offs and edge cases

The assumptions matter differently for prediction and for inference. If you only need predictions on data like your training data, a violated assumption mostly costs you a poor fit. If you need to say a coefficient is significant or give an interval, the assumptions about the errors carry the weight.

• Gauss-Markov versus normality. Unbiasedness needs only zero-mean errors; the "best linear" property adds equal variance and uncorrelated errors; normality adds exact small-sample tests.
• Heteroscedasticity-consistent standard errors. Where only the variance assumption fails, keeping the coefficients and fixing the standard errors is often enough. The statsmodels documentation notes that the printed standard errors assume the error covariance is correctly specified.
• Exogeneity. Omitted variables, measurement error in x and reverse causation bias coefficients however many rows you add, and no residual plot shows it.
• Outliers. One extreme point can move a least-squares line far, as Anscombe's third set shows, because squared residuals reward chasing it.
• Nonlinear models. When the relationship is curved, adding terms or moving to a different model family, such as logistic regression for a yes/no target, changes the question; the assumptions then apply to a different mean function.
• Evidence in production. Residual checks belong in monitoring, because a model trained on one period can face distribution shift later.

How to answer it in an interview

The spoken structure is: list the five, say what each failure breaks, name the check, then name the fix.

Common mistakes

• "Linear regression needs the data to be normal" (wrong). The errors, not the features or the target, are the thing that is assumed normal, and mainly for small-sample tests (right: say normal errors, and say when it matters; a prediction interval for one new point leans on it at any size).
• "A high R-squared means the assumptions hold" (wrong). Anscombe's four sets share R-squared and a fitted line (right: plot the residuals).
• "Collinearity ruins the predictions" (wrong). It inflates the variance of individual coefficients, while predictions inside the data's range can stay fine (right: say it hurts interpretation first).
• "A narrower interval from more rows is always more trustworthy" (wrong). Duplicated rows narrow the interval without adding information (right: rows only count if they are independent).
• "Heteroscedasticity biases the slope" (wrong). It leaves the slope unbiased and misstates its uncertainty (right: say which estimate is affected).

Follow-up questions

What is the normal equation for ordinary least squares?

The normal equation sets the gradient of the squared error to zero and gives the coefficients as the inverse of X-transpose-X times X-transpose-y. It is the closed-form answer to the minimization the scikit-learn guide states, and it fails when X-transpose-X is singular, which is exactly the perfect-collinearity case the assumption rules out. Gradient descent reaches the same answer iteratively.

What is the difference between ordinary least squares and ridge regression?

OLS minimizes the squared residuals alone; ridge adds a penalty proportional to the squared size of the coefficients, the L2 penalty compared with the L1 one in L1 vs L2 regularization. The penalty shrinks coefficients toward zero, which stabilizes them under multicollinearity at the cost of a small bias, the trade described in bias vs variance.

What happens to the coefficients and t-statistics if every sample is duplicated?

The coefficients are unchanged, because the fitted line still minimizes the same average squared error. The standard errors shrink to about 0.698 times their size in the 40-row example, so t-statistics grow by about 1.43 and results look more significant than the data justify.

What happens to R-squared if you double the data?

R-squared is unchanged when you stack an exact copy of the data, because both the unexplained and the total variation double and their ratio stays the same. It was 0.9337 before and after in the example.

How do you handle features that are highly correlated with each other?

Measure the variance inflation factor, then drop or merge the near-duplicates (see feature engineering techniques), use ridge regression, or gather data that varies the features independently. If you only need predictions, the correlation may not need fixing at all.

Sources

1. 1.1.1. Ordinary Least Squares, scikit-learn 1.9.1 user guide — scikit-learn (page shows version only; read 2026-10-24), 24 Oct 2026
2. Graphs in Statistical Analysis (Anscombe), The American Statistician 27(1), 17-21 — Taylor & Francis for the American Statistical Association (record checked in Crossref), 1 Feb 1973
3. Linear regression with statsmodels 0.15.0: ordinary least squares — statsmodels (page shows version only; read 2026-10-24), 24 Oct 2026
4. statsmodels OLSResults.get_robustcov_results (heteroscedasticity-robust covariance HC0 to HC3) — statsmodels (page shows version only; read 2026-10-26), 26 Oct 2026
5. Gauss-Markov theorem — Wikipedia (pointer for the theorem's conditions; read 2026-10-24), 24 Oct 2026