In this article
Statistics and experiments
What is a p-value, and what does it not tell you?
A p-value is the probability, computed assuming a specified statistical model (usually the null hypothesis) is true, of a result at least as extreme as the one observed (Wasserstein and Lazar, 2016). The American Statistical Association (ASA) adds that it does not measure the probability that the hypothesis is true, nor an effect's size or importance, and that decisions should not rest only on a threshold. Report the effect size and an interval beside it.
Why it is asked: confusing it with the probability the hypothesis is true is a frequent error, and the ASA statement was written to address misuse. Start with the product and analytics cases hub, then read the full explanation: A/B testing interview questions.
How do you run and analyse an A/B test?
An A/B test is a controlled experiment that randomly assigns users to a control, usually the existing version, or a treatment, the new version, collects the metrics of interest and runs a statistical test of whether they differ (Kohavi, Henne and Sommerfield, 2007). Decide the metric and sample size before launch, and do not stop when the result first looks good.
Text description of the diagram
- Users are randomly assigned to a control or a treatment.
- A statistical test checks whether the metric differs between them.
Why it is asked: it is the core product-analytics skill. Read the full explanation: A/B testing interview questions.
What is a t-test, and when do you use it?
A two-sample t-test tests whether two population means are equal. The null hypothesis is that they are equal; the test statistic is the difference in sample means divided by its standard error, and you reject the null if its absolute value exceeds the critical value at your significance level, often 0.05 (NIST t-test page, read Oct 2026). Check that the samples are independent and roughly normal.
Why it is asked: it tests whether you know what the test assumes. Read the full explanation: t-test.
What is an outlier, and what do you do about it?
An outlier is an observation that lies an abnormal distance from other values in a random sample from a population (NIST outliers page, read Oct 2026). First find out why it exists: a data error, a different population or a rare real event. Remove it only for a documented reason, and report results with and without it when it changes the conclusion.
Why it is asked: it tests judgment about messy data. Read more: probability and statistics hub.
What is the difference between correlation and causation?
Correlation means two quantities move together; causation means a change in one produces a change in the other. A hidden third variable can create a correlation with no causal link. Kohavi et al. say controlled experiments are designed to test for causal relationships, unlike post-hoc analysis of data that was already collected (Kohavi et al., 2007).
Why it is asked: it separates analysis from storytelling. Read the full explanation: correlation versus causation.
How would you explain Bayes' theorem?
Bayes' theorem updates the probability of a hypothesis after seeing evidence: the posterior is proportional to the prior times the likelihood of the evidence under that hypothesis. A rare condition with a good test still gives many false alarms, because the prior is small. Work one numeric example aloud rather than reciting the formula.
Why it is asked: it tests intuition about base rates. Read the full explanation: Bayes' theorem interview questions.
Machine learning and evaluation
What is overfitting, and how do you detect it?
Overfitting is a failure in which a model fits its training data, even noise, so well that it predicts poorly on unseen data. scikit-learn warns that testing on the training data is a methodological mistake, because a model that repeats the labels it has seen would score perfectly yet fail on new data (scikit-learn cross-validation guide, 1.9.1). Compare training and held-out error.
Why it is asked: it opens most model-evaluation discussions. Read the full explanation: bias and variance.
What is cross-validation?
Cross-validation splits the training data into k folds, trains on k-1 of them, validates on the remaining one, repeats so each fold is held out once, and reports the average score (scikit-learn cross-validation guide). It uses data more efficiently than one split, whose result can depend on a particular random choice. Use stratified folds for imbalanced classes.
Why it is asked: it tests whether you validate honestly. Read the full explanation: feature engineering techniques.
What is data leakage?
Data leakage is information from outside the training set, such as the test set or the future, entering model training so evaluation scores look better than real performance. scikit-learn notes that preprocessing such as standardization or feature selection must be learnt from the training set and applied to held-out data, otherwise knowledge about the test set can leak into the model (scikit-learn cross-validation guide). Wrap preprocessing in a pipeline.
Why it is asked: it is a frequent reason evaluation scores do not hold up after launch. Read the full explanation: feature engineering techniques.
What are precision and recall?
Precision is the share of predicted positives that are truly positive, tp over tp plus fp; recall is the share of true positives the model finds, tp over tp plus fn. scikit-learn describes precision as the ability not to label a negative sample as positive and recall as the ability to find all the positive samples (scikit-learn metrics guide, 1.9.1). Choose the balance by the cost of each error type.
Why it is asked: it tests metric choice. Read the full explanation: confusion matrix.
Why is accuracy misleading on imbalanced data?
When one class is rare, a model that always predicts the common class scores high accuracy while finding none of the rare cases. scikit-learn offers balanced accuracy, the macro-average of recall per class, which drops to one over the number of classes when a classifier looks good only because of the imbalance (scikit-learn metrics guide). Report precision, recall or average precision as well.
Why it is asked: it checks that you look past one number. Read the full explanation: confusion matrix.
What does ROC AUC measure?
The receiver operating characteristic (ROC) curve plots the true positive rate against the false positive rate at various thresholds, and the area under it (AUC) summarises the curve in one number. AUC equals the probability that the model scores a randomly chosen positive above a randomly chosen negative; 0.5 is random ranking and 1.0 is perfect (Fawcett, 2006, section 7). scikit-learn notes that, unlike the F1 score, ROC does not require optimising a threshold (scikit-learn metrics guide). On very imbalanced data, also look at precision-recall.
Why it is asked: it is a widely used ranking metric. Read the full explanation: ROC curve in scikit-learn.
What is the difference between L1 and L2 regularization?
Both add a penalty on the weights to reduce overfitting. L1 adds the sum of absolute weights and can set some weights to exactly zero, so it selects features; L2 adds the sum of squared weights and shrinks all of them without zeroing any. Use L1 for sparse models and L2 when features are correlated.
Why it is asked: it tests the reason behind a standard technique. Read the full explanation: L1 versus L2 regularization.
What are bias and variance?
Bias is error from a model too simple to capture the pattern; variance is error from a model so flexible that it changes a lot with the training sample. Raising flexibility lowers bias and raises variance, so held-out error is usually lowest between the extremes. Regularization, more data and simpler models reduce variance.
Why it is asked: it frames every model-choice question. Read the full explanation: bias and variance.
How do you handle missing data?
First ask why it is missing, because dropping rows can bias the sample when missingness depends on the outcome. Options are dropping rows or columns, imputing with a statistic or a model, and adding an indicator that marks missing values. Learn imputation values from the training set only, which avoids the leakage described above.
Why it is asked: it tests judgment about messy data. Read the full explanation: feature engineering techniques.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →SQL and data wrangling
What is a window function, and how does it differ from GROUP BY?
A window function is a calculation across a set of table rows related to the current row that, unlike an ordinary aggregate with a grouping clause, does not collapse rows into one output row: the rows keep their separate identities. The OVER clause defines the window, PARTITION BY splits rows into groups and ORDER BY sets their order (PostgreSQL 18 documentation, read Oct 2026).
Why it is asked: it separates candidates who only know a grouping clause from those who can rank and compare rows. Read the full explanation: SQL hub.
What is the difference between an inner join and a left join?
An inner join keeps only the rows whose keys appear in both tables; a left join keeps every row of the left table and fills the right table's columns with missing values where there is no match. pandas documents the same types in merge() as inner, left, right, outer and cross (pandas 3.1.0, read Oct 2026).
Why it is asked: it checks that you can predict a row count. Read the full explanation: SQL join diagram.
Why did my row count grow after a join?
Duplicate keys. In a many-to-many join, if a key combination appears more than once in both tables the result holds the Cartesian product of the matching rows, and pandas warns that merging on duplicate keys can multiply the result's size sharply (pandas 3.1.0). Check key uniqueness before joining, and in pandas use the validate argument.
Why it is asked: silent fan-out is a real analysis bug. Read the full explanation: SQL join diagram.
How do you scale and encode features before modelling?
Numeric features on different scales are usually standardized so that distance-based and penalised models treat them evenly; categorical features are encoded, for example one column per category. Fit the scaler and encoder on the training data only, then apply them to held-out data (scikit-learn cross-validation guide).
Why it is asked: it tests pipeline discipline. Read the full explanation: feature engineering techniques.
Product and communication
How would you measure whether a new feature worked?
Define the goal metric before the launch, add guardrail metrics that must not get worse, and run a randomised experiment where you can: randomisation lets a metric difference be read as the effect of the change (Kohavi et al., 2007). When you cannot randomise, say what your estimate assumes.
Why it is asked: it is a common product-sense case. Read the full explanation: product and analytics cases.
What are the 5 P's of data science?
No single standard exists, but the list most often repeated is purpose, people, processes, platforms and programmability (Towards Data Science, "The 5 P's of Data Science Projects"). A variant swaps the last two for platform and performance: purpose, people, process, platform, performance (Trust Insights, 9 February 2023). In an answer, name the list you use, give one sentence per P tied to a project you ran, and ask the interviewer if they have a different list in mind.
Why it is asked: it tests whether you can structure a project from goal to delivery. Read more: interview loops.
How does a staff data scientist differ from a data scientist?
Levels differ by company and no standard ladder is published in the sources used here, so treat any answer as a pattern. A common pattern is scope: a data scientist answers a question well, while a staff data scientist chooses which questions matter, sets standards across teams and is trusted on ambiguous, high-stakes decisions.
Why it is asked: it probes how you describe your own growth. Read more: behavioral interview hub.
How do you answer data science interview questions well?
State the answer first, then the reason, then the trade-off or assumption. Name what you would check before trusting a number, such as sample size, leakage or duplicates. Practise one example from your own work for each topic above, and say your answer aloud before a round, because explaining clearly is part of what is scored.
Why it is asked: communication is judged beside correctness. Read more: interview loops.
- Statistics: a p-value is not the probability the hypothesis is true; report effect sizes.
- Experiments: randomise, fix the metric and sample size first, and know what the test assumes.
- Evaluation: validate honestly, prevent leakage, and choose metrics for the cost of each error.
- SQL: windows keep rows; joins on duplicate keys multiply rows.
- Say what a number assumes before you rely on it.
Sources
- Practical Guide to Controlled Experiments on the Web: Listen to Your Customers not to the HiPPO — KDD 2007 (Kohavi, Henne, Sommerfield), 12 Aug 2007
- American Statistical Association releases statement on statistical significance and p-values — American Statistical Association, 7 Mar 2016
- The ASA's Statement on p-Values: Context, Process, and Purpose — The American Statistician 70(2), Wasserstein and Lazar, 9 Jun 2016
- An introduction to ROC analysis — Pattern Recognition Letters 27(8), Fawcett, 1 Jun 2006
- The 5 P's of Data Science Projects — Towards Data Science (date = read date), 24 Oct 2026
- People are the most important — Trust Insights, 9 Feb 2023
- 1.3.5.3. Two-Sample t-Test for Equal Means — NIST/SEMATECH e-Handbook of Statistical Methods (date = read date), 24 Oct 2026
- 7.1.6. What are outliers in the data? — NIST/SEMATECH e-Handbook of Statistical Methods (date = read date), 24 Oct 2026
- 3.1. Cross-validation: evaluating estimator performance (scikit-learn 1.9.1) — scikit-learn developers (date = read date), 24 Oct 2026
- 3.4. Metrics and scoring (scikit-learn 1.9.1) — scikit-learn developers (date = read date), 24 Oct 2026
- 3.5. Window Functions (PostgreSQL 18) — PostgreSQL Global Development Group (date = read date), 24 Oct 2026
- Merge, join, concatenate and compare (pandas 3.1.0) — pandas developers (date = read date), 24 Oct 2026
