The 45-minute notebook round: what to do, minute by minute

In this article

1. Before you start
2. Why interviewers ask this
3. The notebook round from first principles
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This page belongs to the PyTorch interview questions and ML coding round guide. The code is plain Python with NumPy and scikit-learn style calls; no deep learning is needed.

Why interviewers ask this

A notebook round shows what a take-home or a design discussion hides: how you work when the data is small, messy and the clock is running. Candidates report prompts such as detecting fraudulent posts, spotting anomalous driver behavior from trip data given as a dict, computing features and writing an evaluation loop, and sketching a feature pipeline (reported by candidates at companies including Reddit, Lyft, Brex and Coinbase; the format and length vary by company).

A weak answer jumps to a complex model and reports one accuracy figure. A strong answer asks what a mistake costs, splits first, builds a baseline to beat, and evaluates with a metric that fits rare positives. A Staff-level answer also says what it would do with another hour: better features, a time-based split, a review of the top-scored rows. The signal is judgment under time pressure, not model size.

The notebook round from first principles

Baseline

A baseline is the simplest model or rule that you can build in minutes, such as predicting the majority class or a plain logistic regression. Every later change is judged against it, so a baseline tells you whether extra work paid off.

Average precision

Average precision is a single number that summarizes the precision-recall curve of a classifier by averaging the precision reached at each true positive in the ranked scores. It rewards a model that ranks real positives near the top, and a model with no skill scores close to the share of positives.

In plain words

Picture a practice exam. If you see the answer key while studying, a high mark tells nobody anything. A model works the same way. The test rows are the exam, and they must stay unseen until the end.

The analogy breaks in one place. A person remembers peeking. A model does not, so you must keep the discipline in code.

In one precise sentence: a notebook round asks you to turn a vague task into a split, a baseline, features and a metric, in that order, and to say what each choice costs.

Step 1: Read the task and split before anything else

Spend the first part of the budget, about 5 minutes of the 45,¹ reading the task and asking questions: what is a positive, what does a missed positive cost against a false alarm, is there a time order, how rare are positives? Then split the data before any other line touches it.

The scikit-learn guide (read 26 October 2026) defines leakage as using information at build time that would not be available at prediction time, and says to split into train and test subsets first, "particularly before any preprocessing steps." Its rule for the fit methods is to never include test data. In practice, fit scalers, imputers and encoders on the training rows only, then call transform on both parts. For rare positives, stratify the split so each part keeps the same share of positives. If rows have timestamps, split by time instead. You now have a safe frame for the next steps.

Step 2: Build a baseline you can beat

Give the baseline about 10 minutes.¹ State the dumbest useful model first: always predict the majority class. Then build a simple, honest one, such as logistic regression inside a Pipeline, which keeps each fit inside the training data, a design the scikit-learn page recommends for avoiding leakage.

from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

# split first, keeping the share of positives equal in both parts
X_tr, X_te, y_tr, y_te = train_test_split(
    X, y, test_size=0.25, stratify=y, random_state=0)
# carve a validation part out of the training rows (for the threshold, Step 4)
X_tr, X_val, y_tr, y_val = train_test_split(
    X_tr, y_tr, test_size=0.2, stratify=y_tr, random_state=0)

# the scaler is fitted on training rows only, inside the pipeline
model = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000))
model.fit(X_tr, y_tr)
scores = model.predict_proba(X_te)[:, 1]

Input: the training rows. Output: a score per test row. Say the baseline's number aloud; it is the bar every later idea must clear. For rare positives, a one-line change is LogisticRegression(class_weight={1: 20}) or class_weight="balanced", which counts each positive row more heavily in the loss.

Step 3: Add features that fit the story

Features take the largest share of the budget, about 15 minutes.¹ Start from the task's story, not from the library. For fraudulent posts, ask what a fraudster does differently: posts more often, uses a young account, pastes more links. Each idea becomes a column: posts in the last hour, the log of account age in days, links per post. Take the log of a long-tailed column so a few huge values do not dominate.

When the data arrives as a dict of records, loop over it once, group by entity (a driver, an account), and compute aggregates such as counts per hour, share of canceled trips or average speed. Write each feature as a small function so you can explain it. Add features one at a time and re-check the baseline score. If a new feature helps by a surprising amount, suspect leakage first, such as a column recorded after the event.

Step 4: Evaluate with precision and recall, not accuracy

Evaluation gets about 10 minutes.¹ With rare positives, report precision and recall at a stated threshold plus a ranking metric such as average precision. The Saito and Rehmsmeier paper (a 2015 paper in a Public Library of Science journal) shows that the visual interpretation of receiver operating characteristic (ROC) plots can be deceptive on imbalanced datasets, and recommends precision-recall plots there.

Choose the decision threshold on a validation part cut from the training rows, never on the test rows, because the test set must not be used to make choices about the model. Use the test rows once, at the end, so the reported numbers are an honest estimate for new data. The worked example below shows the numbers.

Step 5: Explain the result and the next hour

Keep the last 5 minutes for talking.¹ Say what the model does, the number it reached against the baseline, and what a false alarm and a miss cost. Then say what you would do next: more features, a time-based split, a look at the highest-scored rows by hand. An interviewer often forgives unfinished code but not a score that nobody can explain.

1 · Read and split2 · Baseline3 · Features4 · Evaluate5 · ExplainReplay

Step-by-step description

1. Ask about cost, then split before any preprocessing.
2. Build a model that every idea must beat.
3. Turn the task's story into columns.
4. Report precision, recall and average precision.
5. Say the result, the cost trade-off and the next step.

• Split the data before any preprocessing.
• Build a baseline first, then add features.
• Evaluate with precision and recall, then explain the result.

Worked example

A synthetic fraud dataset shows why Step 4 matters. It has 20,000 posts, of which 200 are fraudulent, so the prevalence, the share of positives, is 0.01.¹ The data are illustrative, not real posts. A stratified split gives 12,000 training rows, 3,000 validation rows and 5,000 test rows, of which 50 are positive.¹

A model that flags nothing scores an accuracy of 0.99 on the test rows and catches none of the 50 frauds, a recall of zero.¹ A plain, unregularized logistic regression trained by gradient descent in the script on the 12,000 training rows reaches a precision of 0.9375 and a recall of 0.6 at a threshold of 0.5; the Step 2 pipeline gives similar but not identical numbers.¹ The threshold with the best F1 (the balance of precision and recall) on the validation rows is 0.4238, and there the model catches 32 of 50 frauds with 6 false alarms (precision 0.8421, recall 0.64).¹

The ranking numbers tell the second story. On the same test scores, the area under the receiver operating characteristic curve (ROC-area under the curve (AUC), a kind of area under the curve) is 0.9851, while average precision is 0.7685 against a no-skill baseline of 0.01.¹ ROC-AUC looks near perfect; average precision shows how much work remains.

Text description of the diagram

• ROC-AUC is 0.9851 on the test scores.
• Average precision is 0.7685 against a no-skill baseline of 0.01.

A last check on leakage: the mean of the posts-per-hour column differs by only 0.003 between the training rows and all rows.¹ On this data a scaler fitted on everything would barely change the score. The rule still matters, because steps that use the label, such as target encoding or choosing features by their link to the label, can inflate a score a lot.

Trade-offs and edge cases

Class weights and resampling both aim at rare positives, but in the example the weighted model has the same ranking quality as the plain one. Weighting mainly moves the operating point: recall rises to 0.86 at a threshold of 0.5 and precision falls to 0.3525.¹ Pick by the cost of a miss against a false alarm, and tune the threshold on validation rows either way.

Accuracy, ROC-AUC and average precision answer different questions. A majority-class rule scores an accuracy equal to the share of the majority class, so rare positives make accuracy uninformative. ROC-AUC does not depend on prevalence, which is why it can look excellent when positives are rare. The no-skill value of average precision equals the share of positives, so it moves with the problem.

Text description of the diagram

• A majority-class rule scores an accuracy equal to the share of the majority class.
• The no-skill value of average precision equals the share of positives.

Other edge cases: if rows are ordered in time, a random split lets the model see the future, so split by time. If labels are scarce or missing, as in driver behavior data, an unsupervised detector such as IsolationForest gives a score, but you still need a few known cases or a hand review of the top-scored rows to evaluate it.

If the round asks for a classifier written from scratch instead, decision trees and naive Bayes from scratch is one possible variant, and domain-specific implementation tasks covers prompts set in a business setting. If the target is a number, such as delivery time, the plan is the same: swap to mean absolute error (MAE) or root mean squared error (RMSE), and use a mean-prediction baseline.

How to answer it in an interview

The spoken structure: clarify the cost, split, baseline, features, honest metric, then the next step.

Common mistakes

• "I standardized the whole dataset, then split." (wrong) The scaler saw test statistics. (right) "I split first and fitted the scaler on the training rows only, inside a pipeline."
• "The model reaches 99% accuracy." (wrong) With 1% positives a model that flags nothing does the same. (right) "Accuracy is uninformative here, so I report precision, recall and average precision against the baseline."
• "I picked the threshold that scored best on the test set." (wrong) That uses test data to make a model choice. (right) "I chose the threshold on a validation split and used the test set once."
• "I added a feature and the score jumped, great." (wrong) A big jump often means leakage. (right) "A large jump makes me check whether the column exists at prediction time."
• "I spent the whole round tuning." (wrong) You never explained the result. (right) "I reserved the final minutes to explain the result, the trade-off and the next step."

Follow-up questions

How do you write a binary classifier to detect fraudulent posts?

Split first with stratification, build a majority-class baseline and a logistic regression in a pipeline, add behavior features such as posting rate, account age and links per post, and evaluate on held-out rows with precision, recall and average precision. The worked example above runs exactly this plan.

How do you compute features, find anomalies and write a basic evaluation loop?

Compute per-entity aggregates, score each entity with a simple rule such as distance from the median in median-absolute-deviation units, then loop over thresholds on a validation set and record precision and recall at each. Pick the threshold that matches the stated cost of a miss.

How do you detect anomalous driver behavior from a dataset provided as a dict?

Loop over the dict once, group trips by driver, and compute features such as trips per hour, cancellation share and average speed. Score each driver against the typical value, then review the highest-scored drivers by hand, since anomalies rarely come with labels. Say which behaviors you would call normal before scoring.

How do you write pseudocode for a feature engineering pipeline?

Write the stages in order: split, fit each transform on training rows, apply it to both parts, join the features, train, evaluate. Name what each stage learns, because the learned values are exactly what must not come from test rows. A code review habit helps here; see code review interview questions.

What code do you write for ML transformations?

Write small, named transform functions (log of a long-tailed column, group aggregates, encoding of categories) that expose a fit step and a transform step. Fit on training rows only, then apply to both parts, so the pipeline behaves the same at prediction time.

How do you improve a classifier on an imbalanced dataset?

Check the metric first, then the threshold, then the features. Class weights or resampling mostly shift the operating point, as the example shows, so better features and cleaner labels are where ranking quality improves. In code, pass class_weight={1: 20} or class_weight="balanced" to LogisticRegression. The wider ML coding round guide covers the neighboring prompts.

Sources

1. Common pitfalls and recommended practices, section on data leakage (scikit-learn documentation) — scikit-learn, 26 Oct 2026
2. The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets — Public Library of Science (PLOS) ONE (Saito and Rehmsmeier), 4 Mar 2015
3. average_precision_score (scikit-learn documentation) — scikit-learn, 26 Oct 2026
4. train_test_split (scikit-learn documentation) — scikit-learn, 26 Oct 2026