In this article
How to read these answers
These prompts usually sit in the product and domain rounds of a data science loop; the product and analytics case guide explains the round itself, and data science interview questions covers the wider list. Every answer below follows one pattern: name the harm, choose a quantity you can measure, apply a fix, and state what the fix costs. Behavioral prompts get a structure to fill with your own real story, never a script to recite.
Fairness and bias
How would you quantify and measure algorithmic bias in content recommendations?
Name the harm first, such as one group receiving less useful or less diverse content. Then choose a quantity you can compute per group, such as exposure share (the fraction of all impressions that a group's content or users receive) or the error rate of a classifier inside the feed, and report the gaps with uncertainty. Hardt, Price and Srebro warn that satisfying a fairness measure "should not be considered a conclusive proof of fairness" (Hardt et al., October 2016).
Why it is asked: it checks whether you turn a vague worry into a per-group number. Read the full explanation: measuring AI products.
If you remove the sensitive attribute, is the model fair?
No. Other features can encode the attribute, so deleting it leaves the bias in place. Hardt, Price and Srebro call the idea fairness through unawareness and say it is ineffective because of redundant encodings (Hardt et al., October 2016). Keep the attribute available for evaluation, compare outcomes by group, and check what the law allows you to use in the decision itself.
Why it is asked: it separates a slogan from an understanding of proxies. Read the full explanation: Hardt, Price and Srebro, Equality of Opportunity in Supervised Learning.
What is the difference between demographic parity, equal opportunity and equalized odds?
Demographic parity is the requirement that a decision, such as approving a loan, be independent of the protected attribute, so selection rates match. Equal opportunity is a relaxation of equalized odds: only true positive rates must match among people who should get the positive outcome. Equalized odds is the requirement that both true positive and false positive rates match across groups (Hardt et al., October 2016). Choose by which error hurts people.
Why it is asked: it tests whether you can name a definition and its trade-off. Read the full explanation: Hardt, Price and Srebro, Equality of Opportunity in Supervised Learning.
Does a calibrated score mean a model is not discriminatory?
No. A score can be calibrated within each group and still fail equalized odds; Hardt, Price and Srebro note that calibration within groups does not in general satisfy equalized odds, citing Kleinberg et al. (Hardt et al., October 2016). Fairness definitions can conflict, so say which one fits the harm and what you give up by choosing it.
Why it is asked: it probes whether you know that fairness metrics compete. Read the full explanation: Hardt, Price and Srebro, Equality of Opportunity in Supervised Learning.
How would you lead the fix when speech recognition errs more for non-native speakers?
Pin down the claim first. On an illustrative baseline error of 10%, "15% higher" means 11.5% if it is relative and 25% if it is absolute.1 Then measure word error rate per speaker group on matched audio, using consented or on-device aggregated data (see the privacy entry on training without exposing individual data), find whether the gap comes from the acoustic model or the training data, add consented samples from the under-served group, and re-measure. Agree the target gap with product, privacy and research owners before work starts.
Why it is asked: it tests precision about numbers and cross-team leadership. Read the full explanation: measuring AI products.
Why is overall accuracy not enough to show a model works for everyone?
Averages hide subgroup gaps. Koenecke et al. found that all five speech recognition systems they tested showed racial disparities, with a word error rate (WER) averaged across the five systems of 0.35 for black speakers against 0.19 for white speakers, a ratio of about 1.84 (Koenecke et al., March 2020).1 The paper studies race and dialect, not non-native speakers, so use it as the measured analogue: report error per group.
Why it is asked: it checks whether you slice metrics before declaring success. Read the full explanation: Koenecke et al., Racial disparities in automated speech recognition.
How do you compute a fairness gap from a confusion table?
For each group, count true positives among the people who qualify and false positives among those who do not. In an illustrative table, group A has 400 true positives among 500 qualified people (rate 0.8) and 100 false positives among 500 others (0.2); group B has 120 among 200 (0.6) and 40 among 300 (0.1333). The gaps are 0.2 and 0.0667, and the selection rates 0.5 and 0.32, although overall accuracy is 0.7867.1
Why it is asked: it shows whether you can do the arithmetic, not only name the metrics. Read the full explanation: measuring AI products.
What fixes a fairness gap, and what does the fix cost?
Try better features and more data first. If a gap remains, post-processing with group-specific thresholds avoids retraining and needs only aggregate data (Hardt et al., October 2016). It costs utility: at the 82% non-default threshold in their credit-score case study, a race-blind threshold keeps 99.3% of the maximal profit, equal opportunity 92.8%, equalized odds 80.2% and demographic parity 69.8%. Equal opportunity costs 7.2 points of profit against 30.2 for demographic parity, a ratio of 0.2384.1
Text description of the diagram
- A race-blind threshold keeps 99.3% of the maximal profit.
- Equal opportunity costs 7.2 points of profit against 30.2 for demographic parity.
Why it is asked: it tests whether you can state a fairness trade-off in business terms. Read the full explanation: Hardt, Price and Srebro, Equality of Opportunity in Supervised Learning.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →Privacy and data minimization
How do you keep users' data safe and private when building AI systems?
Start from the purpose: collect only what it needs, keep it only as long as needed, limit who can read it, and review what you hold. The National Institute of Standards and Technology (NIST) generative AI profile says such systems raise several risks to privacy, including wrong or inappropriate inferences about people that can contribute to downstream harm (NIST AI 600-1, July 2024). Say which controls you used and how you tested them.
Why it is asked: it checks for concrete controls instead of the word "encryption" alone. Read the full explanation: NIST AI 600-1, Generative AI Profile.
How can you train or evaluate a model without exposing individual users' data?
Start with pseudonymization and aggregation: replace direct identifiers and report group-level results. Differential privacy is a formal standard under which adding or removing one person's data changes a result only slightly, achieved by adding calibrated random noise (Dwork et al., 2006); the noise costs accuracy. Federated learning is training in which devices keep their data and send only model updates to a server that averages them (McMahan et al., 2016); the updates can still leak information.
Why it is asked: it checks that you know privacy techniques and their costs. Read the full explanation: Dwork et al., Calibrating Noise to Sensitivity in Private Data Analysis.
What is data minimization, and is it a legal rule?
Data minimization is the principle that personal data be adequate, relevant and limited to what is necessary for the purposes it is processed for. It is a legal principle of the General Data Protection Regulation (GDPR), Article 5(1)(c) of the GDPR and of the UK GDPR, as the British regulator's guidance quotes it (Information Commissioner's Office). Its tests: sufficient for the purpose, a rational link to it, and no more than you need.
Why it is asked: it separates a legal duty from a good intention. Read the full explanation: data minimization guidance from the British regulator.
Is data minimization the same as storage limitation?
No. Minimization limits how much data you collect and hold for a purpose; storage limitation is a separate principle limiting how long identifiable data is kept. The regulator's checklist says to review the data you hold periodically and delete anything you do not need (Information Commissioner's Office). A strong answer covers both: collect less and keep it for less time.
Why it is asked: candidates often treat minimization as a collection-time issue only. Read the full explanation: data minimization guidance from the British regulator.
Tell me about a time your code collected more user data than necessary.
Pick a real case and tell it in order: what you found, who was affected, what you did that week, and what changed in the process. A strong answer fixes the cause, such as a review step before a new field is logged, and not only the deletion. If you have no such case, describe a smaller real one honestly instead of inventing it.
Why it is asked: it tests ownership, honesty and whether you fix systems, not single incidents. Read the full explanation: data science interview questions.
Tell me about a time you redesigned an ML feature because a privacy review required data minimization.
Name the feature, the field the review challenged, the cheaper input you tested instead, and the measured effect on model quality. Replacing a full date of birth with an age band is one illustration. Show that you treated the review as a design input, report the quality cost honestly, and say what you now check before the next review.
Why it is asked: it shows whether privacy changes your design or only your paperwork. Read the full explanation: domain case rounds outside big tech.
Is responsible AI a legal checklist?
No. The NIST framework is voluntary guidance, intended to be voluntary by its own text, and it says trustworthy AI requires balancing its characteristics based on the context of use (NIST AI 100-1, January 2023). Data minimization under the GDPR, by contrast, is a legal obligation. Separate what the law requires from what good practice adds.
Why it is asked: it tests whether you can tell obligation from judgment. Read the full explanation: NIST AI 100-1.
Misuse, reliability and measurement
Tell me about a time a model you built had an unintended consequence or was misused by a stakeholder.
Describe the intended use, how it was used instead, how you found out (a metric, a complaint or a review), what you changed, and what you now do earlier. Own your part and do not blame the stakeholder. Typical fixes are documenting limits, restricting access, adding a monitoring metric, and requiring a review before reuse for a new purpose.
Why it is asked: it tests accountability and whether you learn at the process level. Read the full explanation: product and analytics cases.
How would you investigate a model giving overconfident but wrong answers in high-risk contexts?
Define the high-risk cases, such as medical or legal questions, and build an evaluation set with known answers. Measure how often the model is wrong and how confident it sounds, then compare the two with a calibration error or a reliability diagram (stated confidence plotted against how often the answers are right). Mitigate with cited sources, abstaining or handing over to a human, and test again. NIST calls valid and reliable a necessary condition of trustworthiness, the base for the other characteristics (NIST AI 100-1, January 2023).
Why it is asked: it checks whether you evaluate reliability before shipping. Read the full explanation: measuring AI products.
How would you evaluate the impact of fake news on Facebook users?
Define fake news operationally, for example a set labeled by fact-checkers. Then separate exposure (who saw it), engagement and outcomes such as changed belief or behavior. Observational comparisons are confounded because who sees such posts is not random; a randomized holdout or a survey after exposure is stronger. State the limits and the harm metric you chose.
Why it is asked: it tests causal thinking on a hard-to-measure harm. Read the full explanation: reading an A/B test like a product owner.
What are the trustworthiness characteristics of an AI system?
NIST lists valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed (NIST AI 100-1, January 2023). Creating trustworthy AI requires balancing them by context of use, and neglecting them can raise the probability and magnitude of negative consequences.
Why it is asked: it gives you a vocabulary for any scenario. Read the full explanation: NIST AI 100-1.
Follow-up questions
Which fairness metric should I mention first?
Start with the harm, then pick the metric whose error matches it. Equal opportunity suits a case where missing a qualified person is the harm; equalized odds also constrains false positives. Say what the choice costs, as the credit-score figures above show.
What if I have no real story for a behavioral question?
Use a smaller real case and say its scale honestly. An invented incident collapses under one follow-up, while a modest true one with a clear process change holds up.
Do these questions appear outside big tech?
Reported prompts here came from large companies, but the same fairness and privacy reasoning applies to any team that ships models; see domain case rounds outside big tech. For a feed-ranking angle, read the Meta product architecture interview guide.
Are the company names on these questions confirmed?
No. They are what candidates reported, with no public source from the companies, so treat them as examples of the style and not as a claim that any company asks them.
- Name the harm, pick a measurable per-group quantity, apply a fix and state its cost.
- Removing a sensitive attribute does not remove bias, and a fairness metric is not proof of fairness.
- Equal opportunity cost far less profit than demographic parity in the cited credit-score case study.
- Data minimization limits what you collect; storage limitation limits how long you keep it.
- NIST's framework is voluntary guidance; the GDPR's minimization principle is a legal duty.
Sources
- NIST AI 100-1, Artificial Intelligence Risk Management Framework (AI RMF 1.0) — NIST (US Department of Commerce), 26 Jan 2023
- Calibrating Noise to Sensitivity in Private Data Analysis — Theory of Cryptography Conference (Dwork, McSherry, Nissim, Smith), 4 Mar 2006
- Communication-Efficient Learning of Deep Networks from Decentralized Data (arXiv 1602.05629) — arXiv (McMahan et al.), 17 Feb 2016
- Equality of Opportunity in Supervised Learning (arXiv 1610.02413) — arXiv (Hardt, Price, Srebro), 7 Oct 2016
- Racial disparities in automated speech recognition — Proceedings of the National Academy of Sciences (Koenecke et al.), 23 Mar 2020
- Data minimisation (UK GDPR guidance) — Information Commissioner's Office, 26 Oct 2026
- NIST AI 600-1, Generative AI Profile of the AI RMF — NIST (US Department of Commerce), 26 Jul 2024
