In this article
Before you start
This question belongs to the behavioral interview questions guide; the closest neighbour is the page on a model that failed in production, which covers the model-side story, while this page covers the answer you say aloud.
Why interviewers ask this
Interviewers ask about a failed AI feature because a launch story shows what a candidate does when the model stops behaving, not when it works. A failure that nobody noticed for weeks tests judgment, ownership and engineering habits at once.
A weak answer picks a trivial slip, blames another team, or describes only the model. A strong answer names a real failure, shows how it was detected, states what the candidate personally did, and says what changed afterwards. A Staff-level answer adds the system-level fix: the monitor, the rollout rule or the ownership change that makes the same failure loud next time.
Reported practice differs between companies on how this is scored, and no primary source defines it, so treat the rubric below as advice. The same shape also works for the greatest weakness question, where a bounded failure and a lasting change are what the listener wants.
An AI feature failure answer from first principles
- Post-mortem
- A post-mortem is a written review of an incident that records what happened, why, how it was found and what will change. Its value to an interview answer is that it gives you a real, ordered story with a lesson attached.
In plain words
Picture a delivery van whose fuel gauge sticks. The van keeps driving and nobody complains until it stops on a motorway. The analogy breaks in one place: a model does not stop, it keeps delivering, only to the wrong addresses more and more often.
In one precise sentence, a good answer is a STAR story about a silent model failure. Most of its time goes to detection, your action and the new guard. The reason to pick this shape is that Google's own guidance frames most production problems as engineering problems rather than algorithm problems, so the story should show engineering habits.
Step 1: Choose a failure and state the stakes in two sentences
The first step is to pick one real failure and give its situation and task in two sentences at most. Recruiter guidance on the STAR convention (a way to order a story, not a standard) advises one or two sentences for the situation, one for the task, and most of the time for action and result (4 Corner Resources). The same page warns that if you are still describing background partway in, you run out of time before the good part.
Input: your memory of an incident. Output: a feature, a user group and a cost, such as "a ranking feature on a search page, where new products lacked a feature and sessions fell back to default results". Choose a failure where you had a real role. You now have the setting; the next step explains why the failure was hard to see.
Step 2: Show how the failure stayed silent and how you found it
The second step explains why nothing alerted anyone, then how you found the problem. Rule 10 of Rules of Machine Learning is "Watch for silent failures", and it gives an example: if a joined table stops updating, the system adjusts and behaviour decays gradually. The same page notes a feature column populated in 90% of examples can drop to 60% after an implementation change.
Then say how you detected it. Ground truth is often missing in production, so teams watch proxy metrics (Google for Developers). Its example is a spam-folder rate rising from 0.5% to 3%, and the numbers below show the jump is six-fold.1
Text description of the diagram
- A spam-folder rate rising from 0.5% to 3% is a six-fold jump.
- Comparing the changes in proxy metrics is more insightful than their raw numbers.
The same source notes that comparing the changes in proxy metrics is more insightful than their raw numbers. A feature column falling from 90% to 60% coverage is a loss of 30 points, which is a one-third relative drop.1
Two named causes: skew and drift
Training-serving skew is a gap between the feature data a model sees in production and the data it was trained on, which shows up as lower live quality. Google Cloud describes it as production feature data deviating from the training distribution. Data drift is a change in the production feature distribution over time; the same document calls it inference drift. Breck et al. add that an upstream change can alter a feature's meaning without producing values strange enough to trip other monitors. You now know the failure, its likely causes and the clue; the third step is what you did.
Step 3: Say what you personally did, in order
The third step is the longest part of the answer, because action is where interviewers learn your behaviour. Recruiter guidance says action and result should get most of the answer's time (4 Corner Resources); treat that as a rule of thumb, not a standard.
Order the actions: contain, diagnose, fix, then review. Contain means stopping the damage. If the model is the problem, roll back to the last good model. If an input is the problem, a model rollback does not help, because the older model reads the same stale input; switch the ranker to skip the affected feature or serve a safe default. Diagnose means finding the cause, such as a stale join, a changed column or a feature computed differently in serving than in training. Fix means repairing the pipeline, not only retraining; Rules of Machine Learning shows a table stale for 6 months where refreshing it alone gave a 2% boost in install rate. Review means writing the post-mortem.
You now have the action; the last step turns it into a lesson.
Step 4: Close with the result and the guard you added
The last step states the result and the guard, a check that makes the same failure loud. Name one outcome, then the mechanism. Candidate guards from the sources include notifying on upstream dependency changes, checking input data invariants, measuring training and serving skew, and alerting on stale models (Breck et al.). Rules of Machine Learning also advises sanity-checking a model before export, giving feature columns owners and practising good alerting hygiene, such as actionable alerts. Breck et al. also describe sending a new model to a fraction of traffic first and increasing it as the model behaves sanely.
End with one sentence of lesson, because a failure story is not finished until it says what you now do differently.
Step-by-step description
- Set the scene in two sentences.
- Show how it was found.
- Say what you did, in order.
- End with the guard and the lesson.
- The answer spends most of its time on action and result.
- A failure story names how the failure was found.
The answer spends most of its time on action and result, and a failure story names how the failure was found.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →The Staff-level answer
A strong answer is short enough to say in about a minute. At an illustrative speaking pace of 130 words per minute, a 126-word answer takes about 58 seconds.1 The answer below is an illustrative composite, not a real person's story.
Tell me about an AI feature you shipped that failed in production.
I owned a ranking feature behind a product search page. A weekly check showed the share of sessions that fell back to default results had risen from one percent to six percent, though accuracy on our held-out set looked fine. The cause was an upstream table that stopped updating, so new products had no row, the feature was missing and the ranker fell back to default ordering.
I told the ranker to skip that feature, restored the table's refresh job, backfilled the missing days and wrote the post-mortem. Then I added alerts on feature coverage and freshness, plus a staged rollout for every model change. The lesson is that a model can fail without any error, so I now always ask what would tell us first.
What makes it Staff-level: the failure is silent, the fix is a system guard rather than a retrain, and the last sentence changes how the candidate works.
The same fallback rate rising from one percent to six percent is a six-fold jump, computed from illustrative values.1 Compare a weak version: "Our model got worse, so we retrained it and it was fine." It has no detection, no personal action, no guard and no lesson.
The rubric
| Dimension | Weak answer | Strong answer | Staff answer |
|---|---|---|---|
| Choice of failure | A trivial slip, or a crash with an obvious cause | A real model-quality failure in a live feature | A silent failure with a clear cost to users |
| Detection | "Someone told us" | A named metric or check that exposed it | A proxy metric, and why accuracy alone could not see it |
| Your action | "We fixed it" | "I" steps in order: contain, diagnose, fix | Fix at the pipeline level, plus the post-mortem you wrote |
| Safeguard | None | One new alert or check | A guard that makes the failure loud, with an owner |
| Lesson | Vague ("communicate more") | One specific habit | A changed practice the team still uses |
Use it as advice, not as any company's scoring sheet: no primary source publishes one.
Three variants of this question
Tell me about a time a model you built did not work as expected after launch
Use the same four steps, but spend the opening sentence on the expectation that was wrong, such as launch metrics that looked good on the held-out set. The story is still detection, action and guard.
Describe a production incident involving an AI system and how you handled it
Lead with the incident timeline in order: what was noticed, how it was contained, who was told, what the fix was. An interviewer here listens for calm sequencing and for the post-mortem more than for the model detail.
What would you do differently on an AI feature you shipped?
Answer with the guard you now add by default: monitoring of inputs and outputs before launch, a staged rollout, and a named owner for each feature. Keep the failure to one sentence, because the question asks about the change.
Common mistakes
- "The model crashed and we restarted it." (wrong) A crash is loud and says nothing about machine learning judgment. (right) "The model kept answering, but quality decayed silently, and here is what exposed it."
- "Accuracy was great on our test data." (wrong) Production can differ from training through skew and drift. (right) "Held-out accuracy looked fine, so I looked at a proxy metric in production instead."
- "We just retrained the model." (wrong) Skew can come from different code paths or a stale joined table, which retraining does not repair (Google for Developers). (right) "I fixed the pipeline first, then retrained."
- "The data team broke it." (wrong) Blaming others hides your own action. (right) "An upstream change broke it, and I owned the detection and the guard."
- "It was a learning experience." (wrong) That is not a lesson. (right) "I now check feature coverage and freshness before every launch."
For wider answer-shape advice see how to answer interview questions and the list of common interview questions.
Follow-up questions
Should I pick a failure that was my fault?
Pick a failure where you had a real role, not necessarily the only one. A story with no personal action tells the interviewer nothing about you, and a story that hides the failure loses the point.
How do I find the failure if there is no ground truth?
Watch proxy metrics, such as a fallback rate or the share of users moving mail to spam, and compare their change over time rather than their raw level (Google for Developers).
What if the failure was caused by another team?
Say so in one neutral sentence, then focus on what you did: detection, containment and the guard, such as a notification on dependency changes. The ownership you took is the story.
Is retraining the model the right fix?
Not by itself. Retraining cannot repair a different code path in serving or a stale input, so check the pipeline and the data first, then retrain if the data was the cause.
How long should the answer be?
About a minute fits a first answer, with room for follow-ups. One recruiter guide suggests 60 to 90 seconds (4 Corner Resources); treat that as a rule of thumb, not a standard. Prepare a short version and a longer one, and have a question ready for the end; the page on questions to ask an interviewer helps there, and so does why should we hire you as the next answer to rehearse.
- Choose a silent failure, not a crash: production models often decay gradually without any error.
- Say how you detected it, using a proxy metric when ground truth is missing.
- Spend most of the time on your own action and the result.
- Close with the guard you added and one specific lesson.
- Do not blame another team and do not stop at retraining.
Sources
- Rules of Machine Learning: Best Practices for ML Engineering — Google for Developers (Zinkevich), 25 Aug 2025
- Productionization (managing ML projects) — Google for Developers, 25 Aug 2025
- The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction — Google Research (Breck et al., IEEE Big Data 2017), 11 Dec 2017
- Introduction to Model Monitoring — Google Cloud Documentation, 15 Oct 2026
- Guidelines for developing high-quality, predictive ML solutions — Google Cloud Architecture Center, 8 Jul 2024
- The STAR Method for Interviews: A Complete Guide With Examples — 4 Corner Resources (recruiter, secondary), 13 Aug 2026
