In this article
Before you start
Part of the wider set of engineering manager interview questions.
Why interviewers ask this
A hiring manager's decisions shape who joins the team for years, so a biased process costs fairness and the quality of the people hired. Candidates report this question in engineering manager loops, usually inside the people management round, as a check on whether you treat hiring as a system you can design or as a judgment you trust yourself to make.
A weak answer says "I hire on merit" or "we are an equal opportunity employer". That states intent and describes no control. A strong answer names where bias enters and gives a control for each point: a written bar, independent scoring, an evidence-based debrief and a check of outcomes. A Staff-level answer also says what the controls cannot fix and how you would find out.
Bias in hiring from first principles
- Anchored rubric
- An anchored rubric is a written scoring guide that lists each skill being hired for and describes, in observable behavior, what a low, middle and high score looks like on a numbered scale, so that two interviewers can compare their scores.
- Independent scoring
- Independent scoring is the practice of each interviewer writing down scores and evidence before the debrief, so that no one's rating is shifted by hearing a colleague's opinion first.
In plain words
Think of judges at a cooking contest. Every judge gets the same scoring sheet, tastes the same dishes and writes a score before anyone speaks. One loud opinion cannot sway the rest. Hiring can work the same way.
The analogy breaks in one place. Judges can taste a dish without knowing the cook. Interviewers see the candidate, so you can hide some signals and must control the rest.
Two habits cause most trouble. The halo effect is the habit of letting one strong trait color every other rating. Affinity bias is the tendency to favor people who resemble you.
In one sentence: a fair hiring process fixes the standard in writing, collects each judgment separately and checks the outcomes, because good intentions remove neither habit.
Step 1: Name where bias enters, because a policy statement is not a control
Bias enters hiring wherever one person judges another, and the first place it can show is the resume screen, so naming those points comes before designing any control. In the best-known field experiment, Bertrand and Mullainathan answered help-wanted ads in Boston and Chicago newspapers with fictitious resumes, and each resume was randomly assigned either a very White sounding or a very African American sounding name. White names received 50 percent more callbacks for interviews. Because the name was assigned to each resume by the authors, the name is what differed between the applicants.
Employers who listed "Equal Opportunity Employer" in their ad discriminated as much as other employers, according to the working paper. A stated policy is therefore not a control.
List every point where a person judges another: the resume screen, each interview, the debrief and the offer. Each needs its own control. You now know why the next three steps are mechanisms, not promises.
Step 2: Write the bar before you see candidates
Writing the hiring bar down before you meet any candidate stops you from fitting the criteria to the person you like. The rubric is written before any candidate is seen. List the four or five skills the role needs (illustrative), and for each one describe what a low, middle and high score looks like in observable behavior.
For an ownership criterion, a low anchor might be "explains a missed deadline by pointing at other teams". A middle anchor might be "fixes their own part and flags the rest". A high anchor might be "finds the cross-team cause and gets it fixed". Anchors turn "seems strong" into something two interviewers can compare.
Then write the questions that surface each criterion and ask every candidate the same ones in the same order. This is the core of a structured interview, which Levashina et al. (2014) review as standardized questions and scoring. Weight the criteria by the job, not by the candidate in front of you. You now have the standard that everything else is measured against.
Step 3: Score alone and hide what the role does not need
Scoring alone, and hiding signals the role does not need, stops one impression from spreading to every rating and every colleague. Each interviewer writes a score and the evidence for each criterion before the debrief. Without that, the first strong opinion in the room can shift the rest.
At the resume step you can try removing names and other details that say nothing about the work, then check whether it changes who advances. This applies only at that stage, because interviews show the person.
Score each criterion separately and straight after the interview, so a charming opening does not color the technical answer. Write what the candidate said or did, not "seemed senior". Evidence is what lets someone else check your number later.
You now have separate, comparable judgments instead of one blended impression.
Step 4: Debrief on evidence, then audit the funnel
The debrief settles the decision on evidence, and an audit afterwards checks whether the process itself leaks. Each interviewer scores alone before the debrief, then the scores are shown together. Spend the time on the rows where scores disagree most and ask each person for the observation behind their number.
A phrase such as "not a culture fit" is not accepted until it is restated as a behavior tied to a criterion. Outcomes are checked by stage after the decision: how many applicants from each group are screened in, interviewed and offered. In the US, the adverse-impact ratio (the "four-fifths rule" of the Uniform Guidelines, 29 Code of Federal Regulations (CFR) 1607.4(D)) compares each group's selection rate with the highest group's rate, and a rate below four-fifths of it is generally regarded as evidence of adverse impact. Do this with your recruiting partner and legal guidance, and treat small numbers as a prompt to look, not a verdict.
The audit tells you whether the other three steps are working.
Step-by-step description
- List every point where a person judges another.
- Fix the bar before seeing any candidate.
- Write scores and evidence before the debrief.
- Decide on evidence, then check outcomes by stage.
- The rubric is written before any candidate is seen.
- Each interviewer scores alone before the debrief.
- Outcomes are checked by stage after the decision.
Worked example 1: the callback gap per 100 resumes
The callback gap becomes concrete when it is put per 100 resumes. The digest (2003) reports that applicants with White names needed to send about 10 resumes to get one callback, and applicants with African American names around 15. For 100 resumes per group, that is about 10 callbacks for White-sounding names against about 6.7 for African American-sounding names.1 The implied ratio is 1.5, derived from rounded figures.
This is a derivation, not a new finding. Quote the paper's own figure of 50 percent more callbacks if asked.
One limit belongs in a careful answer. The working paper measures callbacks for interviews, which is the first screen only. It shows bias at that stage. It does not show that every later stage is biased.
Text description of the diagram
- That is about 10 callbacks for White-sounding names against about 6.7 for African American-sounding names.
- The implied ratio is 1.5, derived from rounded figures.
Worked example 2: scoring two candidates the same way
Independent scores averaged per criterion show where two candidates really differ. In this illustrative example, three interviewers each score two candidates on four criteria, on a scale of one to five, before the debrief.1 Candidate A averages 3.67 and candidate B averages 3.5, so a gut feeling that B "interviewed better" would not match the written scores.
| Criterion | Candidate A (mean) | Candidate B (mean) | Range of B's scores (high minus low) |
|---|---|---|---|
| Problem decomposition | 3.67 | 4.0 | 2 |
| Ownership | 3.67 | 3.33 | 1 |
| Coaching others | 3.33 | 2.33 | 1 |
| Communication | 4.0 | 4.33 | 1 |
Candidate B's problem decomposition scores range over two points (5, 3 and 4), so the debrief starts there: what did each interviewer see? Ask each person for the observation behind their number, and let the evidence, not the average, settle any disagreement. Coaching is B's weakest row, which matters for a people-lead role. The numbers start the conversation; the evidence settles it.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →The Staff-level answer
How do you ensure that biases don't impact your hiring decisions?
I treat bias as a process problem, not a character problem, so I build controls instead of relying on good intentions. Before I see anyone, I write the bar: four or five criteria with anchored scores. Every candidate gets the same questions, and each interviewer writes scores and evidence before the debrief, so the loudest voice does not set the room. I test removing names and similar details at the resume step and check whether it changes who advances. In the debrief I argue from evidence, and I check pass-through by stage afterwards. That reduces bias; it does not eliminate it, so I would rather look for a leak in the data than assume there is none.
What makes it Staff-level: it names the limit, and it measures outcomes instead of declaring the process fair.
The rubric
| Dimension | Weak answer | Strong answer | Staff answer |
|---|---|---|---|
| Correctness | Says "I hire on merit" and stops. | Names where bias enters and gives one control per point. | Also explains why a stated policy alone did not close the gap in the field experiment. |
| Depth | Lists "be fair" as a value. | Explains the rubric, independent scoring and debrief rules. | Explains why each control works and where it stops working. |
| Trade-offs | Claims bias can be eliminated. | Admits controls reduce bias and cost some speed and flexibility. | Weighs structure against candidate experience and tunes it using outcome checks. |
| Communication | Vague or defensive. | Clear sequence: bar, scoring, debrief, audit. | One-line principle first, then a concrete example from a past loop. |
| Follow-up readiness | Stuck when asked about uneven data. | Can say what they would check next. | Knows when to involve legal and recruiting partners and how to read small numbers. |
Three variants of this question
How do you reduce bias when you screen resumes?
Remove names and other details that say nothing about the work, then score every resume against the same written criteria. Review the screen's outcomes by group afterwards, because the experiment above shows the gap can start at this stage.
A panelist says they just did not click with a candidate. What do you do?
Ask for the behavior behind the feeling and map it to a criterion on the rubric. If there is evidence, it counts; if there is only a feeling, it does not change the score, and affinity bias may be at work.
How do you keep referrals from skewing who gets hired?
Referrals come from the networks of people already on the team, so the pool can narrow. Run referred candidates through the same screen, the same questions and the same rubric, and track how each source performs at each stage.
Common mistakes
- "We are an equal opportunity employer, so we are covered." (wrong) In the working paper by Bertrand and Mullainathan, employers who listed that phrase discriminated as much as others. (right) "A policy states intent. A rubric and an audit are controls."
- "Better credentials fix it." (wrong) In the same experiment a higher quality resume brought White names 30 percent more callbacks and African American names a far smaller increase, according to the working paper. (right) "I check whether strong candidates from every group advance at the same rate."
- "We discuss the candidate first and then score." (wrong) The first strong opinion can shift the rest. (right) "Everyone scores alone first, then we compare."
- "I hire on gut because I read people well." (wrong) A gut call is unwritten and cannot be checked. (right) "My gut starts a question; the evidence answers it."
- "Culture fit." (wrong) It is vague enough to hold any bias. (right) "Name the behavior and the criterion it belongs to."
Follow-up questions
Does a structured interview remove bias?
No. A structured interview standardizes questions and scoring (Levashina et al., 2014), which reduces the room for improvisation and makes scores comparable, but interviewers still see the candidate, so you keep the debrief rules and the audit. Say "reduce", not "remove".
What if your own funnel numbers look uneven?
Treat it as a prompt to investigate, not a verdict. Check the sample size, look at which stage the gap opens, review that stage's rubric and notes with your recruiting partner, and involve legal guidance before changing process.
Should you rely on unconscious bias training?
Do not make training the control. A written bar, independent scoring and outcome checks are what you can inspect and test, so build those first and treat training as a supplement.
How does this fit into running a whole hiring loop?
The controls sit at each stage of the loop, from the screen to the offer. The page on hiring as an engineering manager covers how to run the loop and decide.
How do you keep bias out of promotions and feedback?
Use the same pattern: criteria written first, evidence recorded, a comparison across people. The pages on growing engineers and one-on-ones and hard feedback cover the day-to-day side, and your management philosophy should say the same thing.
- Name the points where one person judges another; a stated equal-opportunity policy is not a control.
- Write an anchored rubric before you see any candidate, and ask every candidate the same questions.
- Score alone with evidence before the debrief, and argue from evidence in the debrief.
- Check outcomes by stage after decisions, with recruiting and legal partners, and read small numbers carefully.
- Say that controls reduce bias rather than eliminate it.
Sources
- Are Emily and Greg More Employable than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination (NBER Working Paper 9873) — National Bureau of Economic Research (Bertrand and Mullainathan), 1 Jul 2003
- Are Emily and Greg More Employable than Lakisha and Jamal? (NBER WP 9873, full text) — NBER (Bertrand and Mullainathan), 1 Jul 2003
- Employers' Replies to Racial Names (NBER Digest) — National Bureau of Economic Research (David R. Francis), 1 Sep 2003
- The structured employment interview: narrative and quantitative review of the research literature (Personnel Psychology) — Personnel Psychology (Levashina, Hartwell, Morgeson and Campion), 1 Mar 2014
- Uniform Guidelines on Employee Selection Procedures, 29 CFR 1607 — US Equal Employment Opportunity Commission (eCFR), 25 Aug 1978
