AI Interview Bias: How to Check Your Hiring Tool Before a Regulator Does

What if the fastest part of your hiring process is also the one nobody has checked for bias?

That is where many employers using AI interviews stand in 2026. AI interview bias is the risk that an AI-assisted interview scores candidates differently for reasons unrelated to the job. The cases on record did not come from rogue algorithms. They came from ordinary design choices, and each has a known control.

The numbers behind this article:

• $365,000 paid by iTutorGroup after its software automatically rejected older applicants
• 0.35 vs 0.19 speech-to-text error rates for Black and white speakers across five commercial systems
• 17 vs 1 potential compliance problems found at 32 New York City employers: at least 17 by state auditors, 1 by the city regulator
• .42 validity of structured interviews, the strongest predictor of job performance on record
• 80% the four-fifths threshold for spotting adverse impact

What follows: four real cases, an 8-point check you can run on your own hiring tool, and what the law asks of employers this year.

AI interview bias, defined in one sentence

AI interview bias is a difference in how AI interview software scores, ranks, or screens candidates that tracks a protected characteristic, such as race, sex, age, or disability, and not the requirements of the job.

It can enter at any point where software shapes the outcome: the questions asked, the speech-to-text transcript, the score, the ranking, and the weight a recruiter gives that ranking.

Two things separate it from the bias of a single human interviewer:

• Scale. One inconsistent interviewer affects a handful of candidates. One flawed scoring rule affects everyone who passes through it. In Mobley v. Workday, a US federal court conditionally certified an age-discrimination collective in May 2025 that lawyers following the case say may include millions of applicants.
• Traceability. Software leaves records. That makes bias easier to find than a hiring manager's instinct, and harder to defend if nobody looked.

Amazon, iTutorGroup, HireVue and a speech study: four ways AI interview bias got in

What does AI interview bias look like when it reaches a regulator, a courtroom or a headline? Here are four cases. None involved a system built to discriminate.

Amazon: ten years of resumes, one learned bias

Amazon built a tool to score resumes from one to five stars and trained it on about ten years of its own applications, most of them from men, according to Reuters.

The outcome:

• Resumes containing the word "women's" were penalized
• Graduates of two all-women's colleges were downgraded
• Amazon disbanded the project

The lesson: a model trained on past hiring decisions repeats them.

iTutorGroup: a rule, not a model

According to the EEOC's suit, the company's application software was set to reject women aged 55 or older and men aged 60 or older automatically.

The outcome:

• More than 200 qualified applicants rejected
• $365,000 settlement in September 2023
• Brought under the Age Discrimination in Employment Act

The lesson: bias is not always learned. A configured cut-off counts, and the employer answers for it.

HireVue: dropping a signal that did not earn its place

HireVue announced in January 2021 that it had removed facial analysis from its video interview assessments.

The outcome:

• Its own research found visual analysis no longer significantly added value
• Language analysis had become the stronger predictor

The lesson: a signal that does not predict job performance adds risk and nothing else.

The speech study: when the transcript is the problem

Researchers tested five commercial speech-to-text systems, from Amazon, Apple, Google, IBM, and Microsoft (Koenecke et al., PNAS, 2020).

The outcome:

• 0.35 average word error rate for Black speakers
• 0.19 for white speakers
• 115 speakers and 19.8 hours of audio

The lesson: every voice interview depends on a transcript. If the transcript is worse for some speakers, the score is too.

The study is from 2020, and the systems have changed since. The point for buyers stands: ask any AI interview vendor for current transcription accuracy by accent and speaker group.

The 8 sources of algorithmic hiring bias in an AI interview

Algorithmic hiring bias has eight common sources in an AI interview. Each has a matching control, which is more useful than a general promise of fairness.

Structured interviews outpredict every other method, so start there

What if the most reliable interview method on record is also the one AI makes cheapest to run?

It is. In a structured interview, every candidate for a role gets the same questions, in the same order, scored against the same written rubric.

The evidence:

• .42 average validity, the highest of any selection method in a 2022 review by Sackett, Zhang, Berry and Lievens in the Journal of Applied Psychology
• .31 for cognitive ability tests, long treated as the benchmark
• The US Uniform Guidelines call for selection procedures to be administered and scored under standardized conditions
• Where an unscored procedure shows adverse impact, the same guidelines direct employers to remove the impact or move to a scored procedure that can be validated

What that gives you against bias:

•  Comparability. Scores mean the same thing for every candidate, so differences reflect answers and not the interviewer or the day.
•  Job relevance. Each question maps to a skill the role needs, which is the standard regulators apply to any selection procedure.
• Auditability. With fixed questions and criteria, you can test results by group and trace a score back to the evidence.

AI makes structured interviews cheaper to run at volume. It does not replace the need for them. AI layered on unstructured interviews scales the inconsistency.

The 8-point check for AI interview bias

Could your AI interview process pass these eight checks today? Each one maps to a risk above, and most are process decisions an HR or talent leader can own without a data science team. The pass mark for every check is something you can show, not something you believe.

1. Is every scored criterion tied to the job? Pass: a written list of the skills and behaviours the role needs, with each rubric item traced to it.
2.  Does every candidate for a role get the same questions? Pass: a fixed question set per role, and a rubric that describes a strong, acceptable and weak answer.
3. Do you know exactly what the AI scores? Pass: the vendor confirms in writing that scores come from what candidates say. Facial expression, tone of voice and similar signals stay out unless they are shown to predict performance in your roles.
4. Does a person make the decision? Pass: a named, trained reviewer sees the recording, transcript and score and can override. Colorado's new law describes the same thing: someone who can approve, modify or override and does not simply accept the output.
5. Have you tested outcomes by group? Pass: selection rates compared across groups at each stage within the last 12 months, and again after any change to the tool or the rubric.
6. Do candidates know, and is there another route? Pass: a notice that AI is used and how, plus accommodations and an alternative format on request.
7. Could you produce the records tomorrow? Pass: questions, rubrics, scores, reviewer decisions, model versions and test results on file. California asks for four years and Colorado for three.
8. Have you listed every automatic cut-off? Pass: an inventory of knockout questions and minimum scores, each with a job-related reason. The iTutorGroup case turned on a configured rule, not a model.

Any check you cannot evidence is a gap to close before your next hiring round.

The four-fifths rule: a two-minute adverse impact check

The Uniform Guidelines describe a four-fifths rule of thumb. If one group's selection rate is less than 80% of the rate for the group with the highest rate, enforcement agencies will generally treat that as evidence of adverse impact.

An illustrative example: 60 of 100 candidates in one group pass the AI interview stage, and 42 of 100 in another. The ratio is 42% divided by 60%, or 70%. That is below 80%, so the stage needs a closer look.

The rule is a screen, not a verdict. The guidelines note that smaller differences can still amount to adverse impact when they are statistically and practically significant, and that results based on small numbers may not be reliable. Passing the four-fifths rule does not prove a process is fair.

Colorado, the EU, Illinois, New York: what employers owe in 2026

Who answers when an AI interview tool gets it wrong? The employer does, including when the tool is bought from a vendor. The Uniform Guidelines say so directly: employers who rely on a publisher's validity evidence are cautioned that they are responsible for compliance.

The rules moved more in the past twelve months than in the three years before. Four changes matter most:

• Colorado replaced its AI Act before it took effect. SB 26-189 dropped the 2024 law's broad duty to prevent algorithmic discrimination. In its place are notice, an explanation within 30 days of an adverse decision, human review on request, data correction and three years of records.
• The EU postponed its high-risk rules. Regulation (EU) 2026/1744, published in July 2026, moved obligations for recruitment systems from August 2, 2026 to December 2, 2027.
• Illinois went live. Its amendment to the Human Rights Act has applied since January 1, 2026.
• New York City's enforcement was found wanting. A state comptroller audit in December 2025 reviewed 32 companies. The city's regulator had found one compliance issue among them; the auditors found at least 17. The regulator had received two complaints in two years.

The table shows where each rule stands in October 2026. Confirm the current position with counsel before relying on it.

Three practical points follow:

• A lighter statute does not mean lighter liability. Colorado's new law creates no private right to sue, but it leaves existing discrimination claims, including disparate impact, untouched.
• Disability and age are covered separately. The Uniform Guidelines address race, sex, and ethnic group. The Americans with Disabilities Act and the Age Discrimination in Employment Act apply to AI tools too.
• Vendors may share exposure, but employers do not lose theirs. Mobley v. Workday is still being litigated, and no liability has been decided.

Employers hiring outside the US and EU, including in the UAE, should check local employment and data protection law. This section is general information, not legal advice.

8 questions that test a fair hiring technology claim

How do you tell fair hiring technology from a fairness claim? Ask questions whose answers you can check. Each one below comes from a case or a rule in this article, and a vague answer is an answer.

Be wary of any vendor that describes its product as bias-free. No selection method, human or automated, can promise that. The credible claim is narrower: here is what we measure, here is how we test it, and here is how your team stays in control.

Review responsible AI controls with CloudInterview

What starts as a compliance question ends as a hiring advantage: a process that is consistent, quick to run, and one you can explain to a candidate, a client, or a regulator.

CloudInterview is built on the two controls this article treats as the foundation:

• Structured by design. AI Interviews run structured voice, video, and technical rounds and return consistent scores with detailed feedback.
• Human-led decisions. AI assists with the volume, and your team makes the final hiring call.

A fairer way to scale hiring.

• See how candidate data, access controls, and responsible AI use are handled on the Security page.
• Learn who is behind the platform on the About page.
• Book a demo to review the controls against your own roles and hiring volume.