← Guides

AI hiring bias, and what the law now says about it

Bias isn't a hypothetical risk here — it's measured

Our guide on the agentic problem covers how AI CV matching actually works. This one goes deeper on a specific, uncomfortable finding in that research: AI-based resume screening doesn't just risk incompleteness — it has documented, quantified bias, and that bias isn't stable over time.

A body of academic research on LLM-based resume screening has found a striking pattern: the direction of racial bias in callback simulations has reversed across model generations. Screening systems built on 2023-era models reproduced a bias favoring white-coded names; systems built on 2024-and-later models showed the opposite pattern, or no measurable gap at all. The finding that should actually worry anyone relying on "we use a newer model now" as a fix: bias didn't disappear as models improved — it moved. There is no evidence the industry has found a stable, model-generation-independent way to eliminate it, only that its shape keeps changing.

Anonymizing a CV doesn't fix it either

A natural response is "just strip the name and photo before an AI screens it." A March 2026 study found this doesn't work the way people assume. Even with names, photos, and addresses removed, large language models can still infer race, gender, and socioeconomic status from subtler textual signals — word choice, listed extracurricular activities, university affiliations, even sentence-level writing style. The study's own framing is the useful part: small changes to otherwise-neutral resume content produced statistically significant shifts in how an LLM scored the same underlying candidate. Anonymization removes the obvious signals; it doesn't remove the inferable ones.

There's a related, more technical root cause worth knowing: bias doesn't only enter at the screening-decision layer — it's baked into the language-model embeddings underneath. Male-coded words cluster near career and technical terms in these models; female-coded words cluster near family and care terms, a pattern learned from training data. When a resume-ranking system uses those same embeddings to judge candidates, that geometric bias becomes ranking bias directly, before any explicit "decision" is even made.

The law has caught up, at least on paper

Two regulatory frameworks now put real, specific obligations on employers using AI to screen candidates — this isn't a self-regulation-only space anymore:

The EU AI Act (Regulation (EU) 2024/1689) classifies any AI system used to recruit, filter, or evaluate job candidates as high-risk, under Annex III, point 4(a) — and this classification is explicitly not discretionary. High-risk status triggers a real obligation stack: risk management (Article 9, including continuous monitoring — bias as an ongoing thing to watch, not a one-time check), data governance, technical documentation (Article 11), transparency requirements (Article 13, pushing vendors away from fully black-box systems), and human oversight. Full compliance is required by December 2027.

NYC Local Law 144 has been in active enforcement since July 5, 2023 (well ahead of the EU timeline), administered by the city's Department of Consumer and Worker Protection. It prohibits an employer from using an "automated employment decision tool" unless an independent third party has conducted a bias audit within the past year, examining selection rates by race, ethnicity, and gender, and calculating impact ratios — under the common "four-fifths rule," a selection rate for any group below 80% of the highest-selected group's rate signals adverse impact. A follow-up 2025 academic audit of the audits themselves found real methodological inconsistency across the bias audits companies have actually published — the law requires an audit, but audit quality under it still varies considerably in practice.

What this actually means, practically

  • If you're a job seeker, the takeaway from our companion guide still holds: literal, complete detail matters more than a human reader's ability to infer — but it's also worth knowing that the system judging you carries documented, moving bias that isn't fully solved yet, so a bad outcome isn't necessarily a signal about your actual qualifications.
  • If you're an employer or a recruiter using one of these tools, "we use a reputable vendor" isn't a compliance answer by itself under either framework — the EU AI Act's obligations attach to the deployer, not just the tool vendor, and NYC's law specifically requires an independent audit, not a vendor's own internal testing.
  • Vendor transparency remains genuinely limited. Commercial screening vendors largely don't publish their model architectures or training data composition — there's a real, documented gap between what academic research can study and what's actually running inside deployed commercial tools. The EU AI Act's technical-documentation requirements may eventually force more disclosure, but as of today, most of this remains proprietary.

This is a fast-moving legal and technical area on both sides of the Atlantic — treat this as orientation, not a substitute for real legal advice specific to your situation.

Related guides