AI resume screening
Resume screening tools range from keyword matching to trained ranking models, and the failure modes differ sharply between them. Knowing which you are buying determines what can go wrong and what oversight it needs.
| Topic | Key point |
|---|---|
| Three different things are sold under one name | "AI resume screening" covers approaches that behave differently and fail differently. |
| The label problem, which no amount of model quality fixes | A ranking model needs something to predict. |
| What screening genuinely does well | The criticism above is about ranking. |
| How to evaluate one honestly | Run it on a past requisition where you know the outcome, and check whether it would have surfaced the person you actually hired — and whom it would have dropped. |
| The compliance layer sits on top of all of this | In the EU, employment AI is high-risk under the AI Act, which brings obligations for both the vendor and the employer deploying it — human oversight, |
Three different things are sold under one name
"AI resume screening" covers approaches that behave differently and fail differently. Establish which you are being sold before anything else.
| Approach | What it does | Characteristic failure |
|---|---|---|
| Keyword and rule matching | Filters on terms, titles, qualifications, dates | Rejects capable candidates who use different vocabulary; rewards keyword stuffing |
| Semantic matching | Embeddings to match meaning rather than exact words | Fluent-sounding but irrelevant experience scores well; opaque about why |
| Trained ranking model | Learns from historical hiring or performance outcomes | Reproduces the historical pattern, including its bias, at scale and speed |
The third is where the regulatory and ethical weight sits. It is also the one most often described simply as "AI-powered".
The label problem, which no amount of model quality fixes
A ranking model needs something to predict. In hiring, the available labels are all compromised.
- "Was hired" encodes the decisions of previous recruiters, biases included.
- "Passed probation" is closer to real but is influenced by manager, team and onboarding as much as by the candidate.
- "Performance rating" inherits the biases of performance review, which are well documented.
- "Still employed after two years" conflates capability with life circumstances and with how well the employer treats people.
None of these is "would be good at the job". A model can only learn the label it is given, so ask what the label was. If the vendor cannot say clearly, they either do not know or would rather not discuss it.
What screening genuinely does well
The criticism above is about ranking. Screening has legitimate uses that are worth separating out:
- Hard requirement filters. A required licence, certification or legal right to work is a factual check, and automating it is uncontroversial.
- Deduplication and data extraction. Parsing applications into structured fields removes tedium without making judgements.
- Surfacing rather than rejecting. Bringing candidates a reviewer might have missed to the top of the pile is a different operation from removing candidates from consideration — and carries very different risk.
That last distinction is the most useful design choice available. A system that surfaces keeps the human decision intact. A system that rejects makes the decision and hides it.
How to evaluate one honestly
- Run it on a past requisition where you know the outcome, and check whether it would have surfaced the person you actually hired — and whom it would have dropped.
- Ask what happens to borderline candidates. Where the cut sits, and who set it.
- Demand a reason per candidate, in language a reviewer can act on. "Score 0.72" is not a reason.
- Test with deliberately varied CVs — same substance, different phrasing, different career shapes including gaps — and see what moves.
- Confirm the audit trail: what is logged, retained and exportable, since you may need to explain a decision much later.
The compliance layer sits on top of all of this
In the EU, employment AI is high-risk under the AI Act, which brings obligations for both the vendor and the employer deploying it — human oversight, input-data appropriateness, logging and transparency to affected people. Those obligations apply regardless of which of the three approaches above you bought.
This page is not legal advice. See the companion piece on the EU AI Act and hiring systems for the split of duties, and take advice on your specific deployment.
Frequently asked questions
How does AI resume screening work?
Three different ways, often marketed identically. Keyword and rule matching filters on terms and qualifications. Semantic matching uses embeddings to match meaning rather than exact wording. Trained ranking models learn from historical hiring or performance data. The third carries the most risk because it reproduces whatever pattern was in that history.
Does AI screening discriminate?
It can, and removing protected attributes does not prevent it. Proxies such as postcode, university, career gaps and writing style correlate with protected characteristics, and a model trained on past hiring decisions learns the biases in those decisions. Whether a specific system discriminates is an empirical question that requires testing, not an assurance.
What should I ask a resume screening vendor?
What the model was trained to predict — the actual label. What bias testing was run, against which groups and how recently. Whether the output is a score, a ranking or a decision. Whether a reviewer can see an actionable reason per candidate. And what is logged and exportable.
Is it better to surface candidates or to reject them?
Surfacing is materially lower risk. A system that raises candidates a reviewer might have missed leaves the human decision intact; a system that removes candidates makes the decision and makes it invisible. Where you have a choice, choose the first.
Related guides
EU AI Act and hiring systems
The EU AI Act classifies AI used in employment — recruitment, selection, promotion and termination decisions — as high-risk. That classification carries obligations that most vendors' marketing does not mention, and they land on the employer as deployer, not only on the vendor.
AI recruiting software
Recruiting tools differ less in features than in where they sit relative to the decision. Sorting the market by that — sourcing, screening, assessment, workflow — makes the regulatory and practical trade-offs visible before you shortlist.
AI in HR
AI in HR splits cleanly into administrative work, where it is uncontroversial and already useful, and decisions about people, where it is regulated as high-risk in the EU. Most disappointment comes from buying the second while expecting the first.