Candidate Evaluation Software: A Buyer's Checklist

Candidate evaluation software is a crowded category. The line between a scorecard tool, an assessment platform, an AI screener, and an intelligence layer is blurry, and vendors do not work hard to clarify it. This is a short checklist of the questions that matter, ordered roughly by how often the answer surprises buyers.

The twelve questions

1. Where does the evaluation criteria come from?

Best answer: the hiring manager defines a structured rubric per role, and the system grounds every score in it. Avoid systems that rely on a generic competency library you cannot edit.

2. Where does the score show up?

Best answer: inside the ATS, as a sortable field, alongside the rationale. Avoid systems that require recruiters to open a separate dashboard during their daily work.

3. How fast does a score arrive after submission?

Best answer: minutes. The first 24 hours are when good candidates accept other offers.

4. Is the rationale auditable?

Best answer: yes, with evidence quotes from the resume tied to each rubric line. Anything else is unfalsifiable.

5. Who can change the rubric?

Best answer: the hiring manager, in minutes, without filing a ticket. If rubric edits require vendor involvement, the system will drift from how the team actually evaluates people.

6. How does it handle bias?

Best answer: explicit suppression of demographic signals, optional blind review modes, and a bias audit report available on request.

7. Does it learn from outcomes?

Careful here. Models that learn from past hires can amplify past bias. Best answer: rubric-grounded scoring, with outcome data used for monitoring, not retraining.

8. What does it cost per evaluated application?

Some vendors price per seat, some per role, some per evaluation. For high-volume hiring, per-evaluation pricing can explode. Model it before signing.

9. What is the integration lift?

Best answer: under a day with the major ATS platforms. If a vendor needs weeks of implementation, the model under the hood is probably weaker than the pitch suggests.

10. Does the audit log survive a legal review?

EU AI Act and NYC Local Law 144 both expect an immutable record of decisions and inputs. Ask to see the log format before you buy.

11. What happens to candidate data?

Best answer: data is not used to train shared models, is retained only as long as needed, and is deleted on request. Confirm in writing, not in marketing copy.

12. What is the exit plan?

Best answer: scores and rationale exportable in a structured format, the ATS write-back left intact, and no proprietary lock-in on the rubric you wrote.

The trade-offs that quietly cost teams a hire

Two patterns show up repeatedly in postmortems. First, a tool that ranks well but lives in its own dashboard so recruiters forget to check it. Second, a tool that reads only the resume and ignores application form answers, missing the candidates whose strongest signal is in the cover note. Both are fixable. Both are worth checking before procurement.