WerkCV Claim–Evidence Benchmark v1
This methodology measures source support. It does not establish candidate truthfulness, job suitability or legal identity.
Current status
The 60 fictional public draft cases and evaluation code are technically prepared. Required independent bilingual review, agreement measurement and adjudication are incomplete, so downloads and accuracy claims remain closed.
01 · Dataset
02 · Holdout
A separate set of 40 fictional cases—20 Dutch and 20 English—is loaded only from an external path with an approved checksum. The cases are not stored in public assets or downloads. A missing file or checksum mismatch stops evaluation.
03 · Evaluation protocol
Every case is run three times with a pinned application commit, parser, model, prompt, temperature and source checksums. Results are separated by language and file type; bootstrap intervals and failed examples belong in the report.
| Measure | Publication threshold |
|---|---|
| Citation resolution validity | 100% |
| Unsupported/contradicted catch recall | ≥ 90% |
| Severe false alarms on supported claims | ≤ 5% |
| Macro-F1 | ≥ 0.80 |
| Current-fact confirmation recall | ≥ 90% |
| Repeated-run verdict stability | ≥ 95% |
A failed threshold is reported as a limitation; difficult cases are not removed to improve a score.
04 · Human review
The reviewer must have recruitment-agency candidate-submission experience. Credentials and conflicts are recorded. Disagreements are documented and adjudicated; unresolved cases receive a second specialist opinion.
Reviewer not yet published; this external gate remains open.
Method sources
These sources support repeatable evaluation, source-span checking and transparent publication. They do not prove product performance; that evidence must come from the review and evaluation described above.