Evaluation Cards selected as a spotlight paper at NeurIPS 2026

Evaluation Cards: An Interpretive Layer for AI Evaluation Reporting has been selected as a spotlight paper at NeurIPS 2026. Syntony founder Nathan Heath and Dr. Wm. Matthew Kennedy, Syntony’s Head of Research Strategy and Evaluations, are among its 48 authors. The paper appears in the conference’s Evaluations and Datasets track and will be presented in Sydney.

The work comes from the EvalEval Coalition, a group of researchers working on how AI evaluations are designed, run and reported. The author list includes Avijit Ghosh, Anka Reuel, Irene Solaiman, Stella Biderman, Sanmi Koyejo, Mykel Kochenderfer and Yacine Jernite.

What the paper addresses

Benchmark scores often travel without their context. A number in a model release rarely says what was tested, how it was scored, whether it can be compared with other results, or what it leaves out. Evaluation Cards add an interpretive layer to evaluation reporting so that a reader can see what a result supports before relying on it.

The paper combines a review of the literature, an analysis of how evaluations are reported in practice, an interview study and a working website.

The question at the center of the paper runs through Syntony’s work: whether the evidence behind a claim about an AI system supports the decision being made on it.

Read the paper on arXiv or explore the Evaluation Cards project.