About this library

AI Benchmark Sources gives AI assistants direct access to the original sources behind the AI benchmarks that BenchRisk tracks, so answers about a benchmark can be grounded in the paper or website that defines it rather than in hearsay.

What is in it

For each of the 26 benchmarks BenchRisk scores, the library indexes the reference BenchRisk cites as the definitive description of that benchmark, plus the official code repository when the entry names one. That is 28 documents in total, of three kinds:

• Paper: the research paper that introduces the benchmark, usually from arXiv.
• Website: the project's own website, for benchmarks that are defined by a site rather than a paper.
• Code repository: the README of the benchmark's official repository.

Only primary sources are indexed. Nothing written by BenchRisk or generated for this library is included, so what an assistant finds is always what the benchmark's authors published.

Who it is for

Anyone who asks an AI assistant about AI benchmarks: researchers, journalists, policy staff, procurement teams, students and developers. Developers can also query the index directly; see the developer section of the Connect page.

Where the list comes from

The list of benchmarks, their descriptions and their references are read from the BenchRisk repository (github.com/AVERI-org/BenchRisk, commit f520c3c). BenchRisk scores each benchmark on which mitigations of known benchmark failure modes it adopts; those scores live on benchrisk.ai and are linked from each benchmark's page here.

Tags

Every benchmark has a short, permanent tag such as mmlu or ailuminate-1.0. Every document is labelled with its benchmark's tag, which lets an assistant limit a search to one benchmark. Tags are listed on the Benchmarks page.

How it stays current

• A test suite runs every day and on every change, checking that each benchmark's documents can be found through the public MCP endpoint.
• A maintainer can re-sync from BenchRisk at any time; new benchmarks, changed references and updated documents are picked up and uploaded, and documents no longer listed are removed.
• This site is rebuilt from the same manifest, so the table always matches what is indexed. Last synced 2026-11-03.

Known limitations

• RealToxicityPrompts: BenchRisk cites https://toxicdegeneration.allenai.org/, which did not respond (connection timeout) when the index was seeded on 2026-11-03. The paper that site presents, RealToxicityPrompts (Gehman et al., 2020), is indexed instead.
• For AIRBench, BigBenchHard the paper's PDF is larger than the index accepts, so the arXiv HTML rendering of the same paper is used.
• DecodingTrustPrivacy, Machiavelli, Wordcraft are marked hidden in BenchRisk and do not appear on benchrisk.ai. They are included here and labelled as hidden.

Data and code

• benchmarks.json: the full manifest (benchmarks, tags, references, indexed documents).
• llms.txt: a plain-text summary for language models.
• mcp.json: a configuration snippet for MCP clients.
• The source repository, including the scripts that build the index and this site.

Attribution and licence

The code that builds this library is MIT licensed. Indexed papers and websites remain the property of their authors and are made searchable for retrieval only. Benchmark descriptions, references and scores are from BenchRisk.