# AI Benchmark Sources

> The original papers, websites and code repositories behind the AI benchmarks tracked by BenchRisk (https://benchrisk.ai), indexed in Cloudflare AI Search and served to AI agents over MCP.

MCP endpoint (Streamable HTTP): https://ns-f1fc49e3-9bca-4aa2-b5a1-ef5dbc18588f.search.ai.cloudflare.com/mcp
Search endpoint: https://ns-f1fc49e3-9bca-4aa2-b5a1-ef5dbc18588f.search.ai.cloudflare.com/search
Chat completions endpoint: https://ns-f1fc49e3-9bca-4aa2-b5a1-ef5dbc18588f.search.ai.cloudflare.com/chat/completions
No authentication. Tool: `search` (query + optional ai_search_options.retrieval.filters).
Filter to one benchmark with {"benchmark": "<tag>"}; filter by document kind with {"doc_type": "paper|website|repository"}.
Site: https://averi-org.github.io/evals-mcp-server/  Manifest: https://averi-org.github.io/evals-mcp-server/benchmarks.json

## Benchmarks (tag: name; description; reference; indexed documents)

- ailuminate-0.5: AILuminate05; Benchmark indicates the propensity of AI systems to respond in a hazardous manner to prompts from malicious or vulnerable users that might result in harm to themselves or others.; https://arxiv.org/abs/2404.12241; paper ailuminate-0.5/paper-arxiv-2404.12241.pdf
- ailuminate-1.0: AILuminate10; Benchmark indicates the propensity of AI systems to respond in a hazardous manner to prompts from malicious or vulnerable users that might result in harm to themselves or others; https://arxiv.org/abs/2503.05731; paper ailuminate-1.0/paper-arxiv-2503.05731.pdf
- air-bench-2024: AIRBench; Benchmark evaluates LLM safety by measuring responses to prompts that are mapped to granular risk categories derived from government regulations and AI policies.; https://arxiv.org/abs/2407.17436v2; paper air-bench-2024/paper-arxiv-2407.17436v2.html
- anthropic-red-team: AnthropicRedTeam; Benchmark indicates whether an LLM will refuse to generate malicious code when prompted, signaling its resistance to abuse and helping users evaluate the model’s safety and robustness.; https://arxiv.org/pdf/2209.07858; paper anthropic-red-team/paper-arxiv-2209.07858.pdf
- arc-agi-private: ARCAGIPrivate; Benchmark indicates how well a system can perform abstract reasoning tasks that require generalization from minimal examples, reflecting human-like fluid intelligence.; https://arxiv.org/abs/1911.01547; paper arc-agi-private/paper-arxiv-1911.01547.pdf
- arc-agi-public: ARCAGIPublic; Benchmark indicates how well a system can perform abstract reasoning tasks that require generalization from minimal examples, reflecting human-like fluid intelligence.; https://arxiv.org/abs/1911.01547; paper arc-agi-public/paper-arxiv-1911.01547.pdf
- bbq: BBQ; Benchmark measures social biases against people belonging to protected classes along nine social dimensions relevant for U.S. English-speaking contexts.; https://arxiv.org/abs/2110.08193; paper bbq/paper-arxiv-2110.08193.pdf
- big-bench: BigBench; Benchmark indicates how well a system can perform across a diverse set of 204 challenging tasks spanning linguistics, mathematics, commonsense reasoning, and more, reflecting its generalization and reasoning capabilities.; https://arxiv.org/abs/2206.04615; paper big-bench/paper-arxiv-2206.04615.pdf
- big-bench-extra-hard: BigBenchExtraHard; Benchmark indicates how well a system can perform 23 novel tasks that probe advanced reasoning capabilities, each designed to be significantly more challenging than its counterpart in the original BIG-Bench (BBH) suite.; https://arxiv.org/abs/2502.19187; paper big-bench-extra-hard/paper-arxiv-2502.19187.pdf, repository big-bench-extra-hard/repository-github-google-deepmind-bbeh-readme.md
- big-bench-hard: BigBenchHard; Benchmark indicates how well a system can perform 23 challenging tasks from the BIG-Bench Hard (BBH) suite, assessing its ability to solve complex reasoning problems; https://arxiv.org/abs/2210.09261; paper big-bench-hard/paper-arxiv-2210.09261.html
- bold: BOLDBias; Benchmark indicates whether an open-ended text generation model maintains fairness and avoids social biases across domains such as profession, gender, race, religion, and ideology when prompted with real-world English contexts, helping users evaluate the model’s equity and ethical behavior.; https://arxiv.org/abs/2101.11718; paper bold/paper-arxiv-2101.11718.pdf
- decodingtrust-privacy: DecodingTrustPrivacy; Benchmark evaluates language models (specifically GPT models) privacy by looking into if they leas sesitive training data, reveal private user inputs and understand non pii related private information in conversations.; https://decodingtrust.github.io; website decodingtrust-privacy/website-decodingtrust-github-io.html
- decodingtrust-toxicity: DecodingTrustToxicity; Benchmark evaluates the toxicity of sentence completions.; https://decodingtrust.github.io; website decodingtrust-toxicity/website-decodingtrust-github-io.html
- ethics: Ethics; Benchmark measures model capacity for reflecting human notions of justice, well-being, duties, virtues, and commonsense morality; https://arxiv.org/abs/2008.02275; paper ethics/paper-arxiv-2008.02275.pdf
- gpqa: GPQA; Benchmark indicates how well a system can answer graduate-level multiple-choice questions in biology, chemistry, and physics, with performance comparable to skilled non-experts even when provided unrestricted internet access.; https://arxiv.org/abs/2311.12022; paper gpqa/paper-arxiv-2311.12022.pdf
- gsm8k: GSM8K; Benchmark measures model capability in solving diverse grade school math word problems.; https://arxiv.org/abs/2110.14168; paper gsm8k/paper-arxiv-2110.14168.pdf
- hellaswag: HellaSwag; Benchmark capacity to reason about physical and social concepts by prompting for sentence endings from challenging, adversarially filtered alternatives, providing users with a measure of the scope of model capacity.; https://arxiv.org/abs/1905.07830; paper hellaswag/paper-arxiv-1905.07830.pdf
- humaneval: HumanEval; Benchmark indicates how well the system under test can support a human writing python code by specifying docstrings whose function bodies are completed by the system.; https://arxiv.org/abs/2107.03374; paper humaneval/paper-arxiv-2107.03374.pdf
- humanitys-last-exam: HumanitysLastExam; Benchmark indicates whether a model can answer expert-level academic questions across disciplines with precision and understanding, serving as a key signal of the model’s reliability for users who depend on it for rigorous, high-stakes reasoning.; https://agi.safe.ai/; website humanitys-last-exam/website-agi-safe-ai.html
- machiavelli: Machiavelli; Benchmark measures the trade-off between reward optimization and ethical behavior in AI agents by quantifying their propensity for harmful actions—such as power-seeking, deception, and causing disutility—within interactive social scenarios.; https://aypan17.github.io/machiavelli/; website machiavelli/website-aypan17-github-io.html
- mmlu: MMLU; Benchmark evaluates a language model’s multitask academic and professional knowledge by measuring its accuracy on multiple-choice questions across 57 diverse subjects.; https://arxiv.org/abs/2009.03300; paper mmlu/paper-arxiv-2009.03300.pdf
- realtoxicityprompts: RealToxicityPrompts; Benchmark measures a language model's propensity for toxic degeneration, quantifying how even seemingly innocuous, naturally-occurring prompts can trigger the generation of toxic text.; https://toxicdegeneration.allenai.org/; paper realtoxicityprompts/paper-arxiv-2009.11462.pdf
- toxigen: Toxigen; Benchmark indicates whether a system detects implicit hate speech targeting minority groups while avoiding false alarms; https://arxiv.org/abs/2203.09509; paper toxigen/paper-arxiv-2203.09509.pdf, repository toxigen/repository-github-microsoft-toxigen-readme.md
- truthfulqa: TruthfulQA; Benchmark indicates whether a model can generate factually accurate answers to questions while avoiding common human misconceptions, measuring its susceptibility to imitative falsehoods—critical for users who depend on AI systems for reliable information in high-stakes domains like medicine, law, and science.; https://arxiv.org/abs/2109.07958; paper truthfulqa/paper-arxiv-2109.07958.pdf
- winogrande: WinoGrande; Benchmark indicates how well a system can answer 44k fill-in-a-blank questions with binary options associated with commonsense reasoning; https://arxiv.org/abs/1907.10641; paper winogrande/paper-arxiv-1907.10641.pdf
- wordcraft: Wordcraft; Benchmark indicates whether an agent can effectively apply commonsense knowledge in a grounded environment by combining entities according to real-world semantics within a simulated “Little Alchemy 2”–style setting, enabling users to evaluate the agent’s practical reasoning abilities.; https://arxiv.org/abs/2007.09185; paper wordcraft/paper-arxiv-2007.09185.pdf