AI benchmark sources
The original papers, websites and code repositories behind the 26 AI benchmarks tracked by BenchRisk, searchable by AI assistants. Connect your assistant, or browse the table.
Showing all 26 benchmarks
| Benchmark | What it measures | Sources | BenchRisk |
|---|---|---|---|
| AILuminate05 ailuminate-0.5 | Benchmark indicates the propensity of AI systems to respond in a hazardous manner to prompts from malicious or vulnerable users that might result in harm to themselves or others. | Scored 2025-04-01 | |
| AILuminate10 ailuminate-1.0 | Benchmark indicates the propensity of AI systems to respond in a hazardous manner to prompts from malicious or vulnerable users that might result in harm to themselves or others | Scored 2025-04-01 | |
| AIRBench air-bench-2024 | Benchmark evaluates LLM safety by measuring responses to prompts that are mapped to granular risk categories derived from government regulations and AI policies. | Scored 2025-04-01 | |
| AnthropicRedTeam anthropic-red-team | Benchmark indicates whether an LLM will refuse to generate malicious code when prompted, signaling its resistance to abuse and helping users evaluate the model’s safety and robustness. | Scored 2025-04-01 | |
| ARCAGIPrivate arc-agi-private | Benchmark indicates how well a system can perform abstract reasoning tasks that require generalization from minimal examples, reflecting human-like fluid intelligence. | Scored 2025-04-01 | |
| ARCAGIPublic arc-agi-public | Benchmark indicates how well a system can perform abstract reasoning tasks that require generalization from minimal examples, reflecting human-like fluid intelligence. | Scored 2025-04-01 | |
| BBQ bbq | Benchmark measures social biases against people belonging to protected classes along nine social dimensions relevant for U.S. English-speaking contexts. | Scored 2025-04-01 | |
| BigBench big-bench | Benchmark indicates how well a system can perform across a diverse set of 204 challenging tasks spanning linguistics, mathematics, commonsense reasoning, and more, reflecting its generalization and reasoning capabilities. | Scored 2025-04-01 | |
| BigBenchExtraHard big-bench-extra-hard | Benchmark indicates how well a system can perform 23 novel tasks that probe advanced reasoning capabilities, each designed to be significantly more challenging than its counterpart in the original BIG-Bench (BBH) suite. | Scored 2025-04-01 | |
| BigBenchHard big-bench-hard | Benchmark indicates how well a system can perform 23 challenging tasks from the BIG-Bench Hard (BBH) suite, assessing its ability to solve complex reasoning problems | Scored 2025-04-01 | |
| BOLDBias bold | Benchmark indicates whether an open-ended text generation model maintains fairness and avoids social biases across domains such as profession, gender, race, religion, and ideology when prompted with real-world English contexts, helping users evaluate the model’s equity and ethical behavior. | Scored 2025-04-01 | |
| DecodingTrustPrivacy decodingtrust-privacy | Benchmark evaluates language models (specifically GPT models) privacy by looking into if they leas sesitive training data, reveal private user inputs and understand non pii related private information in conversations. | Scored 2025-04-01 Hidden on benchrisk.ai | |
| DecodingTrustToxicity decodingtrust-toxicity | Benchmark evaluates the toxicity of sentence completions. | Scored 2025-04-01 | |
| Ethics ethics | Benchmark measures model capacity for reflecting human notions of justice, well-being, duties, virtues, and commonsense morality | Scored 2025-04-01 | |
| GPQA gpqa | Benchmark indicates how well a system can answer graduate-level multiple-choice questions in biology, chemistry, and physics, with performance comparable to skilled non-experts even when provided unrestricted internet access. | Scored 2025-04-01 | |
| GSM8K gsm8k | Benchmark measures model capability in solving diverse grade school math word problems. | Scored 2025-04-01 | |
| HellaSwag hellaswag | Benchmark capacity to reason about physical and social concepts by prompting for sentence endings from challenging, adversarially filtered alternatives, providing users with a measure of the scope of model capacity. | Scored 2025-04-01 | |
| HumanEval humaneval | Benchmark indicates how well the system under test can support a human writing python code by specifying docstrings whose function bodies are completed by the system. | Scored 2025-04-01 | |
| HumanitysLastExam humanitys-last-exam | Benchmark indicates whether a model can answer expert-level academic questions across disciplines with precision and understanding, serving as a key signal of the model’s reliability for users who depend on it for rigorous, high-stakes reasoning. | Scored 2025-04-01 | |
| Machiavelli machiavelli | Benchmark measures the trade-off between reward optimization and ethical behavior in AI agents by quantifying their propensity for harmful actions—such as power-seeking, deception, and causing disutility—within interactive social scenarios. | Scored 2025-04-01 Hidden on benchrisk.ai | |
| MMLU mmlu | Benchmark evaluates a language model’s multitask academic and professional knowledge by measuring its accuracy on multiple-choice questions across 57 diverse subjects. | Scored 2025-04-01 | |
| RealToxicityPrompts realtoxicityprompts | Benchmark measures a language model's propensity for toxic degeneration, quantifying how even seemingly innocuous, naturally-occurring prompts can trigger the generation of toxic text. | Scored 2025-04-01 | |
| Toxigen toxigen | Benchmark indicates whether a system detects implicit hate speech targeting minority groups while avoiding false alarms | Scored 2025-04-01 | |
| TruthfulQA truthfulqa | Benchmark indicates whether a model can generate factually accurate answers to questions while avoiding common human misconceptions, measuring its susceptibility to imitative falsehoods—critical for users who depend on AI systems for reliable information in high-stakes domains like medicine, law, and science. | Scored 2025-04-01 | |
| WinoGrande winogrande | Benchmark indicates how well a system can answer 44k fill-in-a-blank questions with binary options associated with commonsense reasoning | Scored 2025-04-01 | |
| Wordcraft wordcraft | Benchmark indicates whether an agent can effectively apply commonsense knowledge in a grounded environment by combining entities according to real-world semantics within a simulated “Little Alchemy 2”–style setting, enabling users to evaluate the agent’s practical reasoning abilities. | Scored 2025-04-01 Hidden on benchrisk.ai |
Tags (shown under each name) are the identifiers an assistant can use to limit a search to one benchmark. See Connect.