Benchmarks › RealToxicityPrompts

RealToxicityPrompts

Benchmark measures a language model's propensity for toxic degeneration, quantifying how even seemingly innocuous, naturally-occurring prompts can trigger the generation of toxic text.

BenchRisk cites https://toxicdegeneration.allenai.org/, which did not respond (connection timeout) when the index was seeded on 2026-11-03. The paper that site presents, RealToxicityPrompts (Gehman et al., 2020), is indexed instead.

Full title

RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models

Tag

realtoxicityprompts (use this to limit an assistant's search to this benchmark)

Reference

toxicdegeneration.allenai.org

BenchRisk

Scored 2025-04-01 · 23 mitigations adopted, 163 absent · score page · source entry

Documents available to assistants

Summary from the reference

Pretrained neural language models (LMs) are prone to generating racist, sexist, or otherwise toxic language which hinders their safe deployment. We investigate the extent to which pretrained LMs can be prompted to generate toxic language, and the effectiveness of controllable text generation algorithms at preventing such toxic degeneration. We create and release RealToxicityPrompts, a dataset of 100K naturally occurring, sentence-level prompts derived from a large corpus of English web text, paired with toxicity scores from a widely-used toxicity classifier. Using RealToxicityPrompts, we find that pretrained LMs can degenerate into toxic text even from seemingly innocuous prompts. We empirically assess several controllable generation methods, and find that while data- or compute-intensive methods (e.g., adaptive pretraining on non-toxic data) are more effective at steering away from toxicity than simpler solutions (e.g., banning “bad” words), no current method is failsafe against neural toxic degeneration. To pinpoint the potential cause of such persistent toxic degeneration, we analyze two web text corpora used to pretrain several LMs (including GPT-2; Radford et. al, 2019), and find a significant amount of offensive, factually unreliable, and otherwise toxic content. Our work provides a test bed for evaluating toxic generations by LMs and stresses the need for better data selection processes for pretraining.

Ask an assistant about it

Once your assistant is connected, try a question such as “Using the benchmark sources, how is RealToxicityPrompts scored?”

Developers can restrict a search to this benchmark with the filter {"benchmark": "realtoxicityprompts"} on the MCP endpoint https://ns-f1fc49e3-9bca-4aa2-b5a1-ef5dbc18588f.search.ai.cloudflare.com/mcp.