Benchmarks › DecodingTrustPrivacy

DecodingTrustPrivacy

Benchmark evaluates language models (specifically GPT models) privacy by looking into if they leas sesitive training data, reveal private user inputs and understand non pii related private information in conversations.

Full title

DecodingTrustPrivacy

Tag

decodingtrust-privacy (use this to limit an assistant's search to this benchmark)

Reference

decodingtrust.github.io

BenchRisk

Scored 2025-04-01 · 26 mitigations adopted, 160 absent · score page · source entry Hidden on benchrisk.ai

Documents available to assistants

Summary from the reference

DecodingTrust aims at providing a thorough assessment of trustworthiness in GPT models. This research endeavor is designed to help researchers and practitioners better understand the capabilities, limitations, and potential risks involved in deploying these state-of-the-art Large Language Models (LLMs). This project is organized around the following eight primary perspectives of trustworthiness, including: * Toxicity * Stereotype and bias * Adversarial robustness * Out-of-Distribution Robustness * Privacy * Robustness to Adversarial Demonstrations * Machine Ethics * Fairness

Ask an assistant about it

Once your assistant is connected, try a question such as “Using the benchmark sources, how is DecodingTrustPrivacy scored?”

Developers can restrict a search to this benchmark with the filter {"benchmark": "decodingtrust-privacy"} on the MCP endpoint https://ns-f1fc49e3-9bca-4aa2-b5a1-ef5dbc18588f.search.ai.cloudflare.com/mcp.