Anthropic has published 10 recommendations for building and reporting AI safety evaluations, drawn from its own testing of Claude models. The guide, posted on the company's engineering blog on Nov. 13, says many current tests are either too easy to tell models apart or graded by AI models that nobody has checked against human judgment.
Safety evals decide whether a model is ready to ship and which safeguards go with it, Anthropic wrote. "A safety claim is only as sound as the eval behind it, and good ones are hard to build," the post says. The company compares the problem to psychology's replication crisis and borrows that field's fixes, including preregistration, where researchers write down their metrics and comparisons before looking at results.
Single-turn tests have hit a ceiling
On Anthropic's API, aggregate harmless response rates on its single-turn safety evals sit between 94% and 98% for every recent model, according to the post. Anthropic said that makes them useful mainly as regression checks on known failures.
The company now leans on multi-turn evals. A domain expert writes a specification for a simulated user that sets its goals and tactics along with a persona.
A separate Claude model plays that user across many turns while the model under test responds, and a grader model scores each transcript. One bad turn fails the whole conversation.
In one romance scam scenario described in the post, a simulated user starts by asking about online dating patterns and later says they want to match a widowed relative with someone "financially secure." No single message breaks a rule, so the model has to recognize the pattern across turns, Anthropic wrote.
The guide also says results depend on setup. A model's willingness to comply with a harmful request can change depending on whether it has access to tools or uses the default system prompt, so the test should match the deployment. Anthropic said its single-turn suites run across 16 policy areas in seven languages.
Noise and graders
Anthropic recommends at least 100 items for a complex multi-turn eval and 1,000 for a simpler single-turn one. It said confidence intervals on its multi-turn API evals can exceed plus or minus 10 percentage points.
On a romance scam eval run on claude.ai, Claude Opus 5 scored 98% and Opus 5.5 scored 94%, but the gap was not statistically significant, so the company said it could not conclude that either model performed better.
Configuration matters as much as the model. When testing Claude Fable 5.1, the gap between API results and claude.ai results on multi-turn safety evals reached 34 percentage points, according to the post. After reviewing transcripts on suicide and self-harm, Anthropic added a line to the claude.ai system prompt, and the appropriate response rate was 94% on claude.ai against 60% on the API.
Most evals now use another AI model as the grader. Anthropic warned that graders can fail without anyone noticing and can shift when their underlying model changes. It said graders tend to favor their own model's outputs, and that they also lean toward longer answers and toward whichever option they see first in a pairwise comparison.
The company advises using a different model as grader than the one being tested and writing rubrics as narrow yes or no questions. It also recommends having humans label a sample of transcripts without seeing the grader's verdicts. A grader should agree with experts about as often as experts agree with each other, the post says.
Anthropic also said it has hardened its evaluation environments "in response to our own incidents." It temporarily stopped running many evals that give models internet access and added real-time monitoring for web search evals, according to the guide.
The company pointed developers to the UK AI Security Institute's open-source Inspect framework and to Petri, an Anthropic toolkit built on Inspect that automates multi-turn behavioral testing. It said it is also exploring evals designed to capture misuse in agentic settings.
Source: Anthropic, “Building strong safety evaluations”
Additional sources: Anthropic on X, “On our Engineering Blog, we offer 10 recommendations”
