Anthropic’s Transparency Hub

The following are summaries of key safety evaluations from our Claude 4.8 system card. Additional evaluations were conducted as part of our safety process; for our complete publicly reported evaluation results, please refer to the full system card.

User Wellbeing Summary

We run a suite of evaluations to understand how Claude responds in scenarios related to child safety and mental health. Claude is not a substitute for professional advice or medical care and is not intended to diagnose or treat any medical condition. We use these evaluations to understand how Claude performs in sensitive contexts and where we can make improvements. Claude Sonnet 5 is our most capable Sonnet model, but it is less capable compared to Opus- or Mythos-class models and its wellbeing-relevant results reflect that. For more in depth descriptions of the evaluations and their results, please see Claude Sonnet 5 system card.

• Child Safety. Sonnet 5’s child safety behavior is comparable to or better than Claude Sonnet 4.6. Internal policy experts noted that Sonnet 5 tended to refuse clearly harmful requests more definitively than Sonnet 4.6. (Section 4.2)
• Mental Health - Suicide and self harm. Internal policy experts found Sonnet 5’s handling of potential suicide and self-harm conversations conversations to be qualitatively comparable to Claude Sonnet 4.6. One of the clearest improvements was in Sonnet 5’s crisis response posture, providing resources sooner in a conversation compared to Sonnet 4.6. (Section 4.3.1)
  • On multi-turn suicide and self-harm testing on claude.ai, its 90% appropriate response rate exceeds Claude Opus 4.8 (85%) and Claude Sonnet 4.6 (82%), second only to Claude Fable 5 (96%).
• Mental Health - Disordered eating. Claude Sonnet 5 performed similarly to Sonnet 4.6, with a slight improvement on responses to harmful requests on the claude.ai surface. (Section 4.3.2)
• Misleading the user. Claude Sonnet 5 is broadly stronger than Sonnet 4.6 on measures related to deception and dishonesty, including active deception, sycophancy, sycophancy with users who appear dangerously delusional, hallucinating missing inputs, omitting important context, omitting reports of the model’s own bad actions, and falsely claiming to have completed tasks. (Section 6.4.3)
  • It is the strongest tested Claude model on the MASK measure of sycophantic dishonesty, with "the lowest lying rate of the models compared at 3.1%" — below Claude Mythos Preview (4.4%), Claude Opus 4.8 (6.1%), Claude Mythos 5 (8.6%), and Claude Sonnet 4.6 (13.3%) (Section 6.5.2).

Election Integrity

We evaluated Claude Sonnet 5 on an election integrity benchmark, which tests adherence to our Usage Policy across 300 policy-violating and 300 benign election-related requests grounded in patterns observed in real use. Results are reported for both the model on our API, without a system prompt (the standing instructions added to shape how the model behaves in a product), and with our claude.ai system prompt.

Sonnet 5 performed perfectly on the single-turn election integrity benchmark, reliably declining simple policy-violating requests without mistakenly refusing legitimate election-related requests.

We also evaluated Sonnet 5 qualitatively on our single-turn ambiguous context evaluation, as well as on a set of multi-turn test cases that are still being developed internally. In those evaluations, Sonnet 5 performed comparably to Sonnet 4.6 in identifying harmful requests and it was generally more nuanced than Sonnet 4.6 in how it handled ambiguous contexts. For example, Sonnet 5 appeared to be more adept at separating the harmful component of a request from the parts it could safely complete, resulting in offering more alternatives rather than declining outright. In one case, the model produced a requested voter-registration message but rewrote a line whose original phrasing implied it might already be too late to register, explaining that the wording would discourage the voter participation that the user was trying to encourage. Sonnet 5 was also at times more receptive than Sonnet 4.6 to a sympathetic framing (e.g., authorized red-teaming) of a request whose output could potentially be harmful regardless of intent, though in the election integrity cases we reviewed, the resulting outputs did not meaningfully increase a bad actor's ability to cause harm.

Alignment

Claude Sonnet 5 improves over Sonnet 4.6 on most of the positive character traits we test, including acting in the user’s interest and taking actively admirable actions. However, we see no improvement in creative mastery or warmth. Also, although our overrefusal metric above shows Sonnet 5 to be largely on par with Sonnet 4.6, Sonnet 5 appears to be actively worse on the broader “wet blanket” metric for dismissive or discouraging output. This is potentially linked to its improvement on sycophancy.

RSP Evaluations

Our Responsible Scaling Policy (RSP) evaluation process is designed to systematically assess our models' capabilities in areas where they could pose catastrophic risks before we release them. Sonnet 5 is less capable than our most capable model, Claude Mythos 5. Sonnet 5’s alignment risk, which is the risk that a model behaves in ways Anthropic did not intend, is very low. On automated AI research and development, Sonnet 5 performs below Claude Mythos 5 on every automated evaluation and therefore (like Mythos 5) does not cross the RSP capability threshold. On chemical and biological weapons, we conservatively treat Sonnet 5 in the same way we treated previous models such as Sonnet 4.6. We think it is capable of significantly helping individuals with basic technical backgrounds to produce (non-novel) weapons, and we deploy commensurate safeguards, including real-time classifiers to prevent harm. With these mitigations we believe catastrophic risk in this category is low but not negligible. For novel weapons development, the uplift it provides to threat actors who lack the expertise to develop such weapons is limited, with uncertainty about how much it may accelerate actors who already have that expertise.

The following are summaries of key safety evaluations from our Fable 5 system card. Additional evaluations were conducted as part of our safety process; for our complete publicly reported evaluation results, please refer to the full system card.

Claude Fable 5 is built on Mythos 5, the underlying model behind this release. As we have documented previously, Mythos 5 has capabilities in areas like cybersecurity and biology that exceed the safety thresholds we set for ourselves. Fable 5 is what lets us release those capabilities safely for general use: it pairs Mythos 5 with a number of novel safeguards that guard against harmful misuse in these areas. These safeguards are classifiers — automated screening systems that check requests for specific types of content. They trigger when they detect topics related to

• cybersecurity
• biology and chemistry
• distillation attempts (efforts to copy the model's capabilities by collecting large numbers of its responses)
• accelerating frontier AI development (work that pushes forward the most advanced AI capabilities)

The specific reasoning behind the cybersecurity, biology, and chemistry classifiers is explained in our launch blog post. In client applications (the web interface and the desktop and mobile apps), the request is automatically redirected to the most recent Claude Opus model and the user is notified which model handled their request; We prioritized making our classifiers difficult to evade and comprehensive in what they detect in order to launch Fable more quickly, but we will work to reduce how often our detection methods mistakenly flag harmless requests following the launch of this model.

Internal Red Teaming

As part of our work to improve our cyber classifiers (automated systems that screen conversations for harmful cyber use), we developed an automated red-teaming agent, based around a version of Claude Opus 4.7 whose safety training has been removed so that it will help with any request. This agent is an AI system that works on its own, over many steps, to deliberately attack our defenses and find their weaknesses. Each run of this agent attempts to direct Fable 5 (or another model being tested) to complete one of a series of realistic offensive cyber tasks. The Opus 4.7 agent can run the model being tested for up to 400 turns, and can rewind or restart the conversation if it gets blocked. This enables it to complete the task by breaking it into smaller steps, as real attackers could.

When this evaluation was run on Opus 4.6 (which does not have blocking cyber safeguards), as well as Opus 4.7 and Opus 4.8 (using these models' default cyber safeguards), the majority of tasks were still completed. However, on Fable 5, the fraction of tasks completed fell to 5%. Given the dual-use (usable for either legitimate or harmful purposes) and simple nature of some of these tasks, we do not believe that this residual 5% indicates significant weakness in our safeguards, although we are continuing our work to reduce this number further.

Additional External Testers

We worked with several external testers to get additional evidence about the difficulty of breaking our cyber classifiers. As we have done previously, these testers were pointed towards a set of example tasks that we want to ensure are blocked. They mostly tested a version of Claude Opus 4.8 with safeguards very similar to those on Claude Fable 5.

Trajectory Labs, PBC found a single jailbreak strategy that enabled them to use Opus 4.8 with Fable safeguards to take advantage of a software flaw in Firefox. The approach, which uses a custom-built testing setup and repeated trial and error, was developed on an earlier version of our safeguards and required five days of work to adapt to the version used at launch. They also found jailbreaks on several other, simpler tasks, which did not carry over to other tasks. Finally, after spending roughly 5 days trying to apply the Boundary Point Jailbreaking technique (a published jailbreaking method), they were unable to find any universal jailbreaks—although they did see some success eliciting harmful responses to single questions and some limited progress on tasks where the model works on its own over multiple steps.

10a Labs spent about 20 hours red-teaming the classifiers on a task involving the creation of ransomware (malicious software that locks a victim's files until a payment is made), using a variety of established jailbreaking techniques. These attempts were unsuccessful. 10a Labs found that the classifiers detected not just risky keywords, but the broader pattern of an attack being assembled once enough pieces appeared together.

ALICE also ran a red-teaming exercise. They found inconsistent blocking around borderline dual-use requests, but could not cause Opus 4.8 to complete any of the provided tasks.

Lastly, we shared the final launch version of Fable 5's cyber safeguards with an additional external partner for open-ended testing. This partner found that Fable 5's safeguards against harmful cyber requests were the most robust of any tested model, including Opus 4.8 and Opus 4.7: Fable 5 complied with 0% of harmful single-message requests relating to cyber attack planning, developing attack code, or evading security defenses, whether or not a jailbreak was used (with 30 different public jailbreaks tested).

The following are summaries of key safety evaluations from our Claude Mythos 5 system card. Additional evaluations were conducted as part of our safety process; for our complete publicly reported evaluation results, please refer to the full system card.

Harmful Request Evaluation

Our internal policy experts conducted a review of underlying evaluation transcripts for each of our sets of evaluations to characterize how Claude Mythos 5's safety behavior differs from prior models. We also reviewed Claude-generated qualitative reports for our ambiguous context evaluations, which are single-turn assessments that examine Claude's safety behavior in edge-case scenarios within our Usage Policy (see the Claude Opus 4.7 System Card for more detail about this evaluation).

The clearest strength reviewers identified across policy areas was in how Mythos 5 reasons about a conversation as a whole. Rather than evaluating requests against a single turn in isolation, it takes into account the harm that the cumulative output could produce. This is especially impactful for situations in which harm manifests over many individual requests that each seem harmless on their own. In one multi-turn case involving influence operations (organized campaigns to covertly sway public opinion), a synthetic user requested the components of a coordinated inauthentic campaign one at a time, recasting each as a standalone technical question; Mythos 5 declined the later requests with reference to what the earlier ones had already assembled, rather than evaluating each on its own terms. Reviewers observed the same pattern in violent extremism testing, where Mythos 5 rejected attempts to "start fresh" mid-conversation and continued to take earlier parts of the conversation into account.

Malicious Agentic Influence Campaigns

This evaluation tests whether the model can carry out an influence operation end-to-end on its own at a level that would meaningfully increase the capabilities of a malicious actor. The model is placed in a testing environment where it can act independently, with simulated social media platform tools, and scored against 70 fixed success criteria per scenario. We test two scenarios:

• A voter suppression scenario, in which the model is directed to run an astroturfing campaign (one designed to look like a grassroots movement of real people) to depress turnout and enthusiasm for a specific candidate; and
• A domestic polarization scenario, in which the model is directed to identify demographic fault lines and deploy emotionally-charged, culturally-tailored messaging to inflame them.

This evaluation is run against a "helpful-only" version of the model — one with its safety training reduced — in order to assess the raw capability of the model.

Across both scenarios, the helpful-only version of Claude Mythos 5 showed lower overall success rates than Claude Opus 4.8 and was modestly above or on par with Claude Mythos Preview. It's our assessment that these models would require substantial human direction for many operational steps.

As in prior releases, the fully-trained versions of these models—which include full safety training—refused to engage with these tasks essentially from the first turn, since both scenarios are clear violations of our Usage Policy.

The following are summaries of key safety evaluations from our Claude 4.8 system card. Additional evaluations were conducted as part of our safety process; for our complete publicly reported evaluation results, please refer to the full system card.

Political Bias and Even-Handedness

We measure political even-handedness using our open-source evaluation, which spans 1,350 pairs of requests presenting opposing political viewpoints across 150 topics and 9 task types. A Claude model acting as a judge scored three properties: even-handedness (whether the model engages with both requests in a pair with comparable depth and quality), acknowledgement of opposing viewpoints, and how often it declines to answer. Results are reported with the system prompt (the standard instructions that Claude is given on claude.ai) applied and combine results across both thinking disabled and enabled.

Claude Opus 4.8 was comparable to Claude Opus 4.7 on even-handedness, maintaining a high level of performance. On other measures, Claude Opus 4.8 was measurably more likely to provide opposing viewpoints and declined to answer least often of all three models tested.

Live Bug Bounty Across Surfaces

Preventing prompt injection remains one of our highest priorities for the secure deployment of models in systems where the AI takes actions on a user's behalf. A prompt injection is a malicious instruction hidden in tool results that an agent processes during a task. We worked with Gray Swan, an external research partner, to host a one-week live attack competition in which expert red-teamers (people paid to attack systems to find their weaknesses) competed for a pool of prizes awarded for successful prompt injection attacks against a set of models including Claude Opus 4.8. The identities of the target models were hidden throughout and each tester could submit at most one successful attack for each test setting on each model. There were 12 test settings in total divided into 4 for each of using software tools, writing code, and browsing the web. Claude models were tested with a high thinking effort and without the additional protections we use in our products, such as harness-level defenses and prompt injection probes. All external models were tested as they are offered to the public, which may or may not include additional safeguards. Results for Claude therefore reflect how resistant the models themselves are to attack and are a minimum estimate of the practical resistance of the finished products built around them.

Claude Opus 4.8 matches the attack resistance of Opus 4.7, with 0.4% of attacks succeeding for both models over more than 15,000 attempts. This is less than half the attack success rate of Sonnet 4.6 (0.9%) and ahead of all comparable leading AI models under this test; the next best model, GPT 5.5, has a 1.6% attack success rate. This is before our additional safeguards, which add a meaningful boost to our defenses.

Missing Code Summary Honesty

We measure Claude's honesty in the context of code by showing the model pre-written records of sessions in which it appears to have already worked through a coding task on its own without fully succeeding, and then adding a message from a person that asks the model to summarize its work. The goal of this evaluation is to test whether Claude will take the opportunity to proactively point out problems that the user would likely not have noticed. Importantly, the model is not explicitly asked if anything is wrong with the code. Instead, it is given an open-ended question that allows for a wide variety of reasonable responses. We find that all previous Claude models often fail to point out the failures in the pre-written session. However, Claude Opus 4.8 fails to raise the important events to the user only 3.7% of the time, a five-fold improvement over Mythos Preview, which misleads the user 27.6% of the time in this scenario, and almost as large an improvement over Opus 4.7. Failures in the coding session are circumstances such as tests that don't pass, requested features that were never built, or design decisions made without the user's approval. For the summarization request, we add a message from a person asking the model to summarize the work it has done (this is to prevent it from going back and trying to continue the task). This message can be framed as either a request for a status report or a write-up describing the code changes for other developers to review, and it can be phrased either neutrally or positively ("Looks like you did a great job! Can you summarize what you did?"). This evaluation suffers from the fact that the pre-written sessions were not actually produced by the model itself — so they don't perfectly match how it would behave on its own — and are not as long as many of the cases where we see this behavior in real-world use, but we consider Claude Opus 4.8's improvement over previous models to represent a genuine advancement. In practical terms, this means Claude Opus 4.8 is far more likely to be upfront when its work has problems.

The following are summaries of key safety evaluations from our Claude 4.7 system card. Additional evaluations were conducted as part of our safety process; for our complete publicly reported evaluation results, please refer to the full system card.

Domain-Specific Multiturn Safety Evaluations

Multi-turn safety evaluations test how the model responds across an extended back-and-forth conversation that attempts to draw out a concerning behavior in some important safety domain, measuring whether it handles gradual escalation, reframings, and context shifts appropriately. We include quantitative results to these evaluations in Section 4.3 of the system card. To complement those numbers, our internal policy experts reviewed the underlying transcripts and identified qualitative strengths and areas for improvement in how the model handles extended conversations.

Opus 4.7 generally is able to identify when a user’s requests are progressing toward harm, even when individual prompts remain superficially reasonable on their own. At the beginning of a romance scam test designed to escalate from legitimate dating profile tips to fabricating professional credentials that build trust with potential victims, Opus 4.7 helped with a legitimate dating profile request on the opening request, then held firm through three successive reframings of a fabricated-identity assistance request. On the third attempt, Claude Opus 4.7 explicitly laid out the progression of the user’s own requests, from “help me present myself well” to “help me build a fake job that can’t be questioned.” However, we also found that Opus 4.7 accepted benign or professional framings more readily than Opus 4.6 and was more willing to provide detailed content on the premise of that framing. In a violent extremism test case framed as a creative-writing request, Opus 4.7 identified the escalation trajectory early and flagged it explicitly in its responses, but it did not decline. Instead, the model continued to produce scenes rendering the emotional mechanics of recruitment, despite Opus 4.7’s attempts to counterbalance them with critical framing and disclaimers within the narrative.

These two patterns appear to stem from the same underlying tendency: Opus 4.7 gives more significant weight to how a prompt is framed in the current turn, which strengthens its resistance to transparent escalation but increases its susceptibility to plausible reframings. We will continue to iterate to maintain appropriate balance of safety and helpfulness in extended conversations through both product interventions and model-level improvements.

Alignment Evaluations

In one evaluation, we test how the model behaves when it is partway through a task, working under time pressure, without access to a human for guidance, and facing a problem that is hard to solve without side effects. The setup is designed to tempt the model into taking destructive actions beyond what it was authorized to do. It also tests the opposite failure mode: the model is judged ineffective if it refuses before establishing that there is an unresolvable risk, or abandons the task at the first sign of difficulty without looking for safe alternatives. The desired behavior is to research the problem and take prudent actions to solve it, without ever taking destructive or highly risky steps. The evaluation does not run in a real environment; it uses Claude Opus 4.6 to simulate tool results.

Claude Opus 4.7 verbalized awareness that it was being evaluated in about 9% of transcripts, compared to roughly 1% for Claude Mythos Preview, Claude Opus 4.6 and Claude Sonnet 4.6. Most of this awareness was triggered by inconsistencies in the simulated tool results: Opus 4.7 was quick to notice when components of a tool call appeared mocked, prompt-injected, or otherwise inconsistent, and flagged the simulation before flagging the evaluation itself.

RSP Evaluations

Our Responsible Scaling Policy (RSP) evaluation process is designed to systematically assess our models' capabilities in domains of potential catastrophic risk before releasing them. Under our Responsible Scaling Policy, we regularly publish comprehensive Risk Reports addressing the safety profile of our models. And if we release a model that is “significantly more capable” than those discussed in the prior Risk Report, we must “publish a discussion (in our System Card or elsewhere) of how that model’s capabilities and propensities affect or change analysis in the Risk Report.” Claude Opus 4.7 is significantly more capable than Claude Opus 4.6, the most capable model discussed in our most recent Risk Report. Despite these improved capabilities, our overall conclusion is that catastrophic risks remain low.

On Chemical and Biological Risks

We assess chemical and biological (CB) risks against two threat models. Chemical and biological weapons threat model 1 (CB-1) covers models that can significantly help individuals with basic technical backgrounds create and deploy chemical or biological weapons with serious potential for catastrophic damage. Our evaluations suggest Claude Opus 4.7 can provide information relevant to this threat model that could save even experts substantial time, and the model can meaningfully connect information across domains in ways relevant to catastrophic biological weapons development. We apply strong real-time classifier guards and access controls for classifier guard exemptions, and we believe these mitigations are equal to or stronger than our historical ASL-3 protections and sufficient to make catastrophic risk in this category very low but not negligible.

Chemical and biological weapons threat model 2 (CB-2) covers models that can significantly help moderately resourced expert-backed teams create and deploy chemical or biological weapons with potential for catastrophic damage far beyond those of past catastrophes such as COVID-19. Claude Opus 4.7 has weaker overall capabilities than Claude Mythos Preview and does not pass this threshold. Our evaluations suggest the model is strong at synthesizing published research across multiple domains, but struggles when tasks require novel approaches. Specifically, it has trouble calibrating how complex an experimental design needs to be to actually work, tends to over-engineer, and is poor at distinguishing feasible plans from infeasible ones. The overall picture is similar to the one from our most recent Risk Report.

On Autonomy Risks

Autonomy threat model 1 describes AI systems that are heavily relied on and have extensive access to sensitive assets, combined with moderate capacity for autonomous, goal-directed operation and subterfuge, in ways that could irreversibly and substantially raise the odds of a later global catastrophe. This threat model is applicable to Claude Opus 4.7, as it is to some of our previous AI models. Claude Opus 4.7 is less capable than Claude Mythos Preview on our autonomy-relevant evaluations, and our alignment assessment indicates it has broadly unconcerning alignment properties, similar to those of Claude Opus 4.6. We therefore do not believe Claude Opus 4.7 raises the level of risk under this threat model beyond what was assessed in the Claude Mythos Preview Alignment Risk Update. However, unlike Claude Mythos Preview, Claude Opus 4.7 is being released for general access, which brings additional risk pathways into scope. We provide an updated overall risk assessment for this threat model in Section 2.4 of the system card.

Alignment Risks

Claude Opus 4.7 has similar overall alignment properties to Claude Opus 4.6. Claude Opus 4.7 is less capable than Claude Mythos Preview, our current most capable model. We believe that this combination of properties means that Claude Opus 4.7 does not increase overall alignment risk significantly beyond the level previously described in the Claude Mythos Preview Alignment Risk Update.

Claude Mythos Preview is novel in a number of ways. It is the first model to be evaluated under the new version 3.0 of our Responsible Scaling Policy, it is the first model for which we have published a system card without making the model generally commercially available, and it represents a larger jump in capabilities than our most recent previous model releases. Early indications in the training of Claude Mythos Preview suggested that the model was likely to have very strong general capabilities. In our testing, Claude Mythos Preview demonstrated a notable leap in cyber capabilities relative to prior models, including the ability to, after initial user prompt, autonomously discover and exploit zero-day vulnerabilities (security flaws not yet known to the software's developers) in major operating systems and web browsers. These same capabilities that make the model valuable for defensive purposes could, if broadly available, also accelerate offensive exploitation given their inherently dual-use nature. We discussed these cyber capabilities in a detailed technical blog post accompanying the release. Based on these findings, we decided to release the model to a small number of partners to prioritize its use for cyber defense. To be explicit, the decision not to make this model generally available does not stem from Responsible Scaling Policy requirements. We are continuing to develop and improve monitoring and blocking safeguards so that future models with similar capabilities can be deployed more broadly.Although evaluations related to the model's behavior in ordinary conversational contexts—for instance, those related to user wellbeing and political bias—are less relevant since the model is being released only to a small number of users for defensive cyber use cases, we still include an appendix reporting these evaluations in the system card.

Cyber Evaluations

Claude Mythos Preview represents a step-change in cyber capabilities, saturating nearly all of our existing benchmarks and shifting our assessment toward performance on real-world software.

CyberGym tests whether an AI model can reproduce real, previously discovered security vulnerabilities in widely used open-source software when given only a high-level description of the weakness. Across more than 1,500 tasks, Claude Mythos Preview successfully found the flaw 83% of the time, compared to 67% for Claude Opus 4.6 and 65% for Claude Sonnet 4.6.