Four days ago this blog wrote about Terminal-Bench 3.0 as a benchmark that never got independently verified — a live leaderboard that kept returning different numbers depending on when you looked, superseded by version 4.0 before anyone could pin a citable score to it. As of September 1, that problem is over. Terminal-Bench 4.0’s leaderboard is populated with real, dated, cost-annotated runs, and — on the same day — Anthropic shipped Claude Fable 5.1 and Claude Mythos 5.1. The timing wasn’t coordinated by this blog’s editorial calendar; it’s just how the week landed. It’s still worth unpacking both halves together, because the leaderboard is the closest thing to independent confirmation the industry has for what Anthropic just claimed on its own.
What actually shipped#
Fable 5.1 and Mythos 5.1 are, per Anthropic’s own launch post and system card, the same underlying model shipped at two safeguard tiers — a pattern Anthropic has kept consistent since the original Fable 5/Mythos 5 split in June. Fable 5.1 is generally available through the API (claude-fable-5-1) and across AWS, Google Cloud, and Microsoft Azure. Mythos 5.1 stays gated to vetted organizations through the Cyber Verification Program and Life Sciences Verification Program, US-only.
The concrete changes:
- 1M token context, 128K output tokens — unchanged ceiling from Fable 5, but Anthropic says the model handles more of that window productively on long-horizon coding and research tasks.
- Pricing held flat at $10/$50 per million input/output tokens, but cache-read cost dropped 75%, to $0.25 per million tokens. Anthropic’s own estimate is ~25% lower cost for typical workloads and up to ~45% lower for highly agentic, cache-heavy sessions — which is most of what Claude Code does.
- Mid-conversation effort adjustment. Claude Code’s changelog corroborates this directly: v2.1.257 (also shipped September 1) added
sas a modifier to/effort, letting a session change its effort level without restarting. This is a real, shippable feature, not just marketing copy in the model announcement. - Enterprise Frontier Safeguards (EFS) — a zero-data-retention mode where inference artifacts stay on customer infrastructure, rolling out in phases starting this fall. Not yet live.
- Safeguard tuning: Anthropic claims ~60% fewer false-positive interventions from cybersecurity classifiers and 85% fewer on elementary/medical biology queries versus Fable 5 — the same complaint pattern (over-blocking legitimate technical work) this blog has flagged repeatedly since the original Fable 5 cybersecurity classifier rollout.
- Anti-distillation and provenance controls: new API accounts can no longer manually edit context in ways Anthropic’s system card ties to distillation-attack mitigation (a direct descendant of the Alibaba distillation-accusation story from June), plus EU AI Act–compliant text watermarking with a detection API in private preview.
Claude Code’s same-day v2.1.257 release made Fable 5.1 the new default Fable model and added a few adjacent hardening items worth flagging on their own: a “Containment Escape” rule in auto mode specifically targeting cloud metadata-credential fetches and cross-tenant reach, a CLAUDE_CODE_SUBAGENT_MODEL_FORCE environment variable, and a one-time confirmation prompt before auto mode reads a file outside the working directory for the first time. A same-day follow-up, v2.1.258, fixed a macOS 12 Monterey launch regression and a remote/scheduled-session permission bug — both introduced in 2.1.255, both closed within hours of the flagship release. That’s the same rapid-patch discipline this blog has tracked through the entire v2.1.2xx run since August.
The benchmark numbers Anthropic reported itself#
Anthropic’s own system card lists a spread of scores for Fable 5.1: 55.8% on Terminal-Bench 4.0 (60.9% for Mythos 5.1), 52.6% on Terminal-Bench-Science 0.1, 60.9%/65.0% on Humanity’s Last Exam (without/with tools), 77.9%/41.7% on OSWorld 2.0 (partial/strict credit), 73.4% on CursorBench 3.2.0, and 31.4% on AutomationBench. It also cites concrete scientific-research outputs — protein binder designs reaching roughly 10x higher binding affinities than competition entries with an almost 50% hit rate across 12 targets (versus a typical 10-15%), Venus elevation mapping improved from 10-20km to 2-3km resolution, and GPU kernel optimizations up to 2.5x faster.
All of that is self-reported. None of it is nothing — Anthropic has a real track record of self-reported numbers holding up under independent scrutiny more often than competitors’ have this year — but it’s still one company grading its own homework on launch day. That’s exactly why the Terminal-Bench 4.0 leaderboard populating on the same day matters.
What the independent leaderboard actually shows#
Terminal-Bench 4.0’s public leaderboard, hosted by Stanford, Harbor, and the Laude Institute, now has ten dated, cost-annotated entries:
| Rank | Model | Effort | Agent | Score | Tokens used | Cost |
|---|---|---|---|---|---|---|
| 1 | Opus 5 | max | Claude Code | 51.8% ± 3.4 | 6.5B | $6.0k |
| 2 | Fable 5 | max | Claude Code | 44.5% ± 3.8 | 3.8B | $7.3k |
| 3 | GLM-5.3 | max | Claude Code | 41.8% ± 3.2 | 8.7B | $2.7k |
| 4 | GPT-5.6 Sol | max | Codex | 37.3% ± 3.8 | 4.4B | $2.5k |
| 5 | Opus 4.8 | max | Claude Code | 23.6% ± 3.6 | 6.4B | $6.5k |
| 6 | GPT-5.6 Terra | max | Codex | 21.5% ± 3.3 | 5.7B | $1.7k |
| 7 | Grok 4.6 | high | Grok Build | 20.3% ± 3.1 | 4.0B | $3.6k |
| 8 | GPT-5.6 Luna | max | Codex | 17.3% ± 2.8 | 11.6B | $0.3k |
| 9 | Grok 4.5 | high | Grok Build | 12.4% ± 2.6 | 3.4B | $2.1k |
| 10 | Sonnet 5 | max | Claude Code | 12.4% ± 3.1 | 21.6B | $9.6k |
Claude Opus 5 and Fable 5 occupy the top two spots, a 7-point gap ahead of the nearest non-Claude entrant, GLM-5.3. That’s a genuine result, not a marketing framing of one — this is the harness’s own numbers, not Anthropic’s.
Two honest caveats belong right next to that win. First, neither Fable 5.1 nor Mythos 5.1 appears on this leaderboard yet — the last entries are dated through GLM-5.3’s August 14 release, before yesterday’s launch. Anthropic’s self-reported 55.8% for Fable 5.1 would rank first if it held up under Terminal-Bench’s own harness, ahead of Opus 5’s verified 51.8% — but “would rank first if it held up” is exactly the kind of claim this blog declined to repeat for Terminal-Bench 3.0 all of August, and the standard doesn’t change because the number now favors Anthropic. Watch for Fable 5.1 to actually populate the board before treating that figure as settled.
Second, not every Claude entry looks good here. Sonnet 5 ties for last place at 12.4%, and the leaderboard’s own commentary explains why: it “sometimes hit timeouts and output-token-exceeded errors” and burned 21.6 billion tokens doing it — more than three times Opus 5’s 6.5 billion, for a quarter of the score. That’s a real result too, and it’s a useful reminder that “Claude” isn’t one model with one performance profile; effort level, harness, and model tier all matter as much as brand.
One more disclosure worth naming: the grants funding these leaderboard runs come from OpenAI, Anthropic, Z.ai, and SpaceX AI — three of the four labs whose models are being ranked, funding the infrastructure that ranks them. Terminal-Bench 4.0’s methodology (flat 8-hour timeouts based on Anthropic’s own infrastructure-noise research, tasks pulled for saturation or public solutions, versioning as a “continuous benchmark” rather than a fixed scoreboard) is a real improvement over 3.0’s opacity. But a benchmark funded by its own entrants deserves more scrutiny, not less — especially from a blog that just spent a month declining to cite this exact leaderboard.
The actual takeaway#
Two Claude models in the top two spots on an independently-hosted, cost-and-token-annotated leaderboard is a real result, arriving the same day Anthropic shipped a model whose own self-reported numbers would extend that lead further. That’s a good day for Anthropic by any measure. It’s also a good day to note what isn’t verified yet: Fable 5.1’s Terminal-Bench 4.0 score, Mythos 5.1’s real-world cyber/bio performance outside its restricted-access programs, and whether GLM-5.3’s far cheaper run ($2.7k for 41.8%, versus Opus 5’s $6.0k for 51.8%) starts looking more attractive to teams optimizing for cost per solved task rather than raw leaderboard rank. Watch for the leaderboard to add Fable 5.1 — that’s the number that actually closes the loop.
Sources: Anthropic — Introducing Claude Fable 5.1 and Claude Mythos 5.1; Claude Fable 5.1 & Mythos 5.1 System Card (PDF); Terminal-Bench 4.0 leaderboard; Terminal-Bench 4.0 methodology notes; Claude Code changelog.