↓ Skip to main content
  1. Articles/

Qwen3.8-Max Finally Ships a Benchmark Table — and It's a Mixed Picture

·1128 words·6 mins·
Florent Clairambault
Author
Florent Clairambault
CTO & software engineer — writing daily about spec-driven development and agentic coding

Qwen3.8-Max Finally Ships a Benchmark Table — and It’s a Mixed Picture

Two weeks ago, this blog looked at Qwen3.8-Max-Preview and found nothing to actually evaluate: a 2.4-trillion-parameter headline, a self-graded “second only to Fable 5” claim, and no benchmark table, license, or active-parameter figure to back it up. The verdict then was simple — don’t factor it into any evaluation matrix until Alibaba ships real numbers. On August 3, Alibaba shipped them.

What Actually Landed
#

Qwen3.8-Max is now generally available through Alibaba Cloud’s Model Studio API and QwenWork, the company’s workplace-agent platform positioned against Claude Cowork and ChatGPT Work. The headline architecture detail that was missing from the preview is now public: 2.4 trillion total parameters with 95 billion active per inference pass via a Gated DeltaNet mixture-of-experts design, consistent with the attention mechanism Qwen used in the 3.5 and 3.6 generations. Context window is 1 million input tokens with up to 131,000 tokens of output. Pricing landed at $2.00 per million input tokens and $6.00 per million output tokens — roughly a fifth of Claude Fable 5’s $10/$50 rate card.

Open weights for both Qwen3.8-Max and a smaller Qwen3.8-27B variant are promised for the week of August 10. If that holds, it would be the first time a Max-tier Qwen model has gone open-weight rather than staying API-only — a real departure from how Alibaba handled the 3.6 generation, where Qwen 3.6-Max-Preview stayed closed while only the smaller Qwen 3.6-27B shipped with weights. No license text has been published yet, so there’s nothing to compare against Kimi K3’s revenue-gated “Modified MIT”-labeled terms until the actual repository goes live.

The Benchmark Table
#

Alibaba’s own release table, since aggregated by multiple outlets including Apidog’s technical breakdown, puts Qwen3.8-Max against Claude Opus 4.8, Claude Fable 5, and GPT-5.6 Sol on six benchmarks:

BenchmarkQwen3.8-MaxOpus 4.8Fable 5GPT-5.6 Sol
Terminal-Bench 2.186.684.684.688.8
SWE-bench Pro67.769.280.064.6
PaperBench93.080.388.890.5
GPQA Diamond92.692.092.694.1
IFBench82.862.263.572.7
HLE43.645.753.347.2

Read past the “beats Anthropic” framing in Alibaba’s own marketing and the picture is genuinely split. Qwen3.8-Max does edge both Claude flagships on Terminal-Bench 2.1 by two points, and it clears GPT-5.6 Sol on SWE-bench Pro (67.7% vs. 64.6%) — a real result if it holds up independently, since SWE-bench Pro is specifically designed to resist the kind of benchmark gaming that inflates scores on more contaminated suites. But it’s not close to Fable 5 on the same benchmark, trailing by more than 12 points (67.7% vs. 80.0%), and it trails Fable 5 by nearly 10 points on HLE, a benchmark that tends to track general reasoning depth rather than agentic tool-use. GPT-5.6 Sol still leads Terminal-Bench 2.1 outright at 88.8%. This is not the across-the-board parity claim “second only to Fable 5” implied two weeks ago — it’s a model that’s genuinely strong on agentic terminal tasks and instruction-following (IFBench: 82.8, well ahead of every listed competitor), and meaningfully behind on the hardest software-engineering and reasoning benchmarks.

Still Nobody Else Has Scored It
#

Every number above is Alibaba’s own. As of this writing, Qwen3.8-Max doesn’t appear on Artificial Analysis’s leaderboard, Scale AI’s SWE-bench Pro trackers, or Hugging Face — there’s no independent lab result to check the self-reported table against. That’s precisely the failure mode this blog flagged with GPT-5.6 Sol’s METR eval-gaming episode, Grok 4.5’s self-tested comparison chart, and Meta’s Muse Spark 1.1, where Vals AI’s independent Terminal-Bench rerun came in more than 11 points below Meta’s own number. Vendor tables are a starting point, not a verdict, and Qwen3.8-Max’s table is exactly as unverified today as its predecessor’s bare parameter count was two weeks ago — it’s just a more specific, more checkable set of claims now.

The one genuinely third-party data point available is a leaderboard placement rather than a benchmark run: Qwen3.8-Max debuted on Arena.ai’s human-judged Frontend Code Arena at 1,668 points, 37 points behind Claude Opus 5’s leading configuration (1,705) but ahead of Meta’s Muse Spark 1.1 and more than a dozen other frontier models. Arena placements are a different kind of signal than SWE-bench — closer to a live human-preference vote than a graded task suite — but it’s real independent ranking data where the rest of the table isn’t.

The Price Story Is the More Interesting One
#

Even taking Alibaba’s own numbers at face value, the benchmark gap to Fable 5 on SWE-bench Pro and HLE is real enough that Qwen3.8-Max isn’t a drop-in frontier replacement for the hardest agentic coding work. But at $2/$6 per million tokens against Fable 5’s $10/$50, it’s competing on a completely different axis: a model that beats GPT-5.6 Sol on SWE-bench Pro and both Claude flagships on Terminal-Bench 2.1, at roughly a fifth of Fable 5’s price and well under half of Opus 4.8’s $5/$25 rate. That’s the same pattern GLM-5.2 and DeepSeek V4-Pro established earlier this year — open-weight and cost-tier Chinese models don’t need to beat the frontier outright to be a rational default for cost-sensitive or high-volume agentic workloads; they need to be close enough on the benchmarks that matter for the task at hand. Alibaba’s Hong Kong-listed shares rose 7% to HK$125.20 on the announcement, which is as much a read on that pricing story as on the raw capability numbers.

What to Actually Do
#

Don’t move a production workload onto Qwen3.8-Max’s benchmark table alone — treat every number in it as a vendor claim until Artificial Analysis, Scale AI, or another independent evaluator publishes a rerun. But the earlier verdict — “there’s nothing to benchmark against” — no longer applies. There’s now a specific, falsifiable set of claims, a real price point, and a concrete open-weight date (the week of August 10) to watch. If the weights land on schedule with a genuine license and the SWE-bench Pro and Terminal-Bench numbers survive independent verification within a reasonable margin, Qwen3.8-Max becomes a legitimate budget-tier option for agentic coding workloads that don’t need Fable 5’s ceiling. Until then, it’s a promising, specific, unverified claim — which is real progress from three weeks ago, but still not a data point to build a migration plan on.


Sources:

AI Tools

Alibaba's Qwen3.8-Max: A 2.4-Trillion-Parameter Model With No Benchmarks Attached

Alibaba previewed Qwen3.8-Max on July 19 — a 2.4-trillion-parameter multimodal model it calls ‘second only to Fable 5’ — three days after Moonshot’s Kimi K3 claimed the largest open-weight release yet. Alibaba published the parameter count and nothing else: no benchmark table, no model card, no license, no active-parameter figure for what’s presumably a sparse MoE.

Jul 20, 2026 · 5 min read

AI Tools

Kimi K3: A 2.8-Trillion-Parameter Open-Weight Model Just Beat Fable 5 on Terminal-Bench

Moonshot AI’s Kimi K3 (announced July 16) is a 2.8-trillion-parameter model — the largest open-weight release yet — pricing at $3/$15 per million tokens, scoring 88.3 on Terminal-Bench 2.1 (ahead of Claude Fable 5’s 84.6) and topping Arena.ai’s Frontend Code leaderboard. Open weights ship July 27 under Moonshot’s Modified MIT license, but the model needs preserved reasoning history to perform reliably, and it’s notably more verbose — and pricier per task — than its Terminal-Bench win suggests.

Jul 19, 2026 · 5 min read

AI Tools

Kimi K2.7-Code: 30% Fewer Thinking Tokens, 100% Proprietary Benchmarks

Moonshot AI released Kimi K2.7-Code on June 12, claiming 21.8% gains over its predecessor and 30% fewer thinking tokens. It’s open-weight, cheaply priced, and architecturally interesting — but every benchmark is proprietary. Here’s what we can and can’t trust.

Jun 14, 2026 · 5 min read

AI Tools

Kimi K2.6: The Open-Weight Model That Scales to 300 Sub-Agents

Moonshot AI’s Kimi K2.6 landed on April 20 as the most capable open-weight coding model ever released: 1T-parameter MoE, 58.6% SWE-bench Pro, 66.7% Terminal-Bench 2.0, and an Agent Swarm that scales to 300 sub-agents executing 4,000 coordinated steps — at $0.60 per million input tokens.

May 12, 2026 · 6 min read

AI Tools

GLM-5.1: The Open-Source Model That Just Beat Everyone on SWE-bench Pro

Z.AI released GLM-5.1 today — a 754B open-weight model under MIT license that scored 58.4% on SWE-bench Pro, beating GPT-5.4, Claude Opus 4.6, and Gemini 3.1 Pro. Its headline demo: an 8-hour autonomous session that built a complete Linux desktop environment across 655 iterations. The closed-model monopoly on frontier coding capability just got its first serious challenge.

Apr 8, 2026 · 6 min read