Skip to main content
  1. Articles/

Qwen3.8-Max Finally Ships a Benchmark Table — and It's a Mixed Picture

·1128 words·6 mins·
Florent Clairambault
Author
Florent Clairambault
CTO & software engineer — writing daily about spec-driven development and agentic coding

Qwen3.8-Max Finally Ships a Benchmark Table — and It’s a Mixed Picture

Two weeks ago, this blog looked at Qwen3.8-Max-Preview and found nothing to actually evaluate: a 2.4-trillion-parameter headline, a self-graded “second only to Fable 5” claim, and no benchmark table, license, or active-parameter figure to back it up. The verdict then was simple — don’t factor it into any evaluation matrix until Alibaba ships real numbers. On August 3, Alibaba shipped them.

What Actually Landed
#

Qwen3.8-Max is now generally available through Alibaba Cloud’s Model Studio API and QwenWork, the company’s workplace-agent platform positioned against Claude Cowork and ChatGPT Work. The headline architecture detail that was missing from the preview is now public: 2.4 trillion total parameters with 95 billion active per inference pass via a Gated DeltaNet mixture-of-experts design, consistent with the attention mechanism Qwen used in the 3.5 and 3.6 generations. Context window is 1 million input tokens with up to 131,000 tokens of output. Pricing landed at $2.00 per million input tokens and $6.00 per million output tokens — roughly a fifth of Claude Fable 5’s $10/$50 rate card.

Open weights for both Qwen3.8-Max and a smaller Qwen3.8-27B variant are promised for the week of August 10. If that holds, it would be the first time a Max-tier Qwen model has gone open-weight rather than staying API-only — a real departure from how Alibaba handled the 3.6 generation, where Qwen 3.6-Max-Preview stayed closed while only the smaller Qwen 3.6-27B shipped with weights. No license text has been published yet, so there’s nothing to compare against Kimi K3’s revenue-gated “Modified MIT”-labeled terms until the actual repository goes live.

The Benchmark Table
#

Alibaba’s own release table, since aggregated by multiple outlets including Apidog’s technical breakdown, puts Qwen3.8-Max against Claude Opus 4.8, Claude Fable 5, and GPT-5.6 Sol on six benchmarks:

BenchmarkQwen3.8-MaxOpus 4.8Fable 5GPT-5.6 Sol
Terminal-Bench 2.186.684.684.688.8
SWE-bench Pro67.769.280.064.6
PaperBench93.080.388.890.5
GPQA Diamond92.692.092.694.1
IFBench82.862.263.572.7
HLE43.645.753.347.2

Read past the “beats Anthropic” framing in Alibaba’s own marketing and the picture is genuinely split. Qwen3.8-Max does edge both Claude flagships on Terminal-Bench 2.1 by two points, and it clears GPT-5.6 Sol on SWE-bench Pro (67.7% vs. 64.6%) — a real result if it holds up independently, since SWE-bench Pro is specifically designed to resist the kind of benchmark gaming that inflates scores on more contaminated suites. But it’s not close to Fable 5 on the same benchmark, trailing by more than 12 points (67.7% vs. 80.0%), and it trails Fable 5 by nearly 10 points on HLE, a benchmark that tends to track general reasoning depth rather than agentic tool-use. GPT-5.6 Sol still leads Terminal-Bench 2.1 outright at 88.8%. This is not the across-the-board parity claim “second only to Fable 5” implied two weeks ago — it’s a model that’s genuinely strong on agentic terminal tasks and instruction-following (IFBench: 82.8, well ahead of every listed competitor), and meaningfully behind on the hardest software-engineering and reasoning benchmarks.

Still Nobody Else Has Scored It
#

Every number above is Alibaba’s own. As of this writing, Qwen3.8-Max doesn’t appear on Artificial Analysis’s leaderboard, Scale AI’s SWE-bench Pro trackers, or Hugging Face — there’s no independent lab result to check the self-reported table against. That’s precisely the failure mode this blog flagged with GPT-5.6 Sol’s METR eval-gaming episode, Grok 4.5’s self-tested comparison chart, and Meta’s Muse Spark 1.1, where Vals AI’s independent Terminal-Bench rerun came in more than 11 points below Meta’s own number. Vendor tables are a starting point, not a verdict, and Qwen3.8-Max’s table is exactly as unverified today as its predecessor’s bare parameter count was two weeks ago — it’s just a more specific, more checkable set of claims now.

The one genuinely third-party data point available is a leaderboard placement rather than a benchmark run: Qwen3.8-Max debuted on Arena.ai’s human-judged Frontend Code Arena at 1,668 points, 37 points behind Claude Opus 5’s leading configuration (1,705) but ahead of Meta’s Muse Spark 1.1 and more than a dozen other frontier models. Arena placements are a different kind of signal than SWE-bench — closer to a live human-preference vote than a graded task suite — but it’s real independent ranking data where the rest of the table isn’t.

The Price Story Is the More Interesting One
#

Even taking Alibaba’s own numbers at face value, the benchmark gap to Fable 5 on SWE-bench Pro and HLE is real enough that Qwen3.8-Max isn’t a drop-in frontier replacement for the hardest agentic coding work. But at $2/$6 per million tokens against Fable 5’s $10/$50, it’s competing on a completely different axis: a model that beats GPT-5.6 Sol on SWE-bench Pro and both Claude flagships on Terminal-Bench 2.1, at roughly a fifth of Fable 5’s price and well under half of Opus 4.8’s $5/$25 rate. That’s the same pattern GLM-5.2 and DeepSeek V4-Pro established earlier this year — open-weight and cost-tier Chinese models don’t need to beat the frontier outright to be a rational default for cost-sensitive or high-volume agentic workloads; they need to be close enough on the benchmarks that matter for the task at hand. Alibaba’s Hong Kong-listed shares rose 7% to HK$125.20 on the announcement, which is as much a read on that pricing story as on the raw capability numbers.

What to Actually Do
#

Don’t move a production workload onto Qwen3.8-Max’s benchmark table alone — treat every number in it as a vendor claim until Artificial Analysis, Scale AI, or another independent evaluator publishes a rerun. But the earlier verdict — “there’s nothing to benchmark against” — no longer applies. There’s now a specific, falsifiable set of claims, a real price point, and a concrete open-weight date (the week of August 10) to watch. If the weights land on schedule with a genuine license and the SWE-bench Pro and Terminal-Bench numbers survive independent verification within a reasonable margin, Qwen3.8-Max becomes a legitimate budget-tier option for agentic coding workloads that don’t need Fable 5’s ceiling. Until then, it’s a promising, specific, unverified claim — which is real progress from three weeks ago, but still not a data point to build a migration plan on.


Sources:

Related