BestSuperintelligent
Explore evaluations
Research

Vendor-reported benchmark results

Results that AI vendors publish about their own models, recorded with their harness and caveats. None was reproduced here, so none is a score or a ranking.

All records

7 records across 4 benchmarks, as read on 2026-10-24.

Summary of records

Vendor-reported benchmark results by model
BenchmarkModelReported resultSource
DeepSWE v1.1GLM-5.366.9 score as reported (unit not stated)Vendor page, vendor-reported
DeepSWE v1.1Kimi K367.3 score as reported (unit not stated)Vendor page, vendor-reported
Humanity's Last Exam with toolsClaude Fable 5.165.6 %Vendor page, vendor-reported
Humanity's Last Exam with toolsClaude Opus 5.567.7 %Vendor page, vendor-reported
Terminal-Bench 3.0GLM-5.328.3 score as reported (unit not stated)Vendor page, vendor-reported
Terminal-Bench 4.0Claude Fable 5.155.8 %Vendor page, vendor-reported
Terminal-Bench 4.0Claude Opus 5.566.4 %Vendor page, vendor-reported

DeepSWE v1.1

Software-engineering tasks from the DeepSWE task set.

DeepSWE v1.1 records

GLM-5.3: 66.9 score as reported (unit not stated)

Status: vendor-reported, not reproduced by BestSuperintelligent. Read 2026-10-24. Source.

How it was obtained: Z.ai GLM-5.3 guide (vendor-reported); harness not stated in the passage cited.

Caveats: Different vendors use different harnesses, so cross-vendor comparison is weak. Not reproduced by BestSuperintelligent. Contamination: Not assessed; public benchmark.

Comparable results published with the GLM-5.3 record
SubjectScoreSource kind
GLM-5.366.9vendor-reported
GLM-5.2 (vendor-reported)46.2vendor-reported
Kimi K3, mini-SWE-agent (vendor-reported)67.3vendor-reported

Kimi K3: 67.3 score as reported (unit not stated)

Status: vendor-reported, not reproduced by BestSuperintelligent. Read 2026-10-24. Source.

How it was obtained: Moonshot's Kimi K3 blog says this is from the official DeepSWE leaderboard (https://deepswe.datacurve.ai/) with the mini-SWE-agent harness; the leaderboard itself was not checked.

Caveats: Vendor-reported citation of a leaderboard; not confirmed against the leaderboard. Moonshot's own headline runs use its Kimi Code harness. Not reproduced by BestSuperintelligent. Contamination: Not assessed; public benchmark.

Comparable results published with the Kimi K3 record
SubjectScoreSource kind
Kimi K3, mini-SWE-agent67.3vendor-reported
GLM-5.3 (Z.ai, harness not stated)66.9vendor-reported

Humanity's Last Exam with tools

Multidisciplinary expert-level reasoning questions.

Humanity's Last Exam with tools records

Claude Fable 5.1: 65.6 %

Status: vendor-reported, not reproduced by BestSuperintelligent. Read 2026-10-24. Source.

How it was obtained: Anthropic's Claude Opus 5.5 announcement table (vendor-reported). Anthropic says Opus 5.5 results use adaptive thinking at max effort unless noted, and Terminal-Bench 4.0 is reported at xhigh effort; its footnote gives a standard error of about 2.6 points for Opus 5.5 and cites the public leaderboard's Claude Code harness. Reported "with tools".

Caveats: Tool use changes results; a public question set carries contamination risk. Not reproduced by BestSuperintelligent. Contamination: Not assessed; public benchmark.

Comparable results published with the Claude Fable 5.1 record
SubjectScoreSource kind
Claude Opus 5.567.7%vendor-reported
Claude Fable 5.165.6%vendor-reported

Claude Opus 5.5: 67.7 %

Status: vendor-reported, not reproduced by BestSuperintelligent. Read 2026-10-24. Source.

How it was obtained: Anthropic's Claude Opus 5.5 announcement table (vendor-reported). Anthropic says Opus 5.5 results use adaptive thinking at max effort unless noted, and Terminal-Bench 4.0 is reported at xhigh effort; its footnote gives a standard error of about 2.6 points for Opus 5.5 and cites the public leaderboard's Claude Code harness. Reported "with tools".

Caveats: Tool use changes results; a public question set carries contamination risk. Not reproduced by BestSuperintelligent. Contamination: Not assessed; public benchmark.

Comparable results published with the Claude Opus 5.5 record
SubjectScoreSource kind
Claude Opus 5.567.7%vendor-reported
Claude Fable 5.165.6%vendor-reported
Claude Opus 563.6%vendor-reported
GPT-6 Astra (as reported by OpenAI, per Anthropic)57.2%vendor-reported

Terminal-Bench 3.0

Agentic coding tasks run in a terminal environment.

Terminal-Bench 3.0 records

GLM-5.3: 28.3 score as reported (unit not stated)

Status: vendor-reported, not reproduced by BestSuperintelligent. Read 2026-10-24. Source.

How it was obtained: Z.ai GLM-5.3 guide (vendor-reported); harness not stated in the passage cited.

Caveats: Vendor-reported improvement claim; no independent reproduction. Not reproduced by BestSuperintelligent. Contamination: Not assessed; public benchmark.

Comparable results published with the GLM-5.3 record
SubjectScoreSource kind
GLM-5.328.3vendor-reported
GLM-5.2 (vendor-reported)4.6vendor-reported

Terminal-Bench 4.0

Agentic coding tasks run in a terminal environment.

Terminal-Bench 4.0 records

Claude Fable 5.1: 55.8 %

Status: vendor-reported, not reproduced by BestSuperintelligent. Read 2026-10-24. Source.

How it was obtained: Anthropic's Claude Opus 5.5 announcement table (vendor-reported). Anthropic says Opus 5.5 results use adaptive thinking at max effort unless noted, and Terminal-Bench 4.0 is reported at xhigh effort; its footnote gives a standard error of about 2.6 points for Opus 5.5 and cites the public leaderboard's Claude Code harness.

Caveats: Vendor-reported in Opus 5.5's launch post, where Fable trails Opus 5.5. Not reproduced by BestSuperintelligent. Contamination: Not assessed; public benchmark.

Comparable results published with the Claude Fable 5.1 record
SubjectScoreSource kind
Claude Opus 5.566.4%vendor-reported
Claude Fable 5.155.8%vendor-reported
GPT-6 Astra (as reported by OpenAI, per Anthropic)57.9%vendor-reported

Claude Opus 5.5: 66.4 %

Status: vendor-reported, not reproduced by BestSuperintelligent. Read 2026-10-24. Source.

How it was obtained: Anthropic's Claude Opus 5.5 announcement table (vendor-reported). Anthropic says Opus 5.5 results use adaptive thinking at max effort unless noted, and Terminal-Bench 4.0 is reported at xhigh effort; its footnote gives a standard error of about 2.6 points for Opus 5.5 and cites the public leaderboard's Claude Code harness.

Caveats: Vendor-reported; competitor figures are Anthropic's own runs or reports and are not verified. Not reproduced by BestSuperintelligent. Contamination: Not assessed; public benchmark.

Comparable results published with the Claude Opus 5.5 record
SubjectScoreSource kind
Claude Opus 5.566.4%vendor-reported
Claude Fable 5.155.8%vendor-reported
Claude Opus 552.3%vendor-reported
GPT-6 Astra (as reported by OpenAI, per Anthropic)57.9%vendor-reported
GPT-5.6 Sol (as reported by OpenAI, per Anthropic)37.3%vendor-reported

Frequently asked questions

Are these your benchmark scores?

No. Each figure comes from a vendor's own announcement or guide and is labelled vendor-reported. BestSuperintelligent has not reproduced any of them, so they are claims, not findings.

Why not rank the models by these numbers?

Vendors use different harnesses, effort settings and tools, so scores from different pages are not comparable, and public question sets can leak into training data. Our rule is that nothing ranks until it is independently tested.

What would make a result count as ours?

A test we run under a published protocol, with every run recorded. The research page on reading benchmark claims sets out what a result must meet before we treat it as evidence.