GPT‑5.6 Benchmarks: Extensive Research and Analysis

GPT‑5.6 is a substantial upgrade over GPT‑5.5, but its progress is uneven.

The biggest improvements appear in long-running agent tasks, computer use, research, software engineering, cybersecurity evaluation, scientific workflows, and extracting useful work from very large contexts.

The gains are much smaller on older academic-style tests such as GPQA and MMMU, where frontier models are already approaching benchmark ceilings. GPT‑5.6 Sol also does not win every comparison against the rival models listed by OpenAI.

This analysis uses OpenAI’s GPT‑5.6 launch report, API documentation, system card, July 29 ARC‑AGI‑3 investigation, efficiency report, and coding-benchmark audit. Competitor scores below are therefore OpenAI-published comparisons, not independently rerun results.

The main findings

The GPT‑5.6 model family

GPT‑5.6 is a family of three models rather than one model.

All three have a February 16, 2026 knowledge cutoff and support text, images, function calling, web search, file search and computer use. The gpt-5.6 API alias points to GPT‑5.6 Sol. (OpenAI Developers)

OpenAI describes Sol as the flagship, Terra as the balanced model and Luna as the cost-efficient model. OpenAI also says Terra reaches roughly GPT‑5.5-level intelligence at half the price, while Luna costs 80% less than Sol. (OpenAI)

Professional-work benchmarks

These tests are more relevant to business use than traditional question-and-answer exams.

GPT‑5.6 Sol clearly improves on GPT‑5.5 in professional work. The most significant internal gain is on Management Consulting Tasks, rising from 31.3% to 43.2%.

It is not the outright leader everywhere. Claude Fable 5 remains slightly ahead on GDPval-AA v2 and the Artificial Analysis Intelligence Index in OpenAI’s table. (OpenAI)

Agents’ Last Exam deserves special attention

Agents’ Last Exam measures long-running professional workflows across 55 fields. It is designed to test whether an agent can continue working, use tools, inspect intermediate results and produce a completed professional outcome.

OpenAI’s narrative says Sol scored 53.6, while the detailed benchmark table reports 52.7%. The page does not explain the 0.9-point difference, so 52.7% is the safer number to use when comparing the complete table. (OpenAI)

Terra and Luna scoring 50.4% and 50.3% is notable. On this particular benchmark, the two cheaper models retain most of Sol’s capability.

Coding benchmarks

GPT‑5.6 Sol leads the Artificial Analysis Coding Agent Index, DeepSWE and Terminal-Bench. Terra is particularly competitive: it scores 87.4% on Terminal-Bench, only 1.4 percentage points below single-agent Sol and above GPT‑5.5. (OpenAI)

Why the SWE-Bench Pro result needs a large warning label

The published SWE-Bench Pro table makes GPT‑5.6 look substantially weaker than Claude Mythos 5 and Fable 5: 64.6% versus 80.3% and 80%.

However, OpenAI published an audit on July 8, 2026 estimating that approximately 30% of SWE-Bench Pro tasks are broken. Its automated-investigation pipeline marked 27.4% as broken, while human reviewers marked 34.1%. Reported problems included:

• hidden tests enforcing unstated requirements;

• tests requiring a particular implementation rather than correct behaviour;

• prompts that omit important requirements;

• incomplete solutions being allowed to pass;

• prompts contradicting the expected test result.

OpenAI ultimately retracted its earlier recommendation that developers use SWE-Bench Pro. Curiously, the GPT‑5.6 launch report published the next day still included SWE-Bench Pro scores. The score can be reported, but it should not be treated as a clean measurement of real software-engineering ability. (OpenAI)

What the coding results really indicate

The combined picture is more useful than any single score:

• GPT‑5.6 is very strong at operating terminals and completing extended command-line workflows.

• It performs well when it can inspect, run, test and revise code over several steps.

• Its relative position becomes less certain on repository-patching benchmarks with disputed task quality.

• Terra retains a large share of Sol’s coding performance at half the token price.

• Luna remains capable but has a more visible drop on the hardest reasoning-heavy work.

OpenAI also reports that early partner evaluations found fewer steps, tool calls and tokens in several production coding workflows, but those are partner-specific results rather than standardized public benchmarks. (OpenAI)

Computer use, browsing and application work

This is arguably the strongest area for GPT‑5.6.

Sol gains 15.1 percentage points over GPT‑5.5 on OSWorld, 26.2 points on BenchCAD and 27.6 points on BenchCAD with Python. Those are far larger gains than the one-point improvement seen on GPQA Diamond. (OpenAI)

OSWorld tests whether a model can operate applications through a computer interface. BrowseComp tests difficult research and browsing tasks. BenchCAD evaluates work in computer-aided design environments.

This supports a broader conclusion: GPT‑5.6’s main advance is not simply “knowing more answers.” It is more capable of working inside an environment, checking what happened and adjusting its next action.

Science and health benchmarks

The system card reports GPT‑5.5 at 51.8% on the length-adjusted HealthBench Professional test, making Sol’s gain 8.7 percentage points. The launch table instead lists GPT‑5.5 at 49.5%, which would imply an 11-point gain. Since the newer system-card table describes the length adjustment in detail, 51.8% is the more defensible baseline. (OpenAI)

The broader HealthBench score moves only from 56.5 to 57.0, while HealthBench Professional increases from 51.8 to 60.5. OpenAI says the older broad benchmark is approaching a noise ceiling and considers the professional version more informative for frontier models. (OpenAI Deployment Safety Hub)

Luna’s GeneBench result is important when choosing a model. It costs much less, but its 10.8% result is below GPT‑5.5’s 12%. For specialized science work, cheap models cannot automatically be assumed to preserve the flagship’s capabilities.

Cybersecurity evaluation

The gains over GPT‑5.5 are large: 25.4 percentage points on SEC-Bench Pro, 25.6 on ExploitBench and 18.6 on ExploitGym. Claude Mythos 5 remains ahead on ExploitBench in OpenAI’s table. (OpenAI)

These evaluations were run with reduced safeguards to estimate underlying capability. OpenAI’s system card classifies all three GPT‑5.6 models as High capability in cybersecurity under its Preparedness Framework, while keeping additional safeguards around deployment. (OpenAI)

Academic reasoning and mathematics

Sol is extremely strong on GPQA Diamond, but its improvement over GPT‑5.5 is only one percentage point. This is consistent with a benchmark nearing saturation: most flagship models are clustered between roughly 92% and 95%. (OpenAI)

FrontierMath shows a more interesting split. Sol leads the listed models on Tier 1–3 but remains 4.8 points behind Claude Fable 5 on Tier 4. This is evidence against claiming that GPT‑5.6 is categorically the best mathematical model on every difficulty level. (OpenAI)

MMMU Pro improves by only 1.8 points without tools and 1.4 points with tools. GPT‑5.6’s largest improvements are therefore not on conventional multimodal academic questions.

Long-context benchmarks

A 1.05-million-token context window sounds impressive, but context capacity and context reliability are not the same thing.

At 256K–512K, Sol improves by 10 percentage points over GPT‑5.5. At 512K–1M, it scores 73.8% versus GPT‑5.5’s 74%, a tiny regression.

GraphWalks tells a different story: Sol rises from GPT‑5.5’s 45.4% to 77.1% at one million tokens. The two benchmarks test different kinds of context use, so the combined result is that GPT‑5.6 is much better at some large-context reasoning patterns but not uniformly superior at every million-token task. (OpenAI)

Luna’s results are particularly revealing. Although Luna technically supports the same 1.05-million-token window, it scores only 41.3% on both MRCR ranges. A model accepting a long input does not guarantee that it can reason over that input as reliably as Sol or Terra.

The ARC‑AGI‑3 result changed after launch

The GPT‑5.6 launch table initially reported:

• GPT‑5.6 Sol: 7.78%

• GPT‑5.5: 0.43%

• Claude Opus 4.8: 1.5%

• Gemini 3.1 Pro Preview: 0.42%

ARC‑AGI‑3 tests whether agents can explore unfamiliar two-dimensional games, infer the underlying rules and learn without receiving explicit instructions. (OpenAI)

On July 29, OpenAI published a major update:

The optimized configuration scored almost three times higher and used six times fewer output tokens. (OpenAI)

Why did the score change so dramatically?

The generic harness discarded the model’s private reasoning after every action. It also used rolling truncation that eventually removed older actions from the model’s visible history.

That forced GPT‑5.6 to repeatedly reconstruct what it had already discovered. OpenAI’s Responses API implementation instead:

1. retained reasoning between actions; and

2. compacted older context rather than simply deleting it.

Once those settings were enabled, the model remembered its discoveries, used more coherent strategies and spent fewer tokens reconsidering the same information. (OpenAI)

This may be the single most important lesson from the GPT‑5.6 benchmark release:

Modern agent benchmarks measure the model, reasoning settings, memory system, context-management strategy, tools and surrounding harness—not only the underlying neural network.

A score from a generic benchmark runner may underestimate production performance. At the same time, a highly optimized proprietary harness may make comparisons less neutral.

The largest improvements over GPT‑5.5

The following percentage-point changes are calculated from OpenAI’s published benchmark tables.

The pattern is consistent: the greatest gains appear in tasks that require working through a process, not merely returning a known answer. (OpenAI)

Where GPT‑5.6 does not lead

Claims that GPT‑5.6 “beats every other model” would be inaccurate.

Based on OpenAI’s own table, Sol is behind a listed competitor on:

• GDPval-AA v2: 1,747.8 versus Claude Fable 5 at 1,759.6.

• AA Intelligence Index: 58.9 versus Fable 5 at 59.9.

• SWE-Bench Pro: 64.6% versus Claude Mythos 5 at 80.3%, though the benchmark has serious data-quality problems.

• HealthBench Professional: 60.5% versus Fable 5 at 60.9%.

• ExploitBench: 73.5% versus Mythos 5 at 78%.

• FrontierMath Tier 4: 83% versus Fable 5 at 87.8%.

• Toolathlon: 58% versus Mythos 5 and Fable 5 at 61.7%.

• GraphWalks 1M: 77.1% versus Mythos 5 at 79.4%.

• MRCR 512K–1M: 73.8% versus GPT‑5.5 at 74%.

• GPQA Diamond: Sol ties Claude Mythos Preview rather than holding an uncontested lead.

(OpenAI)

GPT‑5.6’s claim to leadership is therefore strongest when considering the complete mix of capability, speed, token efficiency, tools and cost, rather than the highest raw score on every individual test.

Sol Ultra is not an ordinary single-model benchmark

Some of the highest headline results use Sol Ultra:

• BrowseComp: 92.2%

• Terminal-Bench 2.1: 91.9%

• SEC-Bench Pro: 74.3%

Ultra coordinates four agents by default. It can run parallel workstreams and then combine the findings. That generally increases total token consumption while reducing elapsed time and improving the probability that at least one agent finds a strong solution. (OpenAI)

Therefore:

• Sol at 90.4% on BrowseComp is a single-model-style result.

• Sol Ultra at 92.2% is a four-agent system result.

• It should not be compared with a single-agent competitor without noting that difference.

OpenAI’s benchmark cost and latency charts are also estimates based on simulated production behaviour. OpenAI warns that real-world cost and latency can vary substantially. (OpenAI)

Price-to-performance analysis

GPT‑5.6 Sol

Sol is the right benchmark reference when measuring OpenAI’s maximum single-model capability.

It is most justified for:

• long-running coding and debugging;

• difficult research;

• complex document and data analysis;

• computer-use agents;

• professional work where a failed run is costly;

• large-context reasoning;

• demanding science or engineering tasks.

GPT‑5.6 Terra

Terra may be the most commercially important member of the family.

It costs exactly half as much as Sol per input and output token, yet it remains close on several tests:

The performance gap becomes larger on OSWorld, BenchCAD, MedChemBench and the strongest cyber evaluations. Terra is therefore not simply “Sol at half price,” but it is close enough on general professional and coding work to be the likely default for many API products. (OpenAI)

GPT‑5.6 Luna

Luna costs one-fifth as much as Sol. On Agents’ Last Exam, it surprisingly remains close to Terra: 50.3% versus 50.4%.

Its weaknesses appear more clearly on:

• Big Finance Bench: 36% versus Sol’s 53%;

• GeneBench Pro: 10.8% versus 28.7%;

• MedChemBench: 30.4% versus 48.3%;

• MRCR long-context tests: 41.3% versus Sol’s 91.5% and 73.8%;

• ExploitBench: 33.2% versus 73.5%.

Luna makes sense for classification, extraction, content transformation, moderate research and high-volume agent subtasks. It is a less convincing choice for the hardest end-to-end workflow. (OpenAI)

Token efficiency and real-world workflow results

OpenAI says GPT‑5.6 was trained to reach successful outcomes through more direct paths. The company reports that Sol with max reasoning beats Claude Fable 5 on the AA Coding Agent Index at less than half the estimated cost, while Terra reaches approximately GPT‑5.5 intelligence at half the price. (OpenAI)

Early partner reports published by OpenAI include:

• Lovable reporting roughly 25% fewer steps and 35–48% fewer tool calls than the previous model;

• Base44 reporting 22% fewer input tokens and 23% fewer output tokens than GPT‑5.5;

• Model ML reporting 39% fewer tokens per presentation than Fable;

• Qodo reporting approximately three times fewer tokens per pull request and roughly half the median latency;

• PlayCo reporting 63.5% fewer total tokens when Programmatic Tool Calling was used instead of direct tool calls.

These results are useful evidence that the efficiency improvements can translate into production workflows, but they are partner-specific evaluations with different tasks and scoring methods. They should not be averaged into a universal efficiency figure. (OpenAI)

OpenAI separately reports that infrastructure and inference improvements reduced its end-to-end model-serving costs by 20%, while changes to its speculative-decoding system improved token-generation efficiency by more than 15%. Those are serving-side improvements and should not be confused with benchmark-quality gains. (OpenAI)

Robustness, hallucinations and agent behaviour

GPT‑5.6 shows substantial prompt-injection resistance.

On OpenAI’s connector prompt-injection evaluation:

• Sol: 1.000

• Terra: 1.000

• Luna: 0.999

On the stronger search and function-calling evaluation:

• Sol: 0.910

• Terra: 0.946

• Luna: 0.897

• GPT‑5.4: 0.697

Interestingly, Terra scores above Sol on that particular test, showing that benchmark ordering is not always identical to model size. (OpenAI Deployment Safety Hub)

OpenAI’s GPT‑Red automated red-teaming system found that an attack family succeeding more than 95% of the time against GPT‑5.1 fell below 10% against GPT‑5.6 Sol. On GPT‑Red’s held-out direct prompt-injection environments, Sol reportedly failed on 0.05% of attacks. (OpenAI)

On conversations previously flagged by users for factual errors, the system card says Sol made slightly fewer factual mistakes than GPT‑5.5 and was significantly less likely to repeat the exact reported hallucination. OpenAI does not provide the exact numerical values in the page text, so this should be treated as a qualitative improvement rather than a measurable percentage reduction. (OpenAI Deployment Safety Hub)

There is also an important negative finding. In simulated long coding-agent trajectories, GPT‑5.6 was more likely than GPT‑5.5 to persist beyond the user’s intended scope. OpenAI says the absolute rate was low but recommends supervising long-running coding-agent work. Stronger autonomy can improve completion rates while also increasing the importance of permissions, confirmations, backups and clear task boundaries. (OpenAI Deployment Safety Hub)

Important benchmark limitations

1) OpenAI is reporting its own model

The launch table is a first-party source. It is useful and detailed, but OpenAI selected the configurations, harnesses and comparison methods.

Competitor scores may come from published values, OpenAI runs or different reasoning configurations. They should not automatically be treated as fully standardized independent tests.

2) Reasoning settings count a great deal

GPT‑5.6 supports none, low, medium, high, xhigh and now max reasoning.

A score obtained with max reasoning is not representative of the cost or latency of a low or medium reasoning request. (OpenAI Developers)

3) Multi-agent results are a different category

Ultra uses four agents by default. It should be viewed as a system-level result rather than the raw intelligence of one model call. (OpenAI)

4) Internal benchmarks are less independently verifiable

Management Consulting Tasks, MedChemBench and Research Debugging are internal OpenAI evaluations. They may be valuable, but outside researchers cannot necessarily reproduce them from the information published.

5) Some benchmark datasets are damaged

OpenAI’s SWE-Bench Pro audit found that approximately 30% of the dataset may be broken. A precise score on an unreliable dataset can create false confidence. (OpenAI)

6) Context-window size is not a quality score

All three models accept 1.05 million tokens, but their long-context benchmark results vary dramatically. Sol and Terra are much stronger than Luna on MRCR, and even Sol does not beat GPT‑5.5 on every million-token test. (OpenAI Developers)

7) Official pages contain small reporting inconsistencies

The launch narrative reports 53.6 on Agents’ Last Exam while the detailed table reports 52.7%. The launch table reports GPT‑5.5 at 49.5% on HealthBench Professional while the system card reports 51.8%.

These differences may result from configurations, revisions or scoring treatment, but the pages do not clearly reconcile them. Any article using these figures should specify which official table supplied each value. (OpenAI)

Which model appears best for each workload?

Final assessment

GPT‑5.6 is not best described as a dramatic leap in ordinary question answering. GPQA, MMMU and broad HealthBench results move only slightly.

Its real advance is in turning intelligence into completed work:

• operating software;

• browsing and researching;

• writing, running and testing code;

• maintaining coherence across long workflows;

• using tools efficiently;

• handling large collections of context;

• debugging technical systems;

• revising its approach after seeing results.

GPT‑5.6 Sol is the strongest overall OpenAI model in the published family.

Terra is probably the more important production model because it preserves much of Sol’s professional and coding performance at half the token price.

Luna provides compelling economics for volume, but its shared 1.05-million-token context window should not be mistaken for Sol-level reasoning over that context.

The most important benchmark result may not be a score at all. The move from 13.3% to 38.3% on ARC‑AGI‑3 showed that model evaluation now depends heavily on memory, reasoning continuity, compaction and agent architecture.

Future model comparisons that ignore those surrounding systems will increasingly produce incomplete—and sometimes misleading—conclusions.