DeepSWE Value on a 1M-Token Codebase: Flash, Pro, or Luna?
Published August 22, 2026
Series: News — Model appraisal
Published: 2026-08-22
Author: Victor Salmon
Dataset: Clock Lobster LLM-Cost-Comparison
Short answer: on a million-token codebase, DeepSeek V4 Flash 0731 is the practical recommendation for routine implementation. GPT-5.6 Luna is the better middle ground when the task needs more independent judgment. DeepSeek V4 Pro buys a higher DeepSWE score, but the extra reasoning tokens and higher output price mean that it produces fewer successful solutions per $100 in this modelled workload.
This is not a universal ranking. It is a costed scenario for independent programmers working on large repositories, where the whole codebase is available to the agent on each solution attempt. That distinction matters more than a leaderboard screenshot.
What does the comparison show?
The table below combines the published DeepSWE v1.1 score with a one-million-token uncached input and a ten-thousand-token visible answer budget. Thinking or effort tokens are counted as billed output. “Successful solutions per $100” is the number of attempts affordable at that cost multiplied by the DeepSWE pass rate.
| Model | DeepSWE | Input $/M | Output $/M | Effort multiplier | Cost / attempt | Attempts / $100 | Successful solutions / $100 |
|---|---|---|---|---|---|---|---|
| DeepSeek V4 Flash 0731 | 54.4% | $0.14 | $0.28 | 1.34× | $0.144 | 696 | 378 |
| DeepSeek V4 Pro 0813 | 62.7% | $0.435 | $0.87 | 1.57× | $0.449 | 223 | 140 |
| GPT-5.6 Luna | 67.2% | $0.20 | $1.20 | 1.25× | $0.215 | 465 | 313 |
DeepSeek V4 Pro 0813 is the GA checkpoint represented here, using the published 62.7% DeepSWE result and $0.435/$0.87 pricing.
How I accounted for the extra token burden
A cheap input token does not make a large-repository task cheap. If the agent needs the whole codebase, the first pass is already roughly one million input tokens. On top of that, a reasoning model may generate hidden effort tokens before it writes the answer or calls a tool.
For a comparable calculation, I used:
- 1,000,000 uncached input tokens for the repository;
- 10,000 visible output tokens for the implementation and tool loop;
- the local measured thinking-token ratios for DeepSeek Flash (34%) and Pro (57%);
- a conservative 25% effort allowance for Luna, because the published GPT-5.6 table reports DeepSWE and pricing but not a directly comparable hidden-token ratio.
That gives Flash 13,400 billed output tokens, Pro 15,700, and Luna 12,500. The cost formula is simple: 1M × input price + effort-adjusted output × output price. If your repository is 1.4M tokens, scale the input portion by 1.4. If your harness reuses a prompt cache, the real bill can be materially lower.
Why a higher benchmark can be an expensive trap
A lot of independent programmers are solving this problem in different ways. Some use one model for everything. Some keep a frontier model open for architecture and hand routine changes to a cheaper agent. Some split the repository into services, keep a tight task ledger, or use tests as the interface between a planner and an implementer.
The trap is looking at a higher benchmark for an inexpensive model and deciding that you should spend for a much more powerful tool than you need. A benchmark measures capability on a task distribution. It does not tell you whether your next change needs that capability, whether the model will see a clean specification, or whether the extra reasoning will prevent more rework than it costs.
In this scenario, Pro has the highest effort burden and the most expensive output. It scores better than Flash, but the score improvement does not pay back the price difference when the whole million-token repository is sent every time. Luna is a more interesting compromise: its DeepSWE score is higher than Pro’s in the published table, while its post-cut price makes it less costly per attempt.
The workflow I would actually use
Planning and building are different jobs.
I would use a very smart model — DeepSeek V4 Pro, GPT-5.6 Sol, or Claude Opus — to reduce uncertainty first. Ask it to inspect the repository, identify the smallest safe change, name the files and invariants, and write a plan that another agent can follow. This is where a stronger model earns its keep: not by writing every line, but by making the problem smaller.
Then I would orchestrate the plan with DeepSeek V4 Flash 0731 or DeepSWE. Flash is my recommendation here because the combination of 1M context, agentic coding score, and very low price makes repeated implementation and test loops affordable. DeepSWE is also a sensible choice when its harness, tool use, or task specialization fits your workflow better.
The expensive model should clarify the work. The inexpensive model should do the work that has already been clarified.
What this table does not tell you
DeepSWE is a useful long-horizon signal, not a promise that any model will solve your repository. The benchmark’s tasks, harness, tool permissions, test quality, patch size, and retry policy all differ from your project. The cost model also assumes uncached input and one fixed output budget. Real agents may spend less, spend much more, or fail early.
For a fair decision, measure your own task mix. Take ten representative issues, run the same harness, record total billed tokens and human rework, and compare cost per accepted change — not just cost per attempt.
Capability is a budget. Spend the most of it where it shrinks the problem, then let cheaper models carry the pieces.
Browse the full benchmark series · Use the cost-per-word calculator · See how Clock Lobster builds agent workflows
Sources: DeepSWE v1.1 leaderboard, OpenAI GPT-5.6 release benchmark table, OpenAI July 30 pricing update, and the Clock Lobster benchmark catalog.