Benchmark Comparison Table
Switch views to focus on popular, use-case, or experimental results. Models are rows; benchmarks are columns. Click any column header to sort.
New benchmark — Published 2026-08-23
Prompt Caching at 3 Billion Tokens: What It Saves
DeepSeek V4 Flash tested with ordinary prompts versus a stable skill prefix. The measured cache pattern saves about $30/month at 3B input tokens; a mostly warm workflow could save about $124/month.
Read the post →
Interactive tool
LLM Cost-per-Word Calculator →
Paste your prompt, pick from 31 models, see the real per-word cost. Uses the measured tokenizer-efficiency (E) values from the benchmark below — not the marketing price sheet.
Benchmark — Published 2026-07-31
DeepSWE Points per Dollar: Six Coding Models Ranked by Output Value
We combine DeepSWE long-horizon coding scores with output-token pricing to see which model delivers the most DeepSWE points for every output dollar. MiMo v2.5 Max leads at 236 points per $.
Read the post →
Session 5 — Published 2026-07-09
Tokenizer Efficiency: Why Input Token Prices Are Meaningless Without It
50 model variants tested on code, prose, and blended text. The gap between the most and least efficient tokenizer is 74% — meaning a model with cheaper per-token pricing can cost more per word. Full methodology, charts, and CSV dataset included.
Read the post →
Session 6 — Published 2026-07-09
The 16 Challenges: Why Output Token Prices Hide Thinking Tokens and Efficiency
50 model variants across 16 task types in 9 categories — Q&A, reasoning, coding, creative, role-play, instruction-following, safety, multilingual, and extraction. Why category matters, how thinking tokens inflate cost, and why you can't compare output token prices across models without adjusting for both.
Read the post →
Session 6b — Published 2026-07-09
5 Ways to Compress LLM Output: A Cost-Saving Toolkit for Enterprise
Structured Markdown Compression, JSON Envelope, Diff-Only Delta, Verb-Noun Grammar, and Word Deletion tested across 20 model configs spanning 18 families. Word Deletion cuts output by 49%; Diff-Only Delta by 53%; one method backfires with a negative ratio. Full results, charts, and implementation guidance.
Read the post →