Coding agents · 2 benchmarks · 25 languages · 1,878 zstd runs
Verifying claims on what language is best for coding agents. Recomputed from the per-run data behind danluu.com/pl-tokens.
The two groups differ by less than one test case out of 34. Among mainstream languages the gap is 0.04. At high effort, 19 of the 25 languages are too close to call. Switching won’t buy you anything.
Running the cheap model eight times over lost to one expensive run in all 25 languages. Effort is the biggest lever you have. It buys 3.6 more cases and costs 4.2 times as much. If you do loop, keep the context. Clearing it lost 21 times out of 25.
Run the cheap model three times and keep the best result. That gets you 30.5 cases out of 34 for about 70% of what one expensive run costs. Two catches: you need tests to tell which run won, and the expensive run still scores higher in 18 of the 25 languages.
The bottom is the same on both benchmarks. So is part of the top: Python, JavaScript and C# do well on each. The middle is noise. C++ comes 2nd on zstd and 17th on Pandoc. Rust comes 13th and 2nd. Avoid the floor, but don’t pick a winner off a leaderboard.
| Claim you have probably seen | Verdict | What these runs show | Source |
|---|---|---|---|
| Dynamic languages are cheaper to run | Not supported | Across all 25 languages the dynamic group cost slightly more per run ($5.07 vs $4.63) and scored within one case of the static group. | Google AI summary |
| Clojure needs 2.6× fewer tokens than C | Measured something else | That counted tokens in short Rosetta Code puzzles. Asked to build a real zstd decoder, C cost less ($3.80 vs $5.26 a run) and got more right (29.3 vs 23.0 of 34 cases). | Alderson |
| Array languages like J are the most token-efficient | Measured something else | Same puzzle-sized measurement. On zstd, J came last of 25 for correctness and second-costliest per run. | Alderson follow-up |
| Languages with lots of bad training code, like PHP, do worse | Not supported | PHP came 3rd of 25 on Pandoc and sat mid-pack on zstd. | folklore |
| Pick a powerful language like Haskell, since rewriting is cheap now | Not supported | Haskell came 5th of 25 on zstd but only 14th on Pandoc — no better than average overall. | folklore |
| Stick to a popular, mainstream language | Weak support | Only assembly clearly suffers: the two assembly targets passed 11% of Pandoc's tests, against 19–31% for the other 23 languages. | folklore |
Does doing well on one benchmark predict doing well on the other? And what patterns show up in the costs and failures across languages?
Every number here comes from the per-run data behind danluu.com/pl-tokens, run on GPT-5.6 Sol. That is 938 medium and 940 ultra zstd runs across 25 languages, plus Pandoc runs scored on tests the agent never saw. Both benchmarks are shown as percentages so they can share an axis, but they are not equally hard. On zstd a language passes 85–95% of the 34 cases. On Pandoc the best of the 25 passes 30.7%. Cost and wall-clock are the source’s own per-run averages. TypeScript and x86-64 asm have no published Pandoc wall-clock, so their time comparison reads “n/a”. The verdicts above are this dashboard’s reading of two benchmarks. Two benchmarks cannot settle a claim about programming in general.