Aider polyglot leaderboard
https://aider.chat/docs/leaderboards/
- Kind
- benchmark
- Maintainer
- Paul Gauthier (Aider)
- Feeds axes
- coding
- Measures
- Whether a model can solve hard, self-contained programming exercises in six different programming languages, and follow precise editing instructions while doing it.
- Does not measure
- Finding its way around a large existing project, real bug reports, speed, cost.
- Known issues
- The exercises are public, so models may have seen them during training; the whole benchmark is maintained by one person; results depend on his specific tool.
- Trust
- medium
- Last reviewed
- 2026-08-01
The maintainer of Aider (a popular AI coding tool) tests each model on 225 hard practice exercises across six programming languages. The model has to actually solve the problem and deliver the answer in exactly the format the tool requires — sloppy instruction- following costs points even when the underlying solution is right.
It complements SWE-bench nicely: SWE-bench is large messy real-world projects in one language; this is small hard puzzles across six languages. A model that scores well on both is genuinely good at code.
How to read a score: check the "edit format compliance" column alongside the solve rate. A model with a high solve rate but low compliance is like a brilliant employee who ignores instructions — the raw score flatters it.