SWE-bench Bash Only (500 instances)
- Kind
- benchmark
- Maintainer
- TODO
- Measures
- TODO — adopted on autopilot; to be filled after a first review.
- Does not measure
- TODO
- Known issues
- Adopted on autopilot from a pipeline proposal; kind, axes, and description not yet reviewed.
- Trust
- low
- Last reviewed
- 2026-10-07
This run's SWE-bench fetch confirms Bash Only is now the default Verified view: the same 500 problems, every model run inside one shared agent environment instead of each lab's own scaffolding. Scaffold variance is the named honesty gap in the coding method we already track, so this changes how the coding axis is read, not just what feeds it.