Study finds flaws in agent benchmark Terminal-Bench and reports score increases after fixes

According to a Juejin report published on November 7, 2026, researchers at Stanford University's Ré Lab found that roughly 30 percent of tasks in the Agent benchmark Terminal-Bench contained defects. The report describes the team's approach as engineering the problem of benchmark decay, meaning that after a benchmark is released, models keep improving, and task defects therefore need to be systematically identified and fixed rather than ignored. Once the defects were repaired, the tested models' overall scores commonly rose by 10 percentage points, the report says. The work was carried out and announced by Ré Lab at Stanford University. The report does not provide further detail on the full list of affected tasks or on the revised results for individual models beyond the reported score gains.