Replit releases ViBench 2.0 benchmark and model build results

On October 29, 2026, Replit released ViBench 2.0, a benchmark for long-horizon software building. Agents start from an empty project and build a whole product one feature at a time, without seeing what comes next. The benchmark is designed to test whether models can sustain work across an entire product, not just individual tasks.

Replit then published its build evaluation results: three models passed more than 40% of their builds, while no other model reached 20%. GPT-6.1 Sol led with a 65% pass rate, edging out GPT-6 Astra at 60%, at roughly $12 per build versus about $90. Opus 5.5 passed 49% and finished a build in around 70 minutes, about 2.5x faster than Sol, mostly due to API provider speed. Replit also reported that newer models score higher: on the 8 public products, pass@1 rose from 0% for GPT-5.4 to 31% for GPT-6 Sol and 53% for GPT-6.1 Sol. GPT-6.1 Terra passed 47% at about half Sol's cost.

On failure patterns, Replit noted that every feature works on its own but breaks when they come together. Around three quarters of the top models' failed builds passed every single-feature check, and the failures occurred where features meet. Examples included a code hosting draft PR that never showed its merge conflict, and a document editor in which nobody could comment on a word inside the document. The event currently stands at Replit's publication of the benchmark, its evaluation results, and its failure analysis, with no third-party reproduction or independent verification reported yet.