35 physicists test GPT-6 Astra with unpublished manuscripts

On October 26, 2026, a Juejin report described TheoPhysEval, a benchmark of 50 research-grade physics problems assembled by a team of 35 physicists. Every problem was built from the problem setter's own unpublished research manuscripts: the team took calculations from recent papers, changed some numerical parameters, converted final answers into high-precision floating-point values, and inserted extra parameters that do not affect the result. The authors say this prevents the answers from being matched through public problem sets or training corpora, and that skipping the derivation produces wrong numbers. The 35 problem setters are distributed across institutions including ICTS, Harvard University, KU Leuven, Google DeepMind, TIFR, the University of Tokyo, and UT Austin, and the team estimates that a top expert working without AI would need about one day to solve a typical problem independently.

According to the same report, GPT-6 Astra answered in a Codex environment and solved 49 of the 50 problems, at a cost of roughly $2.3 per problem. After a review found errors in some reference answers, the reported score was corrected from 88% to 98%. The 50 problems span 11 subfields of theoretical physics, including quantum field theory, AdS/CFT, the black hole information paradox, fluid dynamics, biomolecular processes, particle physics, and cosmology. The report calls this kind of task a turn-the-crank exercise, the parameter-changing recomputation work that used to be a first-year PhD student's introduction to research. It also notes that the team explicitly excluded frontier open problems with no known solution, so the benchmark measures computational execution within a given framework rather than whether AI can propose new physics or push past the AdS horizon entropy bound. In an offline test, GPT-6 Astra answered 46 of 50 correctly with search disabled, leaving the accuracy almost unchanged; the report reads this as evidence that the model relies mainly on what is stored in its weights rather than on search. As of now, only this one report is available, and no further developments have been published.