Experiment shows major AI models agree on facts but diverge on judgmental questions

On November 7, 2026, DEV Community's AI channel published the results of a small experiment comparing five mid-tier mainstream models — GPT, Claude, Gemini, Grok and Llama (the mid-tier model from each family). The test set consisted of 24 questions, six each on pricing and strategy, factual questions with a trap, coding approach, and recommendations with tradeoffs. Each question was sent to all five models unchanged. Rather than judging style, the authors compared where the models' answers diverged in substance. On 8 of the 24 questions (33%), the five models materially agreed with one another. On the remaining 16 (67%), at least one material divergence appeared, for a total of 52 divergences. On that basis, the experiment concludes that these models tend to converge on factual recall but split when the task calls for judgment, such as scoring tradeoffs, recommending between options, or deciding which facts are misleading. The findings are limited to nil evidence