Test finds Gemini 4's top thinking tier costs ~4x for little gain
11/17/2026 — 11/17, 02:18·1 sources (all)·1 reports (all)
Key takeaways
- A tester ran 15 competition math problems twice through the Gemini API against Gemma 4 26B/31B, Gemini 3.5 Flash-Lite, 3.8 Flash and three G…
Background & analysis
On November 17, 2026, an AI article published on DEV Community presented the results of a hands-on benchmark comparing the lowest and highest thinking settings available through the Gemini API. The test covered seven models: Gemma 4 26B and 31B, Gemini 3.5 Flash-Lite, 3.8 Flash, and three versions of Gemini 4, namely Hydrogen, Carbon and Argon. Each model was asked to answer the same set of 15 competition math problems twice, once at the lowest thinking tier it accepts and once at the highest, using a step-by-step methodology documented in the GitHub repository xbill9/gemma4-dev.
The measured figures show a steep price gap between the two tiers. Total cost at the highest thinking setting came to roughly 3.9 times the lowest, at $5.95 versus $1.53. For Argon, the average number of thinking tokens per problem rose from 215 to 4129, and per-problem latency for the paid models increased by 3.2-5.9x. Yet across these problems, the extra thinking produced almost no improvement in the answers obtained. The article frames itself as a step-by-step guide to measuring what additional thinking actually buys on hard problems and what it costs to obtain it. No further developments, follow-up measurements or vendor responses have been reported since the piece was published, so the current record on this matter consists of this single set of benchmark results and its stated conclusion.
AI-generated from 1 reports · updated 10 hours ago
Related AI tools
- Heat index
- 74
- All sources
- 1
- All reports
- 1
- First seen
- 11 hours ago
Reports on this story headlines open the original
A write-up ran Gemma 4 26B/31B, Gemini 3.5 Flash-Lite, 3.8 Flash and three Gemini 4 variants through the Gemini API on 15 competition math problems, once at each model's lowest thinking setting and once at its highest. The top setting cost about 3.9x more ($5.95 vs $1.53), pushed mean thinking tokens per problem from 215 to 4,129 on Argon, and took 3.2-5.9x longer, yet bought almost nothing in accuracy on these problems.
DEV Community · AIAI score 53
Other stories people are talking about
- Heat index 94NewGoogle doubles context compression window for three Gemini 4 models1 sources
- Heat index 74Gemini 4 Hydrogen becomes the default model in the Gemini app1 sources
- Heat index 98Gemini 4 Carbon debuts at the top of the Arena overall leaderboard2 sources
- Heat index 67Developer builds an image-restyling pipeline with Gemini vision and LoRA1 sources
- Heat index 95NewDeveloper Shares Gemini API Key Rotation Solution for Next.js Cron Jobs1 sources
How is heat calculated?About the methodHide
Heat counts how many independent sources covered a story in the last 48 hours: one source counts once no matter how many posts it published, decaying with a 24-hour half-life. What ranks first is what many people are talking about.
This page aggregates public feeds. Headlines and summaries are machine-organized and remain the property of the original authors; verify important facts at the source. If you believe a headline or summary infringes your rights, email the contact address in the footer with the page URL and basis of your claim, and we will remove or replace it after review.
- Surge
- Discussion rising fast
- New
- First report within 6 hours
- Rising
- Still gathering discussion

