HaiAI123

Curated Global AI Tools Directory

Industry update
25
heat index
Flat

Open-source tool finds Agents invent tickets from empty API responses

11/14/2026 — 11/14, 03:17·1 sources (all)·1 reports (all)

Key takeaways

  • The author open-sourced reported-or-invented, an MIT-licensed test harness that uses a local mock API to probe gpt-oss-20b on Groq and Gemin…

Background & analysis

On 2026-11-14 at 03:03, the AI section of DEV Community published an article reporting on a new test harness. The author open sourced a tool called reported-or-invented, released under the MIT license, which uses a local mock API to measure how gpt-oss-20b (running on Groq) and Gemini 3.5 Flash-Lite behave when a tool call fails quietly, meaning the response comes back with status 200 but the body is empty, {}, or every field is null. The test fixture is a fictional customer, C-1001, covering orders, support tickets, and checkout-service health, and everything runs locally so each run can be reproduced exactly. The agent side is built with crewAI and is given tools to call.

The results show that on real data both models answered correctly on both attempts. When a failure is loud, such as an HTTP 500 or an error field, most runs stated the problem, at 16/18 and 15/18 respectively. The divergence appears under silent failure: with an empty or all-null body returned alongside status 200, both models treated the empty result as fact, recounted it as if it were genuine, and in turn produced support tickets that did not actually exist. As of now the matter rests at this single test and the released open source tool, with no further developments or responses from the vendors involved.

AI-generated from 1 reports · updated 3 days ago

updream
24-hour heatHeat index 25
Just charted — trend is building up
24 hours agonow

Reports on this story headlines open the original

Nov 14
  1. The author open-sourced a MIT-licensed harness called reported-or-invented that uses a local mock API to test how agents handle status 200 responses with an empty body, {} or all-null fields. Both gpt-oss-20b on Groq and Gemini 3.5 Flash-Lite reported signalled failures most of the time (16 of 18 and 15 of 18 runs), but under silent failure both passed empty results on as fact.

    DEV Community · AIAI score 57

Other stories people are talking about

How is heat calculated?About the method

Heat counts how many independent sources covered a story in the last 48 hours: one source counts once no matter how many posts it published, decaying with a 24-hour half-life. What ranks first is what many people are talking about.

This page aggregates public feeds. Headlines and summaries are machine-organized and remain the property of the original authors; verify important facts at the source. If you believe a headline or summary infringes your rights, email the contact address in the footer with the page URL and basis of your claim, and we will remove or replace it after review.

Surge
Discussion rising fast
New
First report within 6 hours
Rising
Still gathering discussion

Back to AI industry updates →