Adding a circuit breaker to LLM calls with 60 lines and three thresholds

According to a single Juejin post, the author recounts an incident from one winter evening last year: their writing assistant hung for 62 seconds, during which users could only watch a blinking cursor after pressing send, and after roughly a minute an error appeared reading "generation failed, please try again". At the backend, a 5xx error spike lasted 11 minutes.

The author stresses that the failure was not a crash but a stalled, looping request. In response, they describe a circuit breaker for LLM calls built with around 60 lines of code, centered on three thresholds, and claim the approach can be ported directly to other LLM call pipelines. At this point, no additional reporting is available: the three threshold values, their configuration, and the complete implementation have not been made public.