DEV Community evaluates out-of-bounds recognition capabilities of decision models

According to a report on DEV Community, an author wrote about how to evaluate the out-of-scope recognition capability of an AI router. The piece argues that an AI router has to recognize requests outside its supported categories while still accepting useful work, and that those are two distinct things to measure. To that end, between 23 and 28 October 2026 the author compared four decision models and five small language models. The article focuses on one task from that comparison: assigning a domain to a request, with an explicit "out of scope" option included. In the setup, the decision models received typed questions and returned probabilities over the options, while the small language models received the corresponding inputs. The tests revealed differing false-positive rates and classification biases across models when they handled the out-of-domain option. Based on those findings, the author offers guidance for building more stable routing. The available material covers the methodology and the headline conclusions only; specific scores for the individual models are not disclosed in what has been published so far.