llm-d 1.0 Released with Cache-Affinity Routing and Breakpoint Math
11/02/2026 — 11/02, 22:16·1 sources (all)·1 reports (all)
Key takeaways
- LLM-D 1.0, governed by CNCF, has been officially released with support for model servers such as vLLM and SGLang. Its core cache-affinity ro…
Background & analysis
llm-d 1.0 was released on 2026-10-20, with the project governed under the CNCF. According to a report on DEV Community dated 2026-11-02, the supported model servers are vLLM and SGLang.
The report notes that among the listed features, the one that most affects the GPU bill is the default routing rule, described in llm-d's August post "Sticky Until Saturated": send a request to the endpoint that already holds its prefix cache, until that endpoint's queued token load exceeds a limit, after which routing falls back to load alone. The limit is arithmetic, not a matter of taste: tau = R_peak x T_max.
The capabilities the report lists include prefix-cache and load-aware routing, KV cache offloading, prefill/decode disaggregation, wide expert parallelism, batch and async inference, multi-tenant quality of service, router high availability with hot standby, and session-aware routing for agents. At this point, the llm-d 1.0 release stands at that published feature list.
AI-generated from 1 reports · updated 3 days ago
- Heat index
- 25
- All sources
- 1
- All reports
- 1
- First seen
- 3 days ago
Reports on this story headlines open the original
CNCF-governed llm-d 1.0 has been announced, supporting vLLM and SGLang model servers. Its core cache-affinity routing rule uses a mathematical limit to balance prefix cache reuse and queue load, helping to reduce GPU inference costs for large models.
DEV Community · AIAI score 59
Other stories people are talking about
- Heat index 840OpenAI ships Images 2.6 with GIF and SVG output15 sources
- Heat index 675OpenAI says its cyber model pipeline confirmed 1,724 Linux kernel flaws and generated 948 privilege-escalation exploits11 sources
- Heat index 642RisingOpenAI GPT-5.6 Brings Major Agentic and Computer Use Upgrades13 sources
- Heat index 633RisingCat Wu, product lead for Claude Code, joins OpenAI9 sources
- Heat index 630NewSurgeOpenAI releases GPT-6 Cyber cybersecurity model7 sources
How is heat calculated?About the methodHide
Heat counts how many independent sources covered a story in the last 48 hours: one source counts once no matter how many posts it published, decaying with a 24-hour half-life. What ranks first is what many people are talking about.
This page aggregates public feeds. Headlines and summaries are machine-organized and remain the property of the original authors; verify important facts at the source. If you believe a headline or summary infringes your rights, email the contact address in the footer with the page URL and basis of your claim, and we will remove or replace it after review.
- Surge
- Discussion rising fast
- New
- First report within 6 hours
- Rising
- Still gathering discussion