DeepSeek open-sources DeepInfer large-scale inference framework

On October 20, 2026, DeepSeek open-sourced DeepInfer, its inference framework. According to a report by 开源中国 (OSChina), the release is not a model but the middleware layer that handles compute scheduling and cost control: when a service receives a large volume of calls, the inference framework coordinates the compute cluster, and its efficiency directly determines the compute cost of AI services.

A report from 智东西 (Zhidx) the same day described the release as DeepSeek's in-house inference framework for large-scale cloud services, together with deepseek-recipe, a production-grade token codec component. DeepInfer combines high-performance operators with a heavily optimized inference engine and supports distributed inference, speculative decoding, SWA Bounded Replay, and structured output, and it can run on both NVIDIA and Huawei Ascend platforms. According to figures released by the DeepSeek team, the high-throughput, low-latency large-scale inference service for DeepSeek V4.1 Flash is built on this framework; a single cluster currently serves 155 trillion Tokens per day, and overall revenue, calculated at valley pricing, is $38 per card per hour (about CNY 253.81). deepseek-recipe handles encoding and decoding between mainstream LLM API protocols and model Tokens, with processing logic fully aligned with DeepSeek's official API, so third-party deployments can obtain results consistent with the official deployment.

A breakdown published by 掘金 (Juejin) the same day, based on the official repository and the deployment Q&A, mentioned Wide-EP, long-term KVCache, SWA Bounded Replay, and dynamic speculative decoding, and explained how KVCache is kept for three days and how speculative decoding reduces verification by 30%. The three reports differ in emphasis: OSChina framed DeepInfer as middleware for compute scheduling and cost control, Zhidx added the deepseek-recipe component along with throughput and revenue figures, and Juejin focused on the technical mechanisms. Public information currently stops at the point where the framework and component have been open-sourced and the official repository and deployment Q&A are available for review.