NVIDIA blog details three ways to optimize MoE inference on DGX Spark

On October 27, 2026, NVIDIA Developer Blog published a post analyzing inference performance of Inclusion AI's Ling-3.0-flash on NVIDIA DGX Spark and outlining three optimization methods. According to the post, generating tokens faster begins with identifying what limits inference performance. In a batch-size-1 evaluation of Ling-3.0-flash from Inclusion AI on NVIDIA DGX Spark, profiling showed that decoding was memory-bound: reading model weights drove execution time. Two operations stood out, namely the mixture-of-experts (MoE) computation and the output-layer projection that produces scores across the model's vocabulary. Those findings pointed to two ways to improve performance: reduce the weight traffic in each decode step, and generate more tokens per verification round. The post examines three techniques that address those costs, noting that such techniques can effectively speed up inference for mixture-of-experts models. The write-up remains at the level of this analysis and the proposed techniques, without covering any subsequent adoption or results.