Rust inference engine cortiq 0.8.67 released with native CUDA backend

On 2026-11-11 at 10:49, developer Oleg Kirichenko (community handle: infosave) released cortiq 0.8.67, a Rust inference engine, adding a native CUDA backend. The release supports running Spark-X2.6-4B from a single CMF file, and Spark is the first model the backend runs. According to the author's measurements on an NVIDIA A40, after a 4000-token prompt, the same 4-bit file generated at 111.5 tokens/s on CUDA versus 80.2 tokens/s on the other backend. That is roughly a 39% throughput gain on CUDA in this configuration. The gap is expected to widen on longer prompts, since each new token then has more history to look through. The post notes that cortiq has been working on exactly this problem: making a small model fast on long contexts. To date, however, this is the only public report on the release, so verification on other hardware remains open.