Homelab test: running local AI agents with 6 GB of VRAM

On October 3, 2026, a post on DEV Community described a hands-on experiment with a local AI agent running on modest homelab hardware. The author's setup hosts 41 containers alongside a GPU with just 6 GB of VRAM. When he asked the box what it was doing, it reported that it had a 27B-parameter model loaded.

The interesting part was where that model actually lived. Its total footprint was 18.3 GB, but only 0.5 GB of it sat on the graphics card. The remaining 17.8 GB stayed in system RAM and did its arithmetic on the CPU, one token at a time. The author framed the situation with a joke: the 27B model was "running on the GPU" in roughly the same sense that he is running a marathon when he walks to the shop.

The write-up focuses on scheduling and performance for local AI agent workloads on constrained hardware, particularly how resources get allocated when VRAM runs short. It does not report inference speed, latency, or throughput figures, and it does not name the specific model or any quantization scheme — only the parameter count, 27B, and the 18.3 GB total footprint. For now, the account stops at this single observation from one homelab machine, with no follow-up reported.