Why fine-tuning a 7B model needs 112 GB of VRAM

A post published on DEV Community on October 3, 2026, works through the memory math of fine-tuning a 7B model. The article says that when people are asked how much VRAM this takes, the instinct is to answer that the model itself is 14 GB in fp16, so a little more than that should be enough. The real figure it gives is about 112 GB. Within that total, the model weights account for just 14 GB, while roughly 98 GB goes to gradients, optimizer states and activations. The post attributes this accounting to the ZeRO paper. Its argument is that once you see where those other 98 GB go, LoRA and QLoRA stop looking like clever tricks and start looking like the obvious choice. The excerpt opens by describing the 112 GB as the number you face before storing a single activation, then lists activations among the components of the remaining 98 GB, a small tension in how the breakdown is framed. The piece presents itself as an accounting exercise rather than a benchmark: the headline number is not the size of the model. So far no other report has offered different figures or added anything to the picture.