NVIDIA released Nemotron 3.5 Bolt 74B-A9B on October 15 as a vision-language model for local AI agents. Its 24GB memory figure, however, describes text-only serving: the technical report estimates 23.83GB for one sequence at a 128K-token context and excludes the separate vision components needed for image and video input.
Bolt is a compressed version of NVIDIA’s larger Nemotron 3.5 Super VL model. The model card lists 74 billion total parameters and 9 billion active per token; the active count describes the share used to process each token, not the full set of model weights. NVIDIA’s report says its Star Elastic method reduces the number and width of expert networks, then stores their weights at 2.0625 bits per parameter. Quantization-aware distillation is used to recover accuracy after compression. The authors describe the quantization as aggressive and lossy.
The report’s deployment estimate assumes batch size one and a 128K context. It counts weights, runtime state and a speculative decoder in the 23.83GB total. It lists the vision encoder and projector for visual input at another 1.68GB but explicitly excludes them from the 24GB target, which it describes as text-only. The report does not give a combined memory total for image or video inference under the same conditions.
The model card lists image and video inputs and a context length of up to 256K tokens. Those capabilities do not establish that a full 256K multimodal workload fits inside the report’s 24GB text-only budget. Developers considering a local deployment should treat the 24GB number as a bounded configuration claim, not as a memory figure for every advertised input or context.
Sources
Photo: Daniel Lu, NVIDIA DGX Spark photograph, CC BY-SA 4.0. Used unchanged; illustrative hardware, not a Bolt test.
