A Sept. 23 arXiv preprint introduces MicroQonv, a memory layout that lets convolutional neural networks quantize each activation once and reuse the low-precision value in both the forward and backward passes of training. The authors report test accuracy close to full-precision baselines, while DRAMsim3 simulations estimate lower YOLO activation traffic.
Im2col can send one activation into several quantization blocks
An activation is an intermediate value passed from one network layer to the next. During training, the forward pass computes layer outputs; the backward pass propagates gradients to adjust the weights. Convolution applies a small filter across overlapping image patches. To run those operations as fast matrix multiplication, many systems rearrange patches into a larger matrix, a step called im2col.
That transformation repeats values shared by neighboring patches. Microscaling quantizes numbers in small blocks that share a scale factor. When quantization follows im2col, copies of the same activation may fall into different blocks and receive different low-precision values. Reusing one copy for the backward pass is therefore not straightforward.
MicroQonv reuses quantized activations in both training passes
The method uses a channel-batch-first layout that arranges activation channels and batch positions so data blocks can be read contiguously during both the forward and backward calculations. It quantizes the original activation tensor before forming the expanded im2col view; patch data is assembled as needed rather than saved as a second off-chip matrix. The tests use 8-by-8 microscaling blocks containing 64 values.
In five-run accuracy tests, 4-bit NVFP4 MicroQonv kept ResNet-32 on CIFAR-100 and ResNet-18 on the first 100 ImageNet classes within 0.6 percentage points of their 32-bit baselines. The first and last layers remained in full precision.
Simulations estimate lower YOLO activation traffic
The YOLO memory simulations used 8-bit NVFP8 rather than 4-bit NVFP4, with selected sensitive layers at higher precision. The researchers modeled transfers between off-chip DRAM and on-chip SRAM using DRAMsim3's LPDDR4 setting. Against their adaptation of an earlier quantized im2col method, DRAMsim3 estimated 3.5 times less activation traffic for YOLOv8 Nano on Pascal VOC and 2.2 times less for YOLOv26 Nano on KITTI.
The simulation does not include the cost of quantization, and the paper describes building an accelerator as future work. The results therefore do not establish end-to-end speed or energy savings on real hardware.
Sources
- MicroQonv preprint (arXiv)
- DRAMsim3 memory simulator
- Photo by Christian Wiediger, depicting an Nvidia RTX 2060; used under the Unsplash License as an illustration, not as hardware tested in the study. Image source.