Duke University researchers have proposed an accelerator called Vortex for running highly compressed language models. In a cycle-by-cycle hardware simulator, it processed the tested workloads 8.03 times faster than FIGLUT, a recent accelerator using similar lookup tables, while consuming 5.68 times less modeled energy. The September 10 preprint describes a chip design; no Vortex processor has been manufactured.
A language model first processes the user's prompt and then produces one token at a time. Vector quantization compresses its weights by replacing each small group with an index into a learned table. That reduces the data moved from memory, but many processing units may request the same table bank at once and have to wait. Token generation also repeatedly reads weights and expands the cache that holds information from earlier text.
Vortex gives prompt processing and token generation different execution paths. The prompt path handles the large matrix operations at the start of a request. During generation, the chip calculates partial results once and retrieves them through the compressed weight indices. Dedicated units perform the table lookups and compress the growing token cache. Another unit skips groups of calculations judged to have little effect while keeping memory access regular.
The researchers simulated 2-bit versions of Llama2-7B, Llama2-13B and Mistral-7B. Six reasoning and question-answering tests measured how much accuracy remained after calculations were skipped. The hardware model used a 512-token prompt and several response lengths. Its main components were described at register-transfer level and synthesized for a 28-nanometre process before performance was estimated.
Compared with FIGLUT, Vortex averaged an 8.03-fold speed increase and a 5.68-fold reduction in modeled energy across the end-to-end workloads. It could skip 25% to 30% of selected calculation groups in the two Llama models while retaining at least 95% of the corresponding compressed model's accuracy. Most of the simulated gain came from weight compression, with smaller gains from cache compression and skipped calculations.
Every performance and energy figure comes from modeled hardware. The evaluation uses older 7-billion- and 13-billion-parameter models, simulated memory traffic and synthesized components. An FPGA or fabricated chip must now run current long-context models under live requests and report response delay, total system power and accuracy. Those measurements will show whether the simulated gains survive outside the design tools.
Related coverage
Sources
- Haoxuan Shan and colleagues, Vortex, arXiv, September 10, 2026. Abstract, authorship, submission date and headline results.
- Full paper. Architecture, execution sequence, evaluation setup, accuracy results, simulated performance and limitations.
- Lead image: DARPA SyNAPSE, Wikimedia Commons, public domain. The board is illustrative and does not depict Vortex hardware.
