A new experimental serving runtime called JustFit allowed a 27-billion-parameter language model to process 212,992 token positions on a 24 GiB Apple M4 Pro MacBook. The run combined 196,608 input tokens with 16,384 generated tokens. Under the same 21,000 MiB process-memory guard, the upstream software completed 30,720 positions in its largest successful test, giving JustFit 6.93 times the demonstrated single-request context capacity. Yuhua Chen reported the result in a single-author preprint submitted on September 15. The study addresses a practical limit on long local AI sessions: the model can fit in memory while the state created during generation cannot.
Compressing model weights to four bits helps the 27B model fit on the laptop. Long conversations create another growing load. Each attention layer retains key-value, or KV, data that represents earlier tokens, and temporary workspaces can add further demand. In the baseline test, the upstream MLX-VLM software finished a 24,576-token prompt followed by 6,144 generated tokens. It crossed the memory guard when the prompt increased to 32,768 tokens with the same output length. The paper therefore treats runtime state, rather than model weights alone, as the remaining capacity problem.
JustFit manages that state in three stages. KVExec stores the attention history in four-bit TQ4 pages, then reconstructs only the pages needed by the layer currently running. This prevents full-precision caches and conversion workspaces from remaining in memory across the model. PhaseSwap unloads components used only during a particular phase, including the language-model output head, after active requests release them. StateTrans then controls page ownership as the server changes between one speculative request and several concurrent requests, preserving the target model's state instead of rebuilding it. At 229,376 stored positions, the paper reports 3,640 MiB for the TQ4 KV payload, compared with 14,336 MiB for FP16 storage.
The headline capacity result came from a deliberately demanding memory test. It used repetitive input text, greedy decoding, suppressed end-of-sequence stopping, one active request lane and a 229,376-position page pool. Each of the three trials started in a fresh process without a reusable prefix cache. These controls force the system to allocate and exercise a long context, but they do not show how well the model performs on a large software repository, a long book or another practical task. The completed-position count should therefore be read as a serving-capacity measurement.
All three longest trials completed, with peak process memory between 20,960 and 20,977 MiB and median decoding speed of 4.985 tokens per second. Other figures in the paper come from different operating points. A short-output probe with a 32,768-token prompt and 64 generated tokens reached 19.11 tokens per second at 15,899 MiB. A separate two-request test retained 229,376 positions in aggregate and produced 12.55 tokens per second across both requests. The higher rates do not describe the 212,992-position run; output length, concurrency and retained state differed.
An integrated check on 30 AIME 2026 mathematics problems produced 29 correct answers and 696,834 output tokens, suggesting that the full pipeline can sustain lengthy reasoning. It was not an isolated test of the cache format. The paper has not been peer reviewed, covers one laptop and one model, combines several memory changes without a matched ablation, and reports a smallest guard margin of only 23 MiB. Some measurements also span code revisions whose historical metadata is incomplete. The public implementation makes independent testing possible. A fixed release tested on real long documents and coding workloads would show whether the additional capacity remains stable outside the paper's stress protocol.
Related coverage
Sources
- JustFit preprint record, arXiv, submitted September 15, 2026.
- Full JustFit paper, including methods, test protocols and limitations.
- Public implementation branch, GitHub.
- Lead image: KKPCW, Wikimedia Commons, CC BY-SA 4.0. The photograph is illustrative and shows a 2020 M1 MacBook Air, not the M4 Pro machine used in the reported tests.
_-_3.jpg){kind=link}