Microsoft on Oct. 12 detailed Maia 300, its next AI inference system, and said production deployments are planned for 2027. The company says its earlier Maia 200 is already serving production workloads including Copilot.
Inference is the step in which a trained AI model generates output. Microsoft designed Maia 300 for memory-intensive, long-context and multi-step work, and describes the rack as one system of accelerators, memory, networking and cooling. Each rack holds 96 accelerators across 24 four-accelerator trays within a 167-kilowatt power envelope.
Each accelerator has eight stacks of HBM4, or high-bandwidth memory, with 288 gigabytes of capacity and peak bandwidth of 18 terabytes per second. The design uses non-uniform memory access, a layout that lets software place data near the computing region that uses it. Microsoft says this arrangement reduces data movement, which is a central design goal for long-context inference.
The network can extend from a rack to a group of up to 12,288 accelerators across 128 racks, which Microsoft calls a superpod. An Ethernet-based transport is designed to carry communication across the system so jobs can use accelerators in multiple racks.
Microsoft says a full 96-accelerator Maia 300 configuration can deliver up to seven times Maia 200’s rack-scale token throughput. It separately projects up to 40% more tokens per megawatt than leading GPU-based systems. These are company comparisons with different baselines; the two October 12 posts do not provide complete workload conditions or independent test results for the headline figures.
Sources
- Microsoft Official Blog, Maia 200 production status and Maia 300 plans, October 12, 2026
- Microsoft Azure Blog, Maia 300 architecture and rack specifications, October 12, 2026
- Carl Lender, data-center server racks, Wikimedia Commons
.jpg){kind=link}
Image: Carl Lender via Wikimedia Commons, CC BY 2.0. Illustrative only; it does not show Maia hardware or a Microsoft facility.