Summary
Microsoft says its Maia AI accelerator system will expand to multi-gigawatt deployments from 2027, with Maia 300 scheduled to begin production deployments that year. The company expects Maia 300 to improve AI inference efficiency and rack-scale throughput.
Microsoft plans to expand its Azure Maia AI accelerator system to multi-gigawatt deployments from 2027, as it builds its own infrastructure for running AI services. The company says Maia 200 is already serving production workloads, including Microsoft Copilot, while the next generation, Maia 300, is due to begin production deployments in 2027.
Microsoft describes Maia 300 as an inference-focused system. Inference is the process of running a trained model to generate outputs, such as responses to prompts. The company says the system is designed for long-context reasoning, conversational services that need low latency, and multi-step tasks handled by AI agents.
Maia 300 combines accelerators, memory and networking
Rather than treating the accelerator as a standalone chip, Microsoft presents Maia 300 as a system spanning compute, memory and networking. Its design uses a software-defined dataflow architecture, intended to coordinate how data moves between those resources. Microsoft says this approach, together with AI-assisted kernel optimisation, is meant to reduce unnecessary data movement and improve how fully the hardware is used.
That focus reflects a constraint in large AI systems: moving data can limit performance even when substantial computing capacity is available. Maia 300 uses eight HBM4 memory stacks, with a stated peak memory bandwidth of 18.0 terabytes per second and a 950-watt thermal design power. Microsoft also says specialised software can use on-die SRAM as a scratchpad, reducing demands on the higher-capacity HBM memory.
The accelerator has a multi-chiplet design, separating compute, memory-controller and input/output functions. Microsoft says its compute chiplets use TSMC’s N2P process node. The system also supports low-precision formats, including enhanced MXFP4, which the company says can reduce memory and bandwidth needs for computation and for the KV cache—the stored information a model uses to keep track of a conversation or other context.
Microsoft’s projected efficiency and rack-scale performance
Microsoft expects Maia 300 to deliver 40% more tokens per megawatt than leading GPU-based systems. Tokens are the pieces of text a language model processes and generates, so this measure relates AI output to the power used. The figure is a company projection for the system, rather than a result from a published independent comparison.
In a full configuration of 96 accelerators, Microsoft says Maia 300 can provide up to seven times the rack-scale token throughput of Maia 200. The rack is specified at 167 kilowatts, with 27.6 terabytes of HBM capacity and 1.7 petabytes per second of aggregate HBM bandwidth across its high-speed scale-up domain. The company says this arrangement is intended to serve more simultaneous interactions and larger workloads from each rack.
Maia 200 provides a current operating point for Microsoft’s programme: it is serving production workloads and is being deployed across the United States, Europe and Australia. Microsoft reports up to 1.7 times more tokens per watt for built-in AI capabilities in Teams, Copilot and Cowork compared with leading GPU infrastructure. The company says experience running its services on Maia 200 informed Maia 300’s design.
The planned expansion gives Microsoft a route to run more of its own AI workloads on a system it designs around those services. Maia 300’s production deployment is planned for 2027; the announcement does not specify a more precise start date.