Microsoft’s AI Chip Reduces Dependence on Nvidia GPUs

Microsoft is unveiling its third-generation Maia in-house AI chip today, aimed at becoming the main AI inference system for its products and services. The Maia 300 not only enhances the cost-effectiveness and efficiency of AI workloads in the cloud but is also better equipped to manage the latest reasoning models and complex tasks.

Production for the Maia 300 is set to begin in 2027, with a strong emphasis on efficiency. “We anticipate Maia 300 will provide 40 percent more tokens per megawatt compared to leading GPU-based systems,” states Scott Guthrie, executive vice president of Microsoft’s cloud group. “In a full 96-accelerator setup, it achieves up to 7x the rack-scale token throughput of Maia 200.”

These advancements will enable Microsoft to primarily depend on its in-house AI chips rather than those from Nvidia and AMD. “We plan to scale our Azure Maia family to multi-gigawatt deployments starting in 2027, positioning Maia as the primary AI inference system for Microsoft products and services,” adds Guthrie.

Maia 300 follows the Maia 200, which was announced just this January. The Maia 200 is already operational in various production AI workloads across Microsoft, including Microsoft Copilot, GitHub Copilot, Microsoft Foundry, and the company’s own AI models.

“Maia 200 is the top-performing AI infrastructure in our fleet, based on real-world metrics for AI services,” says Guthrie. “Maia 300 is a comprehensive AI system tailored for the new types of AI workloads emerging today: long-context reasoning, latency-sensitive conversational experiences, and sustained, multi-step tasks.”

To achieve these efficiency gains, Microsoft is employing a new chiplet architecture. Built on TSMC’s N2P process node, the Maia 300 architecture separates compute, memory controller, and I/O functions. “This design increases the silicon area dedicated to compute performance and allows each function to utilize the most suitable technology for its role,” explains the Azure hardware and infrastructure team.

The Maia 300 is also integral to Microsoft’s AI model initiatives and could help the company close the gap with competitors like Google DeepMind, OpenAI, and Anthropic. “We need to work across the entire stack, all the way down to the silicon,” says Microsoft AI CEO Mustafa Suleyman. “At that scale, we require a platform that we own and co-design with our models. That’s why running our Microsoft AI models on the Maia chip is a crucial focus for our team.”