What you need to know
- Microsoft has unveiled two custom-designed processors for its Azure server farms, optimized for AI and computing workloads.
- There's a new Arm chip is called "Cobalt 100 CPU," and a dedicated AI processor dubbed "Maia AI Accelerator."
- The chips will roll out early next year, and will first power Microsoft's Copilot services.
Microsoft has announced that it's building two "homegrown" custom-designed processors to handle AI and computing workloads in the cloud. Unveiled at Microsoft Ignite 2023, the new compute chips are based on the Arm architecture, and will begin rolling out in Microsoft's Azure server farms in early 2024.
The Arm processor is called the "Microsoft Azure Cobalt 100 CPU," and is designed to run general purpose compute workloads on the Microsoft Cloud. Like other Arm chips, the Cobalt 100 processor delivers greater efficiency and performance in cloud-based workflows. It being more efficient is important, with Microsoft focusing on performance per watt metrics to ensure these chips run with less power.
There's also the dedicated AI chip, called "Microsoft Azure Maia AI Accelerator" which is optimized for AI tasks and generative AI processing. The company says it will power the largest internal AI workloads on Azure, and that OpenAI has helped Microsoft build out this dedicated AI chip that can handle its large language models.
Microsoft technical fellow, Brian Harry, said "[the] vertical integration – the alignment of chip design with the larger AI infrastructure designed with Microsoft’s workloads in mind – can yield huge gains in performance and efficiency."
Microsoft says designing its own chips for the cloud allows the company to better control all aspects of the Microsoft cloud and the many AI services it's building on top of it. "The chips will nestle onto custom server boards, placed within tailor-made racks that fit easily inside existing Microsoft datacenters. The hardware will work hand in hand with software – co-designed together to unlock new capabilities and opportunities."
In a blog post by Microsoft's Jake Seigal, the company said:
"The ability to build its own custom silicon allows Microsoft to target certain qualities and ensure that the chips perform optimally on its most important workloads. Its testing process includes determining how every single chip will perform under different frequency, temperature and power conditions for peak performance and, importantly, testing each chip in the same conditions and configurations that it would experience in a real-world Microsoft datacenter."
It's clear that Microsoft views building its own silicon an important venture for Azure, being able to control all aspects of the Microsoft Cloud from silicon to service. Plus, it can also work on reducing its carbon footprint by engineering more efficient chips using the Arm architecture.
Additionally, Microsoft says it's already working on a second-generation wave of both the Cobalt CPU and Maia AI Accelerator that will likely be even more powerful and energy efficient.
Azure expands with more chips
In addition to Microsoft's own silicon, the company also announced that it's expanding industry partnerships to provide more infrastructure options for customers. Microsoft says it's adding NVIDIA H200 Tensor Core GPUs to its fleet next year, and is launching a preview of new NC H100 v5 Virtual Machines built for NVIDIA H100 Tensor Core GPUs.
Additionally, Microsoft is adding AMD MI300X accelerated VMs to Azure, which are designed to accelerate the processing of AI workloads for high-range AI model training. These will feature the AMD Instinct MI300X GPU.
The company says it's important to offer a wide range of options in the Microsoft Cloud, whether it be Microsoft's own silicon or third-party hardware.
Microsoft enters the chip game
While these aren't technically the first computing chips from Microsoft (both HoloLens and Surface PCs have shipped with custom silicon before,) these are the first chips from Microsoft aimed at more general computing usage. Of course, the chips are exclusive to the Microsoft Cloud, meaning you won't see them in any end-user hardware.
With that said, it's not unlikely that Microsoft may expand its custom silicon efforts into more of its hardware down the line. While it won't be with these specific chips, it's clear that Microsoft is keen to continue investing in silicon, and if it's able to build a chip that can compete with Intel and Qualcomm, why not ship them in a Surface PC in the future?
Qualcomm just upped its game in the PC space with the launch of the Snapdragon X Elite, a chip that rivals Apple's latest M3 and M3 Pro processors. Intel is rumored to be gearing up to chip its 14th-Gen mobile chips too, which are said to be much more efficient compared to the current gen. Competition is heating up in the consumer space, but for now, Microsoft is remaining squarely in the enterprise and cloud segment.
