Microsoft broadcasts highly effective new chip for AI inference

Microsoft has introduced the launch of its newest chip, the Maia 200, which the corporate describes as a silicon workhorse designed for scaling AI inference.

The 200, which follows the corporate’s Maia 100 launched in 2023, has been technically outfitted to run highly effective AI fashions at quicker speeds and with extra effectivity, the corporate has mentioned. Maia comes outfitted with over 100 billion transistors, delivering over 10 petaflops in 4-bit precision and roughly 5 petaflops of 8-bit efficiency—a considerable enhance over its predecessor.

Inference refers back to the computing strategy of working a mannequin, in distinction with the compute required to coach it. As AI corporations mature, inference prices have grow to be an more and more necessary a part of their total working value, resulting in renewed curiosity in methods to optimize the method.

Microsoft is hoping that the Maia 200 will be a part of that optimization, making AI companies run with much less disruption and decrease energy use. “In sensible phrases, one Maia 200 node can effortlessly run immediately’s largest fashions, with loads of headroom for even larger fashions sooner or later,” the corporate mentioned.

Microsoft’s new chip can also be a part of a rising development of tech giants turning to self-designed chips as a technique to reduce their dependence on NVIDIA, whose cutting-edge GPUs have grow to be more and more pivotal to AI corporations’ success. Google, as an example, has its TPU, the tensor processing items—which aren’t offered as chips however as compute energy made accessible by means of its cloud. Then there’s Amazon Trainium, the e-commerce large’s personal AI accelerator chip, which simply launched its newest model, the Trainium3, in December. In every case, the TPUs can be utilized to dump a few of the compute that will in any other case be assigned to NVIDIA GPUs, lessening the general {hardware} value.

With Maia, Microsoft is positioning itself to compete with these alternate options. In its press launch Monday, the corporate famous that Maia delivers 3x the FP4 efficiency of third technology Amazon Trainium chips, and FP8 efficiency above Google’s seventh technology TPU.

Microsoft says that Maia is already onerous at work fueling the corporate’s AI fashions from its Superintelligence workforce. It has additionally been supporting the operations of Copilot, its chatbot. As of Monday, the corporate mentioned it has invited quite a lot of events — together with builders, lecturers, and frontier AI labs — to make use of its Maia 200 software program growth equipment of their workloads.