Microsoft announces powerful new chip for AI inference

Microsoft has announced the motorboat of its latest chip, the Maia 200, which the institution describes arsenic a silicon workhorse designed for scaling AI inference.

The 200, which follows the company’s Maia 100 released successful 2023, has been technically outfitted to tally almighty AI models astatine faster speeds and with much efficiency, the institution has said. Maia comes equipped with implicit 100 cardinal transistors, delivering implicit 10 petaflops successful 4-bit precision and astir 5 petaflops of 8-bit performance—a important summation implicit its predecessor.

Inference refers to the computing process of moving a model, successful opposition with the compute required to bid it. As AI companies mature, inference costs person go an progressively important portion of their wide operating cost, starring to renewed involvement successful ways to optimize the process.

Microsoft is hoping that the Maia 200 tin beryllium portion of that optimization, making AI businesses tally with little disruption and little powerfulness use. “In applicable terms, 1 Maia 200 node tin effortlessly tally today’s largest models, with plentifulness of headroom for adjacent bigger models successful the future,” the institution said.

Microsoft’s caller spot is besides portion of a increasing inclination of tech giants turning to self-designed chips arsenic a mode to lessen their dependence connected NVIDIA, whose cutting-edge GPUs have go progressively pivotal to AI companies’ success. Google, for instance, has its TPU, the tensor processing units—which aren’t sold arsenic chips but as compute powerfulness made accessible done its cloud. Then there’s Amazon Trainium, the e-commerce giant’s ain AI accelerator chip, which just launched its latest version, the Trainium3, successful December. In each case, the TPUs tin beryllium utilized to offload immoderate of the compute that would different beryllium assigned to NVIDIA GPUs, lessening the wide hardware cost.

With Maia, Microsoft is positioning itself to vie with those alternatives. In its property merchandise Monday, the institution noted that Maia delivers 3x the FP4 show of 3rd procreation Amazon Trainium chips, and FP8 show supra Google’s seventh procreation TPU.

Microsoft says that Maia is already hard astatine enactment fueling the company’s AI models from its Superintelligence team. It has besides been supporting the operations of Copilot, its chatbot. As of Monday, the institution said it has invited a assortment of parties — including developers, academics, and frontier AI labs — to usage its Maia 200 bundle improvement kit successful their workloads.