Microsoft’s chief technology officer Kevin Scott said on Wednesday that the company is having an easier time getting access to Nvidia’s chips that run artificial intelligence workloads than it was a few months ago. Speaking at the Code Conference in Dana Point, California, Scott said the market for Nvidia’s graphics processing units (GPUs) is opening up a little. The GPUs have been heavily in demand since Microsoft-backed OpenAI launched the ChatGPT chatbot late last year.
Nvidia GPUs power AI applications and data centers
Nvidia’s GPUs are widely used for training and deploying artificial intelligence models, especially those that generate text, images, audio, or video from data. These models, known as generative AI, have been attracting a lot of attention and innovation in the past year. OpenAI’s ChatGPT, which runs in Microsoft’s Azure cloud, is one of the most popular examples of generative AI. It can generate realistic and coherent conversations on various topics, based on a large corpus of text data.
However, generative AI models are also very computationally intensive and require a lot of GPU power to run. This has created a huge demand for Nvidia’s GPUs, which are considered the best in the market for AI applications. Nvidia has been struggling to meet the demand, as it also faces competition from other sectors, such as gaming, cryptocurrency mining, and automotive. Nvidia said last month that it expects revenue growth this quarter of 170% from a year earlier. The company has such control of the AI chip market that its gross margin shot up from 44% to 70% in a year. Nvidia’s stock price is up 190% in 2023, far outpacing every other member of the S&P 500.
Generative AI hype cools down, easing GPU pressure
Scott said that one of his responsibilities at Microsoft is controlling the GPU budget across the company. He called it “a terrible job” that’s been “miserable for five years now” in an interview with Patel that was published in May. He said that demand for GPU capacity was far exceeding the supply that the whole ecosystem could produce.
However, he said that the situation is improving now, as generative AI technologies are seeing some slowdown in popularity and usage. He said that traffic to ChatGPT has declined month over month for three consecutive months, according to Similarweb. This makes it easier for Microsoft to access GPUs for its own products and services, as well as for its cloud customers.
“It’s easier now than when we talked last time,” Scott said on Wednesday. “That is resolving. It’s still tight, but it’s getting better every week, and we’ve got more good news ahead of us than bad on that front, which is great.” He said that the increased supply of Nvidia chips “makes my job of adjudicating these very gnarly conflicts less terrible.”
Microsoft invests in silicon innovation and diversification
Scott also said that Microsoft has been investing in silicon innovation and diversification, as it seeks to reduce its dependence on Nvidia and other chip suppliers. He said that Microsoft has been developing its own chips for various purposes, such as security, networking, machine learning, and quantum computing. He also said that Microsoft is working with other chip makers, such as Intel and AMD , to explore new architectures and technologies for AI applications.
However, he admitted that Nvidia’s chips have been the best available option for generative AI workloads in the past few years. He said that Microsoft is planning to start selling access to its Microsoft 365 Copilot service to large organizations with subscriptions to its productivity software in November. Copilot is a generative AI tool that can help users write code, documents, emails, and presentations by suggesting relevant content and formatting.
Scott said that Copilot is powered by Nvidia’s GPUs, and that he expects the demand for such tools to grow in the future. He said that generative AI is still a very exciting and promising field of research and innovation, and that Microsoft is committed to advancing it with its own investments and partnerships.
