How Microsoft can become Generative AI’s biggest winner

Since the release of ChatGPT in November, there has been much speculation about the possible cutthroat application of Advanced Large Language Models (LLM). There were reports a while ago that Microsoft will integrate ChatGPT into its Bing search engine to get ahead of Google. There are also many discussions about something like ChatGPT replacing search altogether.

While I’m not sold on either idea, I think we’re just beginning to explore the huge business potential of LLMs and other generative artificial intelligence technologies.

And Microsoft has a chance to be the big winner of this new wave of innovation that is about to be unleashed. Azure OpenAI Service, now generally available, could be Microsoft’s winning card in the race to dominate the burgeoning generative AI market.

Azure OpenAI service vs OpenAI API

Azure OpenAI Service launched in November 2021, but was only available through a sales model. Now anyone can sign up and access the service if they meet Microsoft’s Responsible AI Principles.

Currently, Azure OpenAI Service supports basic and fine-tuned GPT-3 models, basic and fine-tuned Codex series, and LLM embeddings. Microsoft also added DALL-E 2 to OpenAI Service in October, although it is still not available as part of the public product. According to the Microsoft blog, the company is adding support for ChatGPT soon.

Azure OpenAI Service is basically a copy of OpenAI API, although it has several advantages. For Microsoft customers already using Microsoft’s cloud, it will be much easier to access OpenAI’s technology through Azure. Since many companies already use Microsoft’s machine learning and devops products, it becomes much easier for them to manage their GPT-3 and Codex instances on the Azure platform.

Azure also provides enterprise-level security features required in many industries. And it supports features such as choosing the geographic region of the cloud instance and adding content filters to prevent abuse.

Interestingly, Azure OpenAI Service pricing is more competitive than OpenAI API. In OpenAI API, the prices of refined GPT-3 models are higher than base models. In Azure, both basic and fine-tuned models have the same price. Azure also allows customers to pay for fine-tuned models using an hourly payment model instead of the usual token-based pricing, which is more convenient for applications with high volume model usage.

Microsoft and OpenAI both benefit from the growing market for Azure OpenAI Service and OpenAI API. OpenAI API is powered by Microsoft’s cloud, which means that as the number of customers increases, OpenAI’s Azure bill will grow. On the other hand, Microsoft has a license agreement with OpenAI. The details of the deal have not been made public (aside from the fact that Microsoft exclusive licensing rights to the technology of OpenAI). But with the increasing use of Azure OpenAI Service, Microsoft’s licensing costs will increase.

In the long run, however, I expect Azure to eat into OpenAI’s business as the generative AI market grows and matures. Much more flexible than OpenAI API, Azure also offers many other services essential for large-scale software development and machine learning.

OpenAI API will continue to be a hub for research and innovation, but the high-paying customers who want to build scalable products will slowly migrate to Azure. As a result, OpenAI is becoming increasingly dependent on Microsoft as a source of revenue for its models.

Azure’s robustness, flexibility, and convenience also allow it to compete with emerging open-source and commercial alternatives. Microsoft’s AI-optimized and scalable hardware infrastructure enables it to deliver generative models at competitive prices. At the same time, the complexity and initial cost of setting up the hardware for generative models means that hosted systems such as Azure OpenAI remain the preferred option for many companies that lack in-house talent to build open source models.

The market for RLHF

Before ChatGPT, unsupervised or self-managed learning was the primary way to train LLMs and other generative models. The model is provided with a very large corpus of text, software code, images or other types of data and left to learn relevant patterns on its own. During training, the model masks parts of the data and tries to predict them. It then reveals the masked sections and compares its predictions to the ground truth, correcting its inner parameters to improve its predictions. By repeating this process over and over, the LLM learns statistical representations of the training corpus and can use them to generate relevant sequences of text, computer instructions, image pixels, etc.

ChatGPT showed the power of adding human control to the training process. ChatGPT is trained using reinforcement learning from human feedback (RLHF). Instead of pure unsupervised learning, OpenAI’s engineers used human annotators to guide the model through various stages of the training process. The team first refined a pre-trained model using a series of prompts and answers written by human experts. They then created a ‘reward model’ that ranked the output of the language model. The reward model is trained on output quality scores from human reviewers. Finally, they used the reward model to further train the model and tailor the output to human preferences. ChatGPT’s impressive results show just how far LLMs can be pushed with human help.

With ChatGPT’s success, the market for RLHF-trained LLMs is likely to grow. Companies will want to use the technique to fine-tune LLMs like ChatGPT to follow application-specific instructions. But the RLHF pipeline requires complicated development and management tools, including data preparation and annotation, reward model development, model and data versioning, regular retraining, model monitoring and auditing, and much more.

Fortunately for Microsoft, the Azure platform is well prepared to meet such requirements through its MLops and data warehousing tools. This, along with the scalable cloud infrastructure and software development tools, will give Microsoft an edge in this more specialized niche of generative models.

Microsoft has missed the boat on smartphones and mobile platforms. But the early investment in OpenAI, an AI lab that didn’t have a profitable business model at the time, has given it the opportunity to capture a large share of the market for the next wave of disruptive innovation.