Ray, the machine learning technology behind OpenAI, goes up to Ray 2.0

Over the past two years, one of the most common ways for organizations to scale and run increasingly larger and more complex artificial intelligence (AI) workloads has been the open-source Ray frameworkused by companies from OpenAI to Shopify and Instagram.

Ray enables machine learning (ML) models to scale across hardware resources and can also be used to support MLops workflows in various ML tools. Ray 1.0 came out in September 2020 and has had a series of iterations over the past two years.

Today, the next major milestone was released, with the general availability of Ray 2.0 at the Ray Summit in San Francisco. Ray 2.0 extends the technology with the new Ray AI Runtime (AIR) which is intended to work as a runtime layer for running ML services. Ray 2.0 also includes capabilities designed to simplify building and managing AI workloads.

In addition to the new edition, AnyscaleRay’s principal commercial financier, announced a new business platform for running Ray. Anyscale also announced a new $99 million funding round co-led by existing investors Addition and Intel Capital with participation from Foundation Capital.

“Ray started as a small project at UC Berkeley and it has grown much further than we initially imagined,” said Robert Nishihara, Anyscale’s co-founder and CEO, during his keynote address at the Ray Summit.

OpenAI’s GPT-3 was trained on Ray

It’s hard to underestimate Ray’s fundamental importance and reach in today’s AI space.

Nishihara went through a laundry list of big names in the IT industry that Ray use during his keynote. One of the companies he mentioned is ecommerce platform vendor Shopify, which Ray uses to scale its ML platform that uses TensorFlow and PyTorch. Grocery Instacart is another Ray user taking advantage of the technology to train thousands of ML models. Nishihara noted that Amazon is also a Ray user for multiple types of workloads.

Ray is also a fundamental element for OpenAI, which is one of the leading AI innovators, and is the group behind the GPT-3 Large Language Model and DALL-E image generation technology.

“We use Ray to train our biggest models,” said Greg Brockman, CTO and co-founder of OpenAI, at the Ray Summit. “So it’s been really helpful for us to just scale up to a pretty unprecedented scale.”

Brockman noted that he sees Ray as a developer-friendly tool, and the fact that it’s a third-party tool that OpenAI doesn’t have to maintain is helpful too.

“If something goes wrong, we can complain on GitHub and ask an engineer to work on it so it reduces some of the burden of building and maintaining infrastructure,” Brockman said.

More machine learning goodness is built into Ray 2.0

Before Ray 2.0, Nishihara’s primary goal was to make it easier for more users to take advantage of the technology, while also providing performance optimizations that benefit users large and small.

Nishihara noted that a common pain point in AI is that organizations can be tied to a certain framework for a certain workload, but over time realize that they want to use other frameworks as well. For example, an organization might start out using only TensorFlow, but realize they also want to use PyTorch and HuggingFace in the same ML workload. With the Ray AI Runtime (AIR) in Ray 2.0, it is now easier for users to unify ML workloads across multiple tools.

Model Deployment is another common pain point that Ray 2.0 aims to help solve, with the Ray Serve Deployment Chart capability.

“It’s one thing to deploy a handful of machine learning models. It’s quite another to deploy hundreds of machine learning models, especially when those models can be interdependent and have different dependencies,” Nishihara said. “As part of Ray 2.0, we are announcing Ray Serve implementation charts, which address this issue and provide a simple Python interface for scalable model building.”

Looking ahead, Nishihara’s goal with Ray is to enable wider use of AI by making it easier to develop and manage ML workloads.

“We want to get to the point where any developer or organization can succeed with AI and derive value from AI,” Nishihara said.