How Google is accelerating ML development

Accelerating the development of machine learning (ML) and artificial intelligence (AI) with optimized performance and cost is a key goal for Google.

Google kicked off its Next 2022 conference this week with a series of announcements about new AI capabilities in its platform, including computer vision as a service with Vertex AI vision and the new OpenXLA open-source ML initiative. During a session at the Next 2022 event, Mikhail Chrestkha outbound product manager at Google Cloud discussed additional incremental AI enhancements, including support for the Nvidia Merlin recommendation system framework, AlphaFold batch inference too TabNet support.

[Follow VentureBeat’s ongoing Google Cloud Next 2022 coverage »]

Users of the new technology shared their usage scenarios and experiences during the session.

“Having access to a strong AI infrastructure becomes a competitive advantage to get the most value from AI,” Chrestkha said.

Uber uses TabNet to improve food delivery

TabNet is an in-depth approach to tabular data learning that uses transformer techniques to help improve speed and relevance.

Chrestkha explained that TabNet is now available in the Google Vertex AI platform, making it easier for users to build explainable models at scale. He noted that the Google implementation of TabNet automatically selects the appropriate function transformations based on the input data, the size of the data, and the prediction type to get the best results.

TabNet is not a theoretical approach to improve AI predictions, it is an approach that is already showing positive results in real-world situations. One of TabNet’s first executors is Uber.

Kai Wang, senior product manager at Uber, explained that a platform his company created called Michelangelo now handles 100% of Uber’s ML use cases. Those use cases include estimated time of arrival (ETA), estimated time to delivery (ETD) from UberEats, and driver/driver matching.

The basic idea behind Michelangelo is to provide Uber’s ML developers with infrastructure on which to deploy models. Wang said Uber is constantly evaluating and integrating third-party components, while selectively investing in key platform areas to build in-house. One of the fundamental third-party tools that Uber relies on is Vertex AI, to support ML training.

Wang noted that Uber is evaluating TabNet with Uber’s real-life use cases. An example of a use case is UberEat’s prep time model, which is used to estimate how long it will take a restaurant to prepare food after an order is received. Wang stressed that the prep time model is one of the most critical models currently used at UberEats.

“We compared the TabNet results to the base model, and the TabNet model showed a big improvement in terms of model performance,” Wang said.

Only the FAX for Cohere

Cohere develops platforms that help organizations take advantage of the natural language processing (NLP) capabilities enabled by large language models (LLMs).

Cohere also benefits from Google’s AI innovations. Siddhartha Kamalakara, a machine learning engineer at Cohere, explained that his company has built its own ML training framework called FAX, which now heavily leverages Google Cloud’s TPUv4 AI accelerator chips. He explained that FAX’s job is to consume billions of tokens and train models from hundreds of millions to hundreds of billions of parameters.

“TPUv4 pods are some of the most powerful AI supercomputers in the world, and a full V4 pod has 4096 chips,” Kamalakara said. “TPUv4 allows us to train large language models very quickly and bring those improvements straight to customers.”