Technology Hugging Face, AWS partner in open-source machine learning amid...

Hugging Face, AWS partner in open-source machine learning amid AI arms race


- Advertisment -

View all on-demand sessions from the Intelligent Security Summit here.

Impressive advances in large language models (LLMs) are showing signs of what could be the start of a major shift in the technology industry. AI startups and big tech companies are finding new ways to leverage advanced LLMs for everything from composing emails to generating software code.

However, the promises of LLMs have also led to an arms race between tech giants. In their efforts to build their AI arsenals, big tech companies threaten to push the field toward less openness and more secrecy.

In the midst of this rivalry, Cuddle Sight charts another strategy that will provide scalable access to open-source AI models. Hugging Face partners with Amazon Web Services (AWS) to facilitate the adoption of open-source machine learning (ML) models. In an era where advanced models are increasingly inaccessible or hidden behind walled gardens, an easy-to-use open-source alternative can expand the market for applied machine learning.

Open source models

While large-scale machine learning models are very useful, setting them up and running them requires special expertise that few companies possess. The new partnership between Hugging Face and AWS will attempt to address these challenges.


Intelligent Security Summit on demand

Learn the critical role of AI and ML in cybersecurity and industry-specific case studies. Check out on-demand sessions today.

Look here

Developers can use Amazon’s cloud tools and infrastructure to easily refine and deploy advanced models from Hugging Face’s ML repository.

The two companies began working in 2021 with the release of Hugging Face deep learning containers (DLCs) on SageMaker, Amazon’s cloud-based machine learning platform. The new partnership will extend the availability of Hugging Face models to other AWS products and Amazon’s cloud-based AI accelerator hardware to accelerate training and inference.

“Since we started offering Hugging Face natively in SageMaker, usage has grown exponentially and we now have over 1,000 customers using our solutions every month,” Jeff Boudier, product director at Hugging Face, told VentureBeat. “Through this new partnership, we are now working hand-in-hand with the engineering teams building new efficient hardware for AI, such as AWS Trainium and AWS Inferentia, to build solutions that are ready to run on Elastic Compute Cloud (EC2) and Elastic Kubernetes. Service (EKS).”

The AI ​​arms race

Tech leaders have been talking about the transformative nature of machine learning for several years now. But never has this transformation been felt as it has been in recent months. The release of OpenAI’s ChatGPT language model has set the stage for a new chapter in the race for AI dominance.

Microsoft recently poured $10 billion into OpenAI and is working hard to integrate LLMs into its products. Google has invested $300 million in Anthropic, a rival to OpenAI, and is doing everything it can to protect its online search empire from the rise of LLM-powered products.

These partnerships have clear benefits. With Microsoft’s financial backing, OpenAI has been able to train very large and expensive machine learning models on specialized hardware and deploy them at scale to millions of people. Anthropic will also gain special access to the Google Cloud Platform through its new partnership.

However, the rivalry between major tech companies also has drawbacks for the field. For example, since it began working with Microsoft, OpenAI has stopped open sourcing most of its machine learning models and serves them through a paid application programming interface (API). It has also become locked into Microsoft’s cloud platform and its models are only available on Azure and Microsoft products.

On the other hand, Hugging Face remains committed to continuing to provide open-source models. The collaboration between Hugging Face and Amazon allows developers and researchers to deploy open source models such as BLOOMZ (a GPT-3 alternative) and Stable Diffusion (a rival to DALL-E 2).

“This is an alliance between the open source machine learning leader and the cloud services leader to jointly build the next generation of open source models and solutions to use them. Everything we build together will be open-source and openly accessible,” said Boudier.

Hugging Face also wants to avoid the kind of lock-in that other AI companies face. While Amazon remains the cloud provider of choice, Hugging Face continues to partner with other cloud platforms.

“This new partnership is not exclusive and does not change our relationships with other cloud providers,” said Boudier. “Our mission is to democratize good machine learning, and to do that we need to empower users everywhere they use our models and libraries. We will continue to work with Microsoft and other clouds to serve customers everywhere.”

Openness and transparency

OpenAI’s API model is a useful option for companies that lack in-house ML expertise. Hugging Face also provides a similar service through its Inference Endpoint and Inference API products. But APIs will prove limited for organizations that want more flexibility to customize and integrate the models with other machine learning architectures. They are also inconvenient for research that requires access to model weights, inclines, and training data.

Easy-to-deploy, scalable cloud tools, such as those from Hugging Face, make these types of applications possible. At the same time, the company is developing tools to detect and flag abuse, bias, and other issues with ML models.

“Our vision is that openness and transparency [are] the way forward for ML,” said Boudier. “ML is science-driven and science requires reproducibility. Ease of use makes everything accessible to the end users so people can understand what models can and cannot do, [and] how they should and should not be used.”

VentureBeat’s mission is to become a digital city plaza where tech decision makers can learn about transformative business technology and execute transactions. Discover our Briefings.

Shreya Christina
Shreya has been with for 3 years, writing copy for client websites, blog posts, EDMs and other mediums to engage readers and encourage action. By collaborating with clients, our SEO manager and the wider team, Shreya seeks to understand an audience before creating memorable, persuasive copy.


Please enter your comment!
Please enter your name here

Latest news

Vulkan Vegas 50 free spins 50 spinów na Book of Dead!

Wystarczy zarejestrować się przez nasz link, aby odebrać ekskluzywny bonus powitalny, czyli 50 darmowych obrotów. Później pozostaje już je...

Vulkan Vegas bonus za rejestrację Odbierz darmowy bonus

Druga część bonusu to 25 darmowych spinów, które można wykorzystać w automacie Book of Dead. Oprócz bonusu powitalnego, Vulkan...

Odkryj Emocje Związane z Grami Online

Wszystkie Twoje wygrane są utrzymywane w tajemnicy i nikt inny nie zna Twojego adresu IP ani kwoty, którą wygrałeś...

Sprawdź Najlepsze Kasyno Online w Polsce

Przy pierwszym uruchomieniu strony użytkownik otrzymuje informacje o niezwykle przystępnej ofercie bonusowej. Do tego dochodzi ogromna, bardzo dynamicznie rozwijająca...
- Advertisement -

Mostbet dən pis bukmeker kontorunun əlamətləri Təhsil Portalı

Contentin Virtual İdman mərcləriMostBet bukmeyker şirkəti ölkəmzidə niyə bloklanmır?"Əş-Şəbab"ın liderlərindən biri Somalidə hücum nəticəsində yaralanıbMostBet-də oyunçular üçün təqdim olunan...

PinUp təyyarə oyunu Pin Up aviator

ContentAviator game OYUNUN MÜSLÜMANLARI VƏ ƏSAS YÜKLƏRİYeni Kullanıcılar İçin Ödüller MostbetIlk bahis’i necə elemek olarMostbet identifikasiyasıMobil proqramda necə qeydiyyatdan...

Must read

- Advertisement -

You might also likeRELATED
Recommended to you