Federated learning at the edge can surpass the cloud in terms of privacy, speed, and cost

In the 2000s, the ‘cloud’ started to take off. Programmers and companies began purchasing virtual computing resources on demand to run their software and applications.

Over the past two decades, developers have become accustomed to and dependent on readily available infrastructure managed and maintained by someone else. And this is no surprise. By stripping away hardware and infrastructure, developers and companies can focus on product innovation and user functions.

Amazon Web Services, Microsoft Azure, and Google Cloud have made storage and compute ubiquitous, on-demand, and easy to deploy. And these hyperscalers have built robust, high-margin businesses on top of this approach. Organizations that rely on the cloud have traded capital expenditures (servers and hardware) for operational expenditures (pay-as-you-go compute and storage resources).

Enter federated learning

While the ease of use of the cloud is a boon to any startup team trying to innovate at all costs, cloud-centric architecture is a significant cost as a company scales. In fact, 50% of large SaaS companies’ revenues go to cloud infrastructure.

As machine learning (ML) continues to grow in popularity and usability, organizations store an increasing amount of data in the cloud and train ever-larger models in search of higher model accuracy and greater user benefit. This exacerbates dependency on cloud providers and organizations find it difficult to repatriate workloads to on-premises solutions. To do that, they would basically have to hire an excellent infrastructure team and design their systems from scratch.

Organizations are looking for tools that enable new product innovation and provide high accuracy with low latency while being cost effective.

Enter federated learning (FL) on the edge.

What is Federated Learning (FL) on the Edge?

FL, or collaborative learning, takes a different approach to data storage and computing. For example, while popular cloud-centric ML approaches send data from your phone to centralized servers and collect that data in a silo, FL on the edge keeps data on the device (that is, your mobile phone or tablet). It works in the following way:

Step 1: Your edge device (or mobile phone) downloads an initial model from an FL server.

Step 2: Then the device is trained; data on the device improves the model.

Step 3: The encrypted training results are sent back to the model enhancement server, while the underlying data resides securely on the user’s device.

Step 4: With the on-device model, you perform training and inference at the edge in a fully distributed and decentralized manner.

This loop continues iteratively and the accuracy of your model increases.

Federated learning benefits for the user

When you are not dependent on or hindered by the centralization of data, the user benefits in dramatic ways. With FL on the edge, developers can improve latency, reduce network calls, and drive power efficiency while promoting user privacy and improved model accuracy.

FL on the edge is made possible by the ever-expanding hardware capabilities of the phones in our pockets. Every year the calculations on the device and the battery life improve. As the smartphone processor and hardware in our pocket improves, FL techniques will unlock increasingly complex and personalized use cases.

For example, imagine software that resides on your phone in a privacy-friendly way and can automatically compose replies to incoming emails with your individual tone, punctuation style, slang, and other hyper-personalized attributes – all you have to do is click send.

The appeal of the company is strong

In my conversations with multiple Fortune 500 companies, it’s become abundantly clear how much demand there is for FL at the fringes of all industries. CTOs explain how they searched for a solution to bring FL techniques to life on the edge. CFOs refer to the millions of dollars spent on infrastructure and model implementation that could otherwise be saved with an FL approach.

In my opinion, the three industries that have the most potential to reap the benefits of federated learning are finance, media, and e-commerce. Let me explain why.

Use Case #1: Finance – Improved latency and security

Many large multinational financial companies (Mastercard, PayPal) are eager to use FL on the edge to help them identify account takeovers, money laundering and fraud detection. More accurate models are shelved and not approved for launch.

Why? These models increase latency just enough to negatively impact the user experience – we can all think of apps that we no longer use because they took too long to open or they crashed. Companies cannot afford to lose users for these reasons.

Instead, they accept a higher false negative rate and deal with excessive account hijacking, money laundering, and fraud. FL on the edge enables businesses to simultaneously improve latency while showing a relative improvement in model performance compared to traditional cloud-centric deployments.

In the media sector, companies like Netflix and YouTube want to increase the relevance of their movie or video suggestions. The Netflix price famously awarded $1 million for a 10% performance boost over its own algorithm.

FL on the fringe has the potential to provide a similar impact. These days, when a new show launches or a popular sporting event goes live (such as the Superbowl), companies reduce the signals they receive from their users.

Otherwise, the sheer volume of data (running at the rate of millions of requests per second) creates a bottleneck in the network that prevents them from recommending content at scale. With edge computing, companies can use these signals to suggest personalized content based on understanding individual users’ tastes and preferences.

Use Case No. 3: Ecommerce — more timely and relevant suggestions

Finally, e-commerce and marketplace companies want to increase click-through rates (CTR) and drive conversions based on real-time feature stores. This allows them to reorder customer recommendations and make more accurate predictions without the slowdown of traditional cloud-based recommendations.

For example, imagine opening the Target app on your phone and getting highly personalized product recommendations in a completely privacy-focused way — no identifying data would have left your phone. Federated learning can increase CTR through a more performant, privacy-aware model that provides users with more timely and relevant suggestions.

The market landscape

Thanks to advances in technology, both large companies and start-ups are working to make FL more ubiquitous, so that businesses and consumers alike can benefit. For businesses, this is likely to mean lower costs; for consumers, it can mean a better user experience.

There are already a few early players in the space: Amazon SageMaker enables developers to deploy ML models primarily on edge devices and embedded systems; Google Distributed Cloud extends their infrastructure to the edge; and startup companies Nimbleedge are redefining the infrastructure stack.

While we are in the early innings, FL on the edge is here and the hyperscalers are in an incumbent’s dilemma. The revenue that cloud providers earn for computing power, storage and data is at risk; modern vendors who have adopted edge computing architecture can offer customers premium ML model accuracy and reduced latency. This improves the user experience and increases profitability – a value proposition you can’t ignore for long.

Neeraj Hablani is partner at Neotribe Ventures aimed at start-up companies developing breakthrough technologies.