How self-supervised learning can drive the advancement of medical AI

Self-supervised learning has been a rapidly rising trend in artificial intelligence (AI) in recent years, as researchers look to take advantage of large-scale unannotated data to develop better machine learning models.

In 2020, Meta’s chief AI scientist, Yann Lecun, said that supervised learning, meaning training an AI model on a labeled dataset, would play a declining role as supervised learning became more widely used.

“Most of what we learn as humans and most of what animals learn is in a self-supervised mode, not an reinforcement mode,” he told a virtual session audience at the 2020 International Conference on Learning Representation (ICLR) And in a 2021 Meta blog post, LeCun explained that self-supervised learning “obtains supervision signals from the data itself, often taking advantage of the underlying structure in the data.” Therefore, it can utilize a “variety of surveillance signals across concurrent modalities (e.g. video and audio) and over large data sets – all without relying on labels.”

Increasing use of self-directed learning in medicine

Those benefits have led to a remarkably growing use of self-directed learning in healthcare and medicine, thanks to the vast amount of unstructured data available in that sector, including electronic health records and datasets of medical images, bioelectric signals and sequences and structures. of genes and proteins. Previously, the development of medical applications of machine learning required manual annotation of data, often by medical experts.

This has been a bottleneck to progress, says Pranav Rajpurkar, assistant professor of biomedical informatics at Harvard Medical School. Rajpurkar leads a research lab focused on deep learning for label-efficient interpretation of medical images, design for collaboration between clinicians and AI and open benchmark curation.

“We’ve seen a lot of exciting developments with our stable data sets,” he told VentureBeat.

But it took a “paradigm shift” to go from 100 algorithms that perform very specific medical tasks to the thousands needed without going through a tedious, intensive process. That’s where self-supervised learning, with its ability to predict any unobserved or hidden part of an input from a perceived or unhidden part of an input, has been a game-changer.

Emphasize self-supervised learning

In a recent review paper in Nature Biomedical EngineeringRajpurkar, along with cardiologist, scientist and author Eric Topol and student researcher Rayan Krishnan, highlighted self-supervised methods and models used in medicine and healthcare, as well as promising applications of self-supervised learning for the development of models that use of multimodal datasets, and the challenges of collecting unbiased data for their training.

The article, Rajpurkar said, aimed to “communicate the opportunities and challenges underlying the paradigm shift that we will see in many applications of AI in the coming years, especially in medicine.”

With self-supervised learning, Rajpurkar explained that he, “…can learn about a particular data source, whether that be a medical image or signal, by using untagged data. perform a task I care about, within medicine and beyond without actually collecting large stable data sets.”

Great Achievements Unlocked

In 2019 and 2020, Rajpurkar’s lab saw some of the first major achievements that unlocked self-supervised learning for interpreting medical images, including chest X-rays.

“With a few tweaks to algorithms that helped us understand natural images, we reduced the number of chest X-rays that had to be seen in a particular disease before we could properly start identifying that disease,” he said.

Rajpurkar and his colleagues applied similar principles to electrocardiograms.

“We showed that with some ways of applying self-directed learning, coupled with a little physiological insights into the algorithm, we were able to leverage a lot of unlabeled data,” he said.

Since then, he has also applied self-directed learning to lung and heart sound data.

“What’s really exciting about deep learning as a whole, but especially in the last two years, is that we’ve been able to transfer our methods very well across the different modalities,” Rajpurkar said.

Self-supervised learning across modalities

For example, another soon-to-be-published paper showed that even with zero annotated examples of diseases on chest X-rays, Rajpurkar’s team was able to detect diseases on chest X-rays and classify them almost at the level of radiologists. . in various pathologies.

“We basically learned from images combined with radiological reports dictated at the time of their interpretation, and combined these two modalities to create a model that could be applied in a zero-shot manner – meaning labeled samples were not needed to be able to classify different diseases,” he said.

Whether you’re working with proteins or images or text, the process borrows from the same kind of frameworks, methods and terminologies in a way that’s more uniform than it was even two or three years ago.

“That’s exciting for the field because it means a series of advancements on a general set of tools help everyone work with and on these very specific modalities,” he said.

In the interpretation of medical images, which Rajpurkar has been focusing on for years, this is “absolutely revolutionary,” he said. “Instead of solving and repeating problems one by one”[ing] If I do this process 1000 times, I can solve a much larger set of problems in one go.”

Time to apply methods

These capabilities have created momentum for developing and adopting self-supervised learning methods in medicine and healthcare, and likely other industries that also have the ability to collect data at scale, Rajpurkar said, especially those industries that don’t have the have sensitivity associated with medical data.

In the future, he adds that he is interested in getting closer to solving the full range of potential tasks that a medical expert does.

“The goal has always been to enable intelligent systems that can increase the accessibility of medicine and healthcare to a large audience,” he said, adding that what excites him is building solutions that don’t just solve one small problem: “We are working on a world with models that combine different signals so that doctors or patients can make intelligent decisions about diagnoses and treatments.”