Unlock AI at the edge with new tools from Deci

Edge devices must be able to process delivered data quickly and in real time. And edge AI applications are only effective and scalable if they can make highly accurate image predictions.

Take the complex and mission-critical task of autonomous driving: all relevant objects in the driving scene must be taken into account, be it pedestrians, lanes, sidewalks, other vehicles or traffic signs and lights.

“For example, an autonomous vehicle driving through a busy city needs to maintain high accuracy while operating in real time with very low latency; otherwise, the lives of drivers and pedestrians could be at risk,” said Yonatan Geifman, CEO and co-founder of deep learning company to decide.

The key to this is semantic segmentation, or image segmentation. But there is a dilemma: semantic segmentation models are complex and often slow their performance.

“There’s often a tradeoff between the accuracy and the speed and size of these models,” said Geifman, whose company this week released a series of semantic segmentation models, DeciSeg, to solve this complex problem.

“This can be a barrier to real-time edge applications,” Geifman says. “Creating accurate and computationally efficient models is a real pain point for deep learning engineers, who make great efforts to achieve both the accuracy and speed that will get the job done.”

The power of the edge

According to Allied Market Research, the global market size of edge AI (artificial intelligence) will reach nearly $39 billion by 2030, a compound annual growth rate (CAGR) of nearly 19% over a 10-year period. In the meantime, Astute Analytics reports that the global edge AI software market will reach more than $8 billion by 2027, a CAGR of nearly 30% from 2021.

“Edge computing with AI is a powerful combination that could deliver promising applications for both consumers and enterprises,” said Geifman.

For end users, this translates to more speed, improved reliability and an overall better experience, he said. Not to mention better data privacy, as the data used for processing stays on the local device — cell phones, laptops, tablets — and doesn’t need to be uploaded to third-party cloud services. For enterprises with consumer applications, this represents a significant reduction in cloud computing costs, Geifman said.

Another reason why edge AI is so important: communication bottlenecks. Many machine vision edge devices require heavy analysis for high-resolution video streams. But if the communication requirements are too great in relation to the network capacity, some users will not get the required analysis. “Therefore, moving the computation to the edge, even partially, will allow for operation at scale,” Geifman said.

No critical considerations

Semantic segmentation is key to edge AI and is one of the most widely used computer vision tasks in many business industries: automotive, healthcare, agriculture, media and entertainment, consumer applications, smart cities and other image-intensive implementations.

Many of these applications “are critical in the sense that getting the correct and real-time segmentation prediction can be a matter of life or death,” Geifman said.

Autonomous vehicles, for example; another is semantic segmentation of the heart. For this critical task in MRI analysis, images are broken down into several anatomically meaningful segments that are used to estimate critical factors such as myocardial mass and wall thickness, Geifman explains.

There are, of course, examples beyond mission-critical situations, he said, such as video conferencing, virtual background functions or intelligent photography.

Unlike image classification models — which are designed to determine and label a single object in a given image — semantic segmentation models assign a label to each pixel in an image, Geifman explains. They are usually designed using the encoder/decoder architecture structure. The encoder gradually downsamples the input as the number of feature maps increases, constructing informative spatial features. The decoder receives these functions and gradually upsamples them into a full-resolution segmentation map.

And while it’s often required for many edge AI applications, there are significant barriers to running semantic segmentation models directly on edge devices. These include high latency and the inability to deploy models due to their size.

High-precision segmentation models are not only much larger than classification models, Geifman explains, they are also often applied to larger input images, “increasing their computational complexity quadratically.” This translates into slower inference performance.

For example, defect inspection systems running on production lines that need to maintain high accuracy to reduce false alarms, but not sacrifice speed, Geifman said.

Lower latency, higher accuracy

The DeciSeg models were automatically generated by Deci’s Automated Neural Architecture Construction (AutoNAC) technology. The Tel Aviv-based company says these “perform significantly better” than existing publicly available models, including Apple’s. MobileViT and Google’s DeepLab.

As Geifman explained, the AutoNAC engine considers a large search space of neural architectures. When searching this space, it considers parameters such as baseline accuracy, performance targets, inference hardware, compilers, and quantization. AutoNAC tries to solve a limited optimization problem while achieving multiple goals – that is, maintaining the basic accuracy with a model with a certain memory footprint.

The models deliver more than 2 times lower latency and 3 to 7% higher accuracy, Geifman said. This enables companies to develop new use cases and applications on edge AI devices, reduce the cost of inferences (because AI practitioners no longer have to perform tasks in expensive cloud environments), open up new markets and reduce development times. shorten, according to Geifman. AI teams can solve implementation challenges while achieving the desired accuracy, speed, and model size.

“DeciSeg models enable semantic segmentation tasks that previously could not be performed on edge applications because they required too many resources,” said Geifman. The new range of models “have the potential to transform industries in general.”