Join top executives in San Francisco July 11-12 to learn how business leaders are getting ahead of the generative AI revolution. Learn more
Celestial AI, a developer of optical interconnection technology, has announced a successful Series B financing round, raising $100 million for its Photonic Fabric technology platform. IAG Capital Partners, Koch Disruptive Technologies (KDT) and Temasek’s Xora Innovation fund led the investment.
Other participants included Samsung Catalyst, Smart Global Holdings (SGH), Porsche Automobil Holding SE, The Engine Fund, ImecXpand, M Ventures and Tyche Partners.
According to Celestial AI, their Photonic Fabric platform represents a significant advancement in optical connectivity performance and surpasses existing technologies. The company has raised a total of $165 million in seed funding through Series B.
Taking on the “memory wall” challenge
Advanced artificial intelligence (AI) models, such as the widely used GPT-4 for ChatGPT and recommendation engines, require exponentially increasing memory capacity and bandwidth. However, cloud service providers (CSPs) and hyperscale data centers face challenges due to the interdependence of memory scaling and computing, referred to as the “memory-wall” challenge.
The limitations of electrical connections, such as limited bandwidth, high latency and high power consumption, hinder the growth of AI business models and progress in AI.
To address these challenges, Celestial AI has teamed up with hyperscalers, AI computers, and memory providers to develop Photonic Fabric. The optical link is designed for disaggregated, exascale computing and memory clusters.
The company claims its patented Optical Compute Interconnect (OCI) technology enables the disaggregation of scalable data center memory and enables accelerated computing.
Memory capacity is a major issue
Dave Lazovsky, CEO of Celestial AI, told VentureBeat, “The main issue going forward is memory capacity, bandwidth, and data movement (chip-to-chip interconnectivity) for large language models (LLMs) and recommendation engine workloads. Our Photonic Fabric technology allows you to integrate photonics directly into your silicon chip. An important advantage is that with our solution you can deliver data at any point on the silicon chip up to the computer point. Competitive solutions such as Co-Packaged Optics (CPO) cannot do this, as they only deliver data to the edge of the die.”
Lazovsky claims that Photonic Fabric has successfully addressed the challenging beachside problem by providing significantly increased bandwidth (1.8 Tbps/mm²) with nanosecond latencies. As a result, the platform offers fully photonic compute-to-compute and compute-to-memory links.
The recent financing round has also attracted the attention of Broadcom, which is contributing to the development of Photonic Fabric prototypes based on Celestial AI’s designs. The company expects these prototypes to be ready for shipment to customers within 18 months.
Enabling accelerated computing through optical interconnection
Lazovsky stated that data rates should also increase with the increasing volume of data transferred within data centers. He explained that as these speeds increase, electrical connections face issues such as loss of signal fidelity and limited bandwidth that doesn’t grow with data growth, limiting overall system throughput.
According to Celestial AI, Photonic Fabric’s low-latency data transmission facilitates the connection and disaggregation of a significantly larger number of servers than traditional electrical connections. This low latency also allows latency-sensitive applications to use external memory, a capability previously unattainable with traditional electrical connections.
“We are enabling hyperscalers and data centers to split their memory and compute resources without compromising power, latency and performance,” Lazovsky told VentureBeat. “Inefficient use of server DRAM memory translates to $100 million (if not billions) wasted at hyperscalers and enterprises. By enabling memory disaggregation and memory pooling, we not only help reduce the amount of memory expenditure, but also prove memory usage.”
Storing and processing larger datasets
The company claims its new offering can deliver data from any point on the silicon directly to the computing point. Celestial AI says Photonic Fabric surpasses the limitations of silicon edge connectivity and offers a packet bandwidth of 1.8 Tbps/mm², which is 25 times greater than that offered by CPO. In addition, the company claims that Photonic Fabric achieves ten times lower latency by delivering data directly to the computing point rather than at the edge.
Celestial AI aims to simplify business computation for LLMs such as GPT-4, PaLM, and deep learning recommendation models (DLRMs) that can range in size from 100 billion to over 1 trillion parameters.
Lazovsky explained that since AI processors (GPU, ASIC) have a limited amount of high-bandwidth memory (32 GB to 128 GB), companies today have to connect hundreds to thousands of these processors to handle these models. However, this approach reduces the efficiency of the system and drives up costs.
“By increasing the addressable memory capacity of each high-bandwidth processor, Photonic Fabric enables each processor to store and process greater amounts of data, reducing the number of processors required,” he added. “Providing high-speed chip-to-chip links allows the connected processor to process the model faster, increasing throughput and reducing costs.”
What’s next for Celestial AI?
Lazovsky said the money raised in this round will be used to accelerate production and commercialization of the Photonic Fabric technology platform by expanding Celestial AI’s engineering, sales and technical marketing teams.
“Given the growth of generative AI workloads due to LLMs and the pressure this is placing on current data center architectures, the demand for optical connectivity is rapidly increasing to support the transition from general data center infrastructure to accelerating computing,” Lazovsky told VentureBeat. “We expect to grow the workforce by approximately 30% to 130 employees by the end of 2023.”
He said that as the use of LLMs expands across applications, infrastructure costs will also increase proportionally, leading to negative margins for many software applications on the Internet. In addition, data centers are reaching power limitations, limiting the amount of computing power that can be added.
To overcome these challenges, Lazovsky aims to minimize reliance on expensive processors by providing high-bandwidth, low-latency chip-to-chip and chip-to-memory interconnect solutions. He said this approach is designed to reduce capital expenditures of companies and improve the efficiency of their existing infrastructure.
“By breaking the memory wall and helping improve system efficiency, our new offering aims to help shape the future direction of AI model progression and adoption,” he said. “When memory capacity and bandwidth are no longer a limiting factor, data scientists can experiment with larger or different model architectures to unlock new applications and use cases. We believe that more companies and applications can adopt LLMs faster by reducing the cost of adopting large models.”
VentureBeat’s mission is to become a digital city plaza where tech decision makers can learn about transformative business technology and execute transactions. Discover our Briefings.
