a

Artificial Intelligence

Rethinking GenAI Architectures with Optical I/O

The growing complexity and size of AI models, especially generative AI models, are introducing huge scaling and power challenges. Communication bottlenecks are a significant drag on efficiency. Optical I/O technology changes the game by enhancing link density within and across nodes. By eliminating the bottlenecks caused by traditional interconnects (electrical I/O plus bulky, expensive pluggable optics), optical I/O enables nodes to connect at scale, allowing them to effectively function as a single, giant GPU.

Bandwidth

Bandwidth

5-10x higher bandwidth

Trillion-parameter AI models and advanced HPC designs require ever-increasing bandwidth. Ayar Labs’ optical I/O solution offers a total bidirectional bandwidth of 4 Tbps, opening up new possibilities for generative AI architectures.

Latency

Latency

10x lower latency

Latency limits the size and number of interconnected components in a system. Electrical I/O above 50 Gbps requires forward error correction (FEC) that introduces an additional latency of tens of ns which is not tolerated in the distributed computing systems used for HPC and AI. Ayar Labs optical I/O solution has a latency of 5 ns per chiplet + TOF with no FEC required.

Power

Power

4-8x more power efficient

Power efficiency has a direct impact on heat and reliability. To enable electrical I/O and pluggable optics to traverse systems, racks, and data centers, the 112 Gbps long-reach electrical I/O consumes 6-10 pJ/b of energy. Ayar Labs’ optical I/O solution consumes less than 5 pJ/b (10 Watts).

(Vs. traditional interconnects: pluggable optics+electrical SerDes)

Revolutionizing Generative AI Performance

The systems required for AI training and inference are created by connecting hundreds — if not tens of thousands — of GPUs and other accelerators. The flow of data across these complex systems is increasingly constrained by traditional interconnects that limit compute performance and utilization while driving up power consumption and costs. This presents a significant barrier toward full acceleration of the data center required to achieve affordable and profitable AI.

With traditional interconnects, as AI model complexity increases, the efficiency of systems decreases (e.g., one GPU operating at 80 percent efficiency vs. 64 GPUs at 50 percent efficiency or 256 GPUs at 30 percent efficiency).

Ayar Labs’ optical I/O is optimized for AI, maximizing the compute efficiency and performance of AI infrastructure, while reducing costs, latency, and power consumption.

Disaggregation and Optical Interconnect in AI/HPC Networks

This solution brief explores the needs of near-future generative AI architectures, as well as how optical I/O can help solve some of the problems currently facing industry leaders.

“Optical connectivity will be important to scale accelerated computing clusters to meet the fast-growing demands of AI and HPC workloads. Ayar Labs has unique optical I/O technology that meets the needs of scaling next-generation silicon photonics-based architectures for AI.”

Bill Dally,
Chief Scientist & Senior VP of Research, NVIDIA

Ayar Labs’ Optical I/O Solution

Ayar Labs is driving the AI revolution with optical I/O. The complexity and size of AI models are increasing at a rate that traditional interconnect technology can’t handle. Ayar Labs has developed the industry’s first optical I/O solution that maximizes the compute efficiency and performance of growing AI infrastructure, while reducing costs, latency, and power consumption. Based on open standards and optimized for both AI training and inference, Ayar Labs’ optical I/O solution is backed by a robust ecosystem that enables it to integrate smoothly into AI systems at scale.

Ayar Labs’ in-package optical I/O solution includes the TeraPHY™ optical I/O chiplet and the SuperNova™ multi-wavelength light source.

Moving Data with Light

Ayar Labs’ Solution Combines Two Industry-First Technologies

TeraPHY™ In-Package Optical I/O Chiplet

  • Industry-first monolithic in-package optical I/O chiplet
  • Combines silicon photonics with standard CMOS manufacturing processes
  • Enables ASICs to communicate across a wide range of distances, from mm to km

SuperNova™ Multi-Wavelength Light Source

  • Industry-first multi-wavelength, multi-port optical source
  • Provides up to 16 wavelengths of light and powers up to 16 ports
  • First optical source designed to be compliant with the CW-WDM MSA specification

Our AI Technology Partners

HPE
Intel
NVIDIA

Optical I/O Use Cases for Generative AI

Generative AI Scale Out

Scaling large language models (LLMs) requires spreading computation across multiple GPUs, resulting in bottlenecks due to serialized communication. This results in a cascading delay effect between network layers, leading to more time spent on communication rather than computation as GPUs and chassis are added. In next-gen LLMs requiring entire clusters, the problem intensifies. For example, a configuration with 256 GPUs may only reach 30 percent compute efficiency compared to 80 percent efficiency with a single GPU, leading to diminishing returns for increasing investments.

Optical I/O addresses this issue head on by eliminating communication lags. It supports seamless scaling and more effective use of GPU resources, paving the way for more powerful large-scale generative AI architecture without runaway costs.

Generative AI model complexity is growing exponentially. Traditional interconnects create a bottleneck for data transfer, forcing GPUs to remain idle. Optical I/O connects nodes at scale so they work like one giant GPU.

Revolutionary Optics + Glass Waveguide Redefine AI, Telecom & Data Center Infrastructures

Ayar Labs is working with industry leaders Ericsson and Corning to enable faster, AI-capable telecom radios to handle the explosion of edge traffic and services. Ayar Labs’ optical I/O with Corning’s unique glass waveguide module also enable new architectures for AI and data centers.

Traditional mobile networks need to rapidly transform at the edge to handle the exponential growth in traffic and AI-driven services. Ayar Labs and Corning have developed a next-generation AI optical solution that couples Ayar Labs’ TeraPHY™ optical I/O chiplets and Corning’s unique glass waveguide module with built-in optical connections. Ericsson is partnering with Ayar Labs and Corning in this AI-driven solution as they develop their telecom future radio. This new, integrated optical solution will provide high-density, cost-effective, low latency, energy-efficient connectivity for AI architectures, 6G, data centers, and other compute-intensive applications.

Disaggregated Architectures for AI and HPC

Disaggregated architectures decouple memory from processors, accelerators, and storage to enable flexible and dynamic resource allocation — depending on the current tasks assigned to the data center. A shift to disaggregated architectures increases flexibility and performance, enabling quick and dynamic construction of customized node configurations to offload work that requires larger I/O or contains less computes.

A transition to silicon photonics (optical I/O) enables memory to be pooled with low latency and high performance. Faster photonic interconnects between memory and XPUs (CPUs, GPUs, FPGAs, ASICs, and accelerators) will dramatically improve performance and throughput.

Disaggregated architectures enable resource composability for more efficient utilization in AI/ML, high performance computing, and cloud.

Pin It on Pinterest

Share This