a

Artificial Intelligence

Rethinking Generative AI Architectures with Optical I/O

The growing complexity and size of AI models, especially generative AI models, are introducing huge scaling and power challenges. Communication bottlenecks are a significant drag on efficiency. Optical I/O technology changes the game by enhancing link density within and across nodes. By eliminating the bottlenecks caused by traditional interconnects (electrical I/O plus bulky, expensive pluggable optics), optical I/O enables nodes to connect at scale, allowing them to effectively function as a single, giant GPU.

Bandwidth

Bandwidth

5x higher data rates

Trillion-parameter AI models and advanced HPC designs require ever-increasing bandwidth. Ayar Labs’ optical I/O solution offers a total bidirectional bandwidth of 4 Tbps, opening up new possibilities for generative AI architectures.

Latency

Latency

10x lower latency

Latency limits the size and number of interconnected components in a system. Electrical I/O above 50 Gbps requires forward error correction (FEC) that introduces an additional latency of tens of ns which is not tolerated in the distributed computing systems used for HPC and AI. Ayar Labs optical I/O solution has a latency of 5 ns per chiplet + TOF with no FEC required.

Power

Power

8x more power efficient

Power efficiency has a direct impact on heat and reliability. To enable electrical I/O and pluggable optics to traverse systems, racks, and data centers, the 112 Gbps long-reach electrical I/O consumes 6-10 pJ/b of energy. Ayar Labs’ optical I/O solution consumes less than 5 pJ/b (10 Watts).

Revolutionizing Generative AI Performance

Large-scale generative AI workloads require robust communications and parallel processing, but traditional I/O creates bottlenecks. In-package optical I/O revolutionizes data transfer efficiency and bandwidth by connecting nodes so they effectively function as a single, giant GPU. It redefines generative AI architecture, enabling the pooling of accelerators, processors, memory, and storage across compute nodes for more efficient model training and inference.

Optical interconnects enable data to be transmitted at significantly higher throughput within each node and across nodes. More efficient communication increases GPU utilization so AI tasks complete faster. Fewer GPUs and switches are needed, reducing power and slashing CapEx and OpEx for today’s AI needs, and more efficiently scaling infrastructure for tomorrow’s AI needs.

Generative AI model complexity is growing exponentially. Traditional interconnects create a bottleneck for data transfer, forcing GPUs to remain idle. Optical I/O connects nodes at scale so they work like one giant GPU.

“Optical connectivity will be important to scale accelerated computing clusters to meet the fast-growing demands of AI and HPC workloads. Ayar Labs has unique optical I/O technology that meets the needs of scaling next-generation silicon photonics-based architectures for AI.”

– Bill Dally, Chief Scientist & Senior VP of Research, NVIDIA

NVIDIA

Ayar Labs’ Optical I/O Solution

As the semiconductor industry embraces the chiplet revolution, Ayar Labs’ in-package optical I/O solution is redefining I/O capabilities. Our groundbreaking TeraPHY in-package optical I/O chiplet and CW-WDM MSA-compliant SuperNova light source combine to deliver an I/O solution that obliterates traditional I/O bottlenecks and overcomes process constraints, unlocking revolutionary architectures for artificial intelligence/machine learning (AI/ML), disaggregated data centers, 6G, phased array sensor systems, and more.

TeraPHY Chiplet

Our AI Technology Partners

HPE
Intel
NVIDIA

Optical I/O Use Cases for Generative AI

Generative AI Scale Out

Scaling LLMs requires spreading computation across multiple GPUs, resulting in bottlenecks due to serialized communication. This results in a cascading delay effect between network layers, leading to more time spent on communication rather than computation as GPUs and chassis are added. In next-gen LLMs requiring entire clusters, the problem intensifies. For example, a configuration with 256 GPUs may only reach 30 percent compute efficiency compared to 80 percent efficiency with a single GPU, leading to diminishing returns for increasing investments. Optical I/O addresses this issue head on by eliminating communication lags. It supports seamless scaling and more effective use of GPU resources, paving the way for more powerful large-scale generative AI architecture without runaway costs.

Disaggregation and Optical Interconnect in AI/HPC Networks

This solution brief explores the needs of near-future generative AI architectures, as well as how optical I/O can help solve some of the problems currently facing industry leaders.

Disaggregated Architectures for AI and HPC

Ayar Labs is working with industry leaders Ericsson and Corning to enable faster, AI-capable telecom radios to handle the explosion of edge traffic and services. Ayar Labs’ optical I/O with Corning’s unique glass waveguide module also enable new architectures for AI and data centers.

Traditional mobile networks need to rapidly transform at the edge to handle the exponential growth in traffic and AI-driven services. Ayar Labs and Corning have developed a next-generation AI optical solution that couples Ayar Labs’ TeraPHY™ optical I/O chiplets and Corning’s unique glass waveguide module with built-in optical connections. Ericsson is partnering with Ayar Labs and Corning in this AI-driven solution as they develop their telecom future radio. This new, integrated optical solution will provide high-density, cost-effective, low latency, energy-efficient connectivity for AI architectures, 6G, data centers, and other compute-intensive applications.

Scalable and Sustainable AI: Rethinking Hardware and System Architecture

Scalable and Sustainable AI: Rethinking Hardware and System Architecture

In this webinar panelists discuss the challenges of scaling up AI workloads on existing architectures and the emerging solutions that can dramatically improve performance, efficiency, and scalability. Moderated by EE Times, the webinar features panelists from Ayar Labs, Google, Lawrence Berkeley National Laboratory, NVIDIA, and Tenstorrent.

AI Resources

Unlocking the True Potential of AI with In-Package Optical I/O

4 Tbps Optical FPGA Technology from Intel and Ayar Labs: Spend Your Precious Power on Compute, Not Connectivity

Sergey Shumarayev, Intel senior principal engineer/distinguished architect, shares how optical FPGA technology addresses socket power challenges for data-intensive applications like generative AI.
Optical I/O Chiplets Eliminate Bottlenecks to Unleash Innovation

Optical I/O Chiplets Eliminate Bottlenecks to Unleash Innovation

This technical brief examines the evolution of optical communications in computing systems and the transition to ‘Phase Two’ of Moore’s Law through in-package optical I/O (OIO).

  • Trends Driving In-Package Optical I/O Chiplets
  • Electrical I/O Barriers to High-Performance Architectures
  • Ayar Labs Optical I/O Chiplet
  • Applications of Optical I/O
Meeting the Bandwidth Demands of Next-Gen HPC & AI System Architectures

SuperNova™ Light Source: The Industry’s First CW-WDM MSA-Compliant 16 Wavelength Light Source

Ayar Labs has made the next big leap in optical I/O technology by introducing the industry’s first CW-WDM MSA-compliant 16 wavelength light source, which can drive 256 optical carriers for 16 Tbps of bi-directional bandwidth — a level of bandwidth essential for AI workloads.

The 16 wavelength SuperNova™ light source offers compact packaging, operates at wide temperature ranges, and can supply light for 256 data channels, making it capable of handling the significantly higher throughput required for the massive growth in AI applications.

Pin It on Pinterest

Share This