The UCIe (Universal Chiplet Interconnect Express) 2.0 specification was released in August 2024. This is critical for the interconnect chiplet ecosystem and sets the stage for unleashing next-gen AI architectures. UCIe will be instrumental in driving broad adoption and manufacturability of optical I/O in order to deliver the high bandwidth, energy efficiency, low latency, and reach required for AI innovation and application growth.
If you’re curious to learn more about what UCIe means for the future of optical I/O, you’ve come to the right place. In this FAQ, we’ll explore why the UCIe standard is so important, what you can do with UCIe and optical I/O, how to integrate UCIe-based optical I/O into product designs, and more.
Without further ado, let’s start with the question that many customers and partners frequently ask us:
What exactly is UCIe?
UCIe is an open specification that facilitates die-to-die interconnects between chiplets within a package. Established in 2022, UCIe supports an interoperable, multi-vendor chiplet ecosystem. This is important because it ensures that chiplets from different vendors can effectively communicate with one another. The specification covers the die-to-die I/O physical layer, protocols, and the software stack necessary to handle various data types, including PCIe, CXL, Ethernet, and even proprietary protocols.
Chip designers can incorporate UCIe capabilities into their silicon designs for GPUs, accelerators, ASICs, SoCs, FPGAs, etc. This is done by integrating UCIe PHY IP, which aligns with the published specification, using in-house IP or by sourcing from IP vendors such as Cadence, Synopsys, Alphawave Semi, and Blue Cheetah. This then enables the communication between UCIe-compliant chiplets — like the Ayar Labs TeraPHY ™ optical I/O chiplet — and UCIe-compliant GPUs or accelerators.
UCIe standard packages can support traces up to 25 mm on an organic substrate, while advanced packages can support traces up to 2 mm on an interposer substrate. This specification also accommodates longer distances between packages with UCIe-compliant optical retimers. For instance, TeraPHY chiplets can extend the reach between packages up to hundreds of meters while acting as protocol-agnostic retimers. This eliminates reach constraints through optical technology while using UCIe as the PHY layer between chiplets.
In the past two years, UCIe has gained strong industry support from over 100 companies across various sectors, including semiconductors, packaging, IP suppliers, foundries, system vendors, and cloud service providers. This diverse ecosystem aims to address more customizable package-level integration by connecting best-in-class silicon solutions from different vendors using UCIe die-to-die interconnects and protocols.
Why is UCIe so important, and why is it needed today?
The semiconductor industry is experiencing a transformational shift toward multi-die systems. Several trends are driving this shift:
- Manufacturability challenges of monolithic system-on-chips (SoCs)
- Process node optimization for different SoC functionalities
- Enhanced scalability and composability
These trends highlight the need for standardized die-to-die interconnects, which is exactly what UCIe aims to provide.
The current fragmented ecosystem requires unique solutions for each interconnect, which complicates the manufacturing process and impedes the transition from electrical to optical links. If adopted by a wide range of industry players, including potential customers, UCIe would allow for the creation of universally compatible products. This broad adoption would reduce the need for custom solutions for each customer and make it much more feasible for smaller companies to access advanced technologies like optical I/O.
Copper-based solutions limit the range of connectivity between chips to a few inches in the case of packages and printed circuit boards (PCBs) to a few meters in the case of copper cables. Optical I/O-based solutions break this distance bottleneck, enabling the connection of compute and memory chips physically separated by tens to hundreds of meters, with only the fundamental latency caused by the speed of light being the limiting factor. There are several advantages to system designers:
Signal integrity and latency: Copper-based solutions have many orders of magnitude more loss per unit distance than optical solutions. This requires retimers to be placed periodically along a copper-based channel to recover the data signal, each adding significant cost and latency. Think of data traveling by copper as hitting a traffic signal every few inches, while data traveling by optics are on an open, signal-free highway limited only by their speed.
Bandwidth scaling: The insertion loss of copper-based interconnects scales with increasing signal bandwidths, meaning every generation of copper connectivity can travel increasingly shorter distances before the signal needs to be recovered with a retimer. More expensive packages and PCBs (higher quality, or with more layers) are needed to maintain signal integrity over longer distances. Copper cables need to grow larger (thicker) to reduce signal loss, increasing cost and weight while actually reducing bandwidth density. With optics, however, the loss in the fiber does not scale with bandwidth, enabling a more scalable deployment.
Enablement of high performance: The strength of modern computing deployments to enable cutting-edge applications like generative AI stem from the ability to connect large numbers of complex chips to function together as a single unit. These chips increasingly cannot be fit into a single package or board due to physical limitations (in the case of memory) or thermal limitations (in the case of compute). Systems increasingly need these memory and compute units distributed over many racks while still having ultra-high bandwidth, ultra-low latency connectivity between them. Optical I/O enables these use cases in a manner copper cannot.
Ultimately, an open standard like UCIe fosters a more inclusive ecosystem. This is essential for developing manufacturable optical I/O solutions. Without a common standard in place, the industry risks developing isolated, custom solutions that increase costs, hinder volume scaling, and slow overall adoption of optical I/O technology.
Just how exactly does optical I/O work with UCIe?
Optical I/O, such as Ayar Labs’ solution, enhances the functionality and efficiency of UCIe by overcoming some of the most pressing data bottlenecks in modern computing. A TeraPHY chiplet acts as a retimer that converts data streams from the electrical to the optical domain. This process begins by ensuring that a clean electrical signal is sent into an optical link.
The retimer is essential in this context because it fully recovers both the data and the clock, unlike a simpler redriver that only amplifies the signal. By recovering a clean signal, the retimer ensures optimal performance when transitioning from the electrical to the optical domain, which is then transmitted over the optical link.
One of the major advantages of this technology is its ability to overcome data bottlenecks. Optical I/O addresses this challenge by using multiple wavelengths to carry significantly more data within the same physical shoreline. This is crucial for maintaining high throughput in increasingly compressed physical spaces. The UCIe standard complements this by enabling the transport of more data using less space, thereby addressing increasing demands for high bandwidth within a limited footprint.
What can you do with UCIe and optical I/O? What does it enable today?
This is an important question because organizations today face a significant challenge when it comes to scaling bandwidth with copper interconnects. Traditional SerDes-based copper approaches can only go so far in terms of speed and distance. Optical I/O breaks that barrier by offering far superior capabilities in terms of cost, power efficiency, latency, and reach. The combination of UCIe and optical I/O opens a world of possibilities.
With UCIe and optical I/O, you can:
Achieve seamless die-to-die communication for SoCs positioned side by side, each equipped with UCIe PHY and capable of establishing connections over electrical interfaces spanning up to 25 mm. This setup facilitates seamless communication between chips in close proximitys, ensuring high-speed data transfer without the usual distance constraints.
Enable long-distance communications using optical chiplets, covering distances up to 2 km with high bandwidth. Retimers allow SoCs equipped with UCIe PHY to interface with each other using an optical medium. Optical I/O enables data transfer with bandwidth density comparable to or better than NVIDIA’s NVLink. This allows for efficient data exchange across entire boards, racks, or even multiple racks within data centers.
Facilitate memory disaggregation, overcoming the “memory wall” by enabling xPU-to-memory connectivity. Traditional configurations require high bandwidth memory (HBM) to be situated directly next to the GPUs, which poses connectivity challenges as memory needs increase. Optical I/O provides ultra-high bandwidth density and ultra-low latency transmission, which allows for the expansion of memory resources without the typical constraints.
Enhance AI scale-up by creating GPU clusters that function as a single, large GPU — typically over distances spanning a few meters to tens of meters. While scale-out approaches rely on Ethernet or InfiniBand switching networks at data center-scale transmission distances (less than 2 km), AI scale-up requires a chip-to-chip interface like UCIe to effectively manage extremely high bandwidth and low latency data transfer.
As we look ahead, UCIe’s foundational role as a chip-to-chip interface compatible with optical interconnects is clear. Ayar Labs is committed to this vision, with a roadmap that includes multiple generations of UCIe-compliant TeraPHY optical I/O chiplets. These advancements promise to meet the growing needs for higher bandwidth and shoreline density, allowing us to develop a solution that addresses a broad swath of the compute market.
How do AI scale-up, UCIe, and optical I/O come together to benefit users?
As AI models continue to grow in complexity, connectivity becomes critical. To address this, these models are divided into smaller parts and dispersed across many GPUs, which are interconnected to function as a single, unified GPU. Systems often face delays, spending over 70 percent of their time waiting for data. This bottleneck can only be resolved by either enhancing compute and I/O infrastructure or accelerating the I/O itself, with the latter being where optical I/O comes into play.
Ayar Labs’ optical I/O solution is particularly well-suited for next-generation AI scale-up architectures that require ultra-high bandwidth and connectivity across hundreds to even tens of thousands of GPUs. Known under various terms like scale-up fabric, back-end network, and memory-semantic fabric, this setup breaks through the traditional bandwidth-distance limitations. Together, optical I/O and UCIe not only improve utilization but also dramatically reduce latency and power consumption.
The convergence of AI scale-up, UCIe, and optical I/O has resulted in a robust solution to the growing demands of next-generation AI infrastructure — one that offers far superior capabilities in terms of cost efficiency, power efficiency, latency, and reach compared to traditional copper-based approaches or pluggable optics.
So, how do I best integrate UCIe-based optical I/O into my product designs?
Integrating UCIe-based optical I/O into your product designs includes the following elements:
- Incorporating UCIe PHY IP into your silicon designs, either in-house or from established IP vendors.
- Ensuring UCIe interface implementation in both customer chips and TeraPHY chiplets for efficient communication between these components.
- Using tools and methodologies from EDA vendors, packaging houses, and OSATs to simulate and verify seamless integration.
- Implementing test methodologies and tools — similar to those used in PCB manufacturing and testing — for system validation. Standardized interfaces and test protocols play a key role at this stage in ensuring that everything works as expected.
- Collaborating with EDA tool vendors, packaging houses, and OSATs to cohesively design and manufacture the customer’s chip and the optical I/O chiplet within the same package.
By following these steps and leveraging the robust tools and support from the ecosystem, you can successfully integrate UCIe-based optical I/O into your product designs. Doing so, it becomes clear that UCIe-based optical I/O solutions are well positioned to unlock the potential for high-speed, low-latency, and energy-efficient communication between chiplets.
Finally, what is new in the recently announced UCIe 2.0 specification?
The UCIe 2.0 specification introduces several new features to enhance the performance and usability of system-in-package (SiP) constructions with multiple chiplets. Some of the key highlights include:
- Holistic support for manageability, debug, and testing for any SiP construction with multiple chiplets.
- Support for 3D packaging.
- Improved system-level solutions with manageability defined as part of the chiplet stack.
- Optimized package designs for interoperability and compliance testing.
- Full backward compatibility with UCIe 1.1 and UCIe 1.0.
Please refer to the recent press release issued by the UCIe Consortium for more information on the new 2.0 specification. The complete UCIe 2.0 specification is available to the public by request on the UCIe website.
