IBM to Speed Up GenAI with Optics Solution

IBM unveiled new research in optics technology that could dramatically improve how data centers train and run generative AI models. Researchers have pioneered a new process for co-packaged optics (CPO), the next generation of optics technology, to enable connectivity within data centers at the speed of light through optics to complement existing short-reach electrical wires.

By designing and assembling the first publicly announced successful polymer optical waveguide (PWG) to power this technology, IBM researchers have shown how CPO will redefine the way the computing industry transmits high-bandwidth data between chips, circuit boards, and servers. Although data centers use fiber optics for their external communications networks, racks in data centers still predominantly run communications on copper-based electrical wires. These wires connect GPU accelerators that may spend more than half of their time idle, waiting for data from other devices in a large, distributed training process which can incur significant expense and energy.

Researchers have demonstrated a way to bring optics’ speed and capacity inside data centers. In a technical paper, IBM introduces a new CPO prototype module that can enable high-speed optical connectivity. This technology could significantly increase the bandwidth of data center communications, minimizing GPU downtime while drastically accelerating AI processing. This research innovation, as described, would enable lower costs for scaling GenAI, faster AI model training, and dramatically increased energy efficiency for data centers.

“As GenAI demands more energy and processing power, the data center must evolve and co-packaged optics can make these data centers future-proof,” said Dario Gil, SVP and Director of Research at IBM. “With this breakthrough, tomorrow’s chips will communicate much like how fiber optics cables carry data in and out of data centers, ushering in a new era of faster, more sustainable communications that can handle the AI workloads of the future.”

In recent years, advances in chip technology have densely packed transistors onto a chip. IBM’s 2-nanometer node chip technology can contain more than 50 billion transistors. CPO technology aims to scale the interconnection density between accelerators by enabling chipmakers to add optical pathways connecting chips on an electronic module beyond the limits of today’s electrical pathways. The research paper outlines how these new high bandwidth density optical structures, coupled with transmitting multiple wavelengths per optical channel, have the potential to boost bandwidth between chips as much as 80 times compared to electrical connections.

This innovation, as described, would enable chipmakers to add six times as many optical fibers at the edge of a silicon photonics chip, called “beachfront density,” compared to the current state-of-the-art CPO technology. Each fiber, about three times the width of a human hair, could span centimeters to hundreds of meters in length and transmit terabits of data per second. The IBM team assembled a high-density PWG at 50-micrometer pitch optical channels, adiabatically coupled to silicon photonics waveguides, using standard assembly packaging processes.

The paper additionally indicates that these CPO modules with PWG at 50-micrometer pitch are the first to pass all stress tests required for manufacturing. Components are subjected to high-humidity environments and temperatures ranging from -40°C to 125°C, as well as mechanical durability testing to confirm that optical interconnects can bend without breaking or losing data. Moreover, researchers have demonstrated PWG technology to an 18-micrometer pitch. Stacking four PWGs would allow for up to 128 channels for connectivity at that pitch.