Ahead-looking: We’re approaching some extent the place conventional copper interconnections will not have the ability to carry sufficient information to maintain GPUs and different specialised chips totally utilized. The AI market is urgently demanding a next-generation answer to this interconnection bottleneck, and Broadcom seems to be engaged on an optics-based answer that’s nearer to the chip itself.

Broadcom is creating new silicon photonics expertise geared toward considerably growing the bandwidth out there to GPUs and different AI accelerators. By using co-packaged optics (CPOs), the fabless chip producer goals to combine optical connectivity elements straight into GPUs, enabling increased information charges whereas concurrently lowering energy necessities.

The corporate has been engaged on CPO options for a number of years and showcased its newest developments on the current Scorching Chips conference. Broadcom’s “optical engine” reportedly delivers a complete interconnect bandwidth of 1.6 TB/sec, equal to six.4 Tbit/sec or 800 GB/sec in every path.

This new connection can present “error-free” information switch to a single chiplet, attaining efficiency ranges akin to Nvidia’s NVLink and different specialised information heart options. Nonetheless, Broadcom has not but integrated its optical interconnections right into a market-ready GPU, such because the A100 or MI250X. As an alternative, it used a check chip designed to emulate an actual GPU for demonstration functions.

In accordance with Manish Mehta, Broadcom’s vp of the optical techniques division, copper connections begin to degrade after simply 5 meters. Whereas optical communications have lengthy been seen as the answer to this sign degradation concern, they historically require way more energy than copper-based applied sciences.

For instance, Nvidia estimates that an optics-powered NVL72 system would require an extra 20 kilowatts per rack, on high of the 120 kilowatts the system already consumes.

Broadcom has managed to cut back energy consumption with the usage of co-packaged optics, which locations particular person transceivers in direct contact with the GPU. The corporate utilized TSMC’s chip-on-wafer-on-substrate (CoWoS) packaging expertise to bond a pair of high-bandwidth reminiscence stacks to the compute die. The logic and reminiscence elements of the chip sit on a silicon interposer, whereas Broadcom’s optical engine is positioned on the substrate.

Mehta defined that CPO expertise may join as much as 512 particular person GPUs throughout eight racks, permitting the whole setup to operate as a single system. As compared, Nvidia’s NVL72 can obtain related unified computing capabilities with “simply” 72 GPUs, suggesting that Broadcom’s answer may finally provide a aggressive benefit for next-generation AI workloads.