Moving bits between memory and processor and between chips at a small fraction of today's energy per bit, by shorter and denser electrical links, in-package memory and optical interconnects.
Scope
In scope: energy per bit of memory interfaces and chip-to-chip links, including the transceiver on both ends. Out of scope: the logic that computes on the data, covered by computing/beyond-cmos-logic; the photonic chips themselves, covered by enablers/photonic-integration.
Readiness
TRL 4 (4 of 9)
The best figure found is a 28 nm prototype die-to-die interface; no deployed system figure is sourced yet.
Energy per bit moved headline0.8 orders of magnitude to go
Energy to move one bit between memory and processor, or between chips, including the interface on both ends. Lower is better.
Current (2024-04-01)6.5 × 10⁻¹³ J
Target10⁻¹³ J
Limit–
Conditions. Interface energy of an electrical die-to-die link over a silicon interposer, transceiver only; a prototype, not a deployed memory system.
Why this target. Atlas-set reasoning, not an agency target: at 0.1 pJ/bit, an accelerator streaming 10 TB/s (8e13 bit/s, an assumed HBM-class rate) spends 8 W on the link, against 52 W at the current 0.65 pJ/bit; this brings interconnect energy below a few percent of a kilowatt-class accelerator.
Note. 0.65 pJ/bit. as_of is the approximate preprint month; the abstract gives no date.
Photonic integrationOptical links promise low energy per bit over longer distances than electrical links, and need integrated photonic chips to be made in volume.
Required by
Energy-efficient AI inferenceToken generation reads the model weights and the key-value cache from memory for every token, so memory and chip-to-chip traffic carries a large share of the energy. Need: Energy per bit moved between memory and processor at or below 0.1 pJ, so that streaming weights costs a few watts per terabyte per second.