Product1 publisher2 min readPublished
Photonics aims at 20 of the 400 tonnes of copper inside a 100MW data centre
Operators are betting light replaces copper between the chips inside a data centre. Whether it does turns on hall temperature and cable routing as much as on an optical cost curve that is still expensive.
The Product Desk · Product desk
What happened
- Chris Sharp, chief technology officer at data centre operator Digital Reality, told the BBC he thinks the industry is at the end of copper, and named the wiring between GPUs and CPUs as what slows compute down.
- A typical 100MW data centre takes around 400 tonnes of copper, most of it in the electrical infrastructure that powers the site and in the cooling systems.
- Up to 70 tonnes of that copper is inside the servers, and up to 20 tonnes is in the network wiring connecting them, which is the copper being targeted for replacement.
- Peter O'Brien of Ireland's Tyndall Research Institute said photonics faced years of manufacturing challenges and is now ready to make the leap from the lab into widespread application.
- Andrew Wheeler of Hewlett Packard Labs said the industry is still working out how to bring optical costs down, with final assembly done in packaging houses clustered in Taiwan.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- cost Early buyers fund the optical learning curve while the electrical alternative already has a settled design, test and deployment cost, so the price gap is carried by whoever moves first.
- exposure A refresh schedule that depends on optical modules inherits the assembly capacity of one cluster of packaging houses in Taiwan.
- decision Bend radius, tray layout and installer retraining become sign-off items. Facilities and field service get a share of a purchase the network team is making.
- constraint The power train and cooling plant keep their 310 tonnes whatever happens between the accelerators, so an operator hoping to cut copper exposure in a build has little to gain here.
The wiring in line for replacement comes to 20 tonnes out of roughly 400 in a 100MW facility, or about 5 per cent of the copper on site [3][6][7]. Another 310 tonnes sit in the electrical distribution and the cooling plant, and photonics leaves those alone [4][8].
The case rests on heat. Light does not carry electricity's heating effects, so less energy goes into cooling [9]. Callum Littlejohns, deputy director of the silicon photonics foundry Cornerstone Labs, said "You can save so much energy" [10]. The BBC report says photonics could make data centres a bit less energy hungry. How much, it does not say [23].
Andrew Wheeler, senior vice president at Hewlett Packard Labs, said of the electrical incumbent: "We've really gotten good at bringing the cost down on that electrical side, how to design it, how to manufacture it, how to test it, how to deploy it" [15]. Nvidia is backing the optical side [14].
Wheeler also described the complication in the cooling story. Optical components are very sensitive to heat, and the rest of the equipment in the hall keeps the local environment hot around them. Reliability then depends on operators and equipment makers staying inside strict thermal limits [18].
The work on the floor changes as well. "You can't take tight turns. There are little nuances on how to structure that," Sharp said of fibre [20]. Installers and field support engineers have to learn new installation and servicing skills, and so do the people designing the network [19].
The buyer here is the operator who pays the cooling bill and signs off the rack layout. Start with which budget the vendor is selling into: Littlejohns makes a cooling-energy claim, and he also says multiple data streams can share one channel, which shows up in capacity and port counts [10][11]. Then ask who controls the air temperature and the cable routing where the optics go. An operator without that control absorbs a reliability risk the energy saving was supposed to pay for [18].
Ofer Shapiro, chief executive of the optical company Resolight.ai, said the benefits are only fully realised when light is used to process data as well as carry it, and his company is proposing an architecture that replaces the traditional electronic network switches between servers with all-optical devices [21][22].
What to watch
- Whether any operator publishes measured cooling energy per rack after an optical retrofit.
- Whether optical component failure rates get reported from halls running warmer than vendors' thermal limits.
- Whether packaging and final assembly capacity for optical modules gets announced outside Taiwan.