News

Microsoft's Next Datacenter Cooling Experiment: Liquid Immersion

• By Kurt Mackie
• April 06, 2021

Continuing its research into different datacenter cooling methods, Microsoft this week said it's exploring liquid immersion cooling tanks to run heavy workloads while keeping server blades cool.

These tanks are not filled with water, but with liquid made by 3M that has "dielectric properties," permitting the servers to run "overclocked" while venting heat through the fluid. The liquid boils at a lower temperature (122 degrees Fahrenheit) than the boiling point of water.

Microsoft calls it them "two-phase" immersion cooling tanks because "the vapor rising from the boiling fluid contacts a cooled condenser in the tank lid, which causes the vapor to change to liquid and rain back onto the immersed servers, creating a closed loop cooling system."

Better Than Air Cooling
Microsoft is currently conducting the cooling experiments in its Quincy, Wash. datacenters. The idea behind it is that fluids are better heat conductors than more traditional air-cooled systems, which have reached their technological limits. The liquid cooling effort is yet another attempt to advance beyond Moore's Law limits, but at the datacenter level.

"Liquid cooling enables us to go denser, and thus continue the Moore's Law trend at the datacenter level," explained Christian Belady, a Microsoft Distinguished Engineer and vice president of the datacenter advanced development group in Redmond, Wash.

Microsoft compared the use of these cooling tanks to its own undersea server cooling experiments with Project Natick. In those experiments, a submersible container filled with servers was set down on the cold sea-bottom floor.

"We brought the sea to the servers rather than put the datacenter under the sea," explained Husam Alissa, a principal hardware engineer on Microsoft's datacenter advanced development team, regarding the two-phase liquid immersion tanks.

Alissa also said that "we are the first cloud provider that is running two-phase immersion cooling in a production environment." More generally, Microsoft noted that it's been the cryptocurrency industry that pioneered liquid immersion cooling, typically to track currency transactions.

In addition to using 3M's cooling liquid, Microsoft worked with "datacenter IT system manufacturer and designer" Wiwynn to build its two-phase immersion cooling system.

Practical Applications
Microsoft is currently experimenting with using the cooling tanks to deal with bursty traffic, such as the traffic generated by Microsoft Teams meetings when people meet at the same time. The cooling solution is also seen as being useful to address high-performance Microsoft Azure artificial intelligence and machine learning workloads.

With the experiments, Microsoft is examining the notion that the cooling tanks will reduce failures. If so, it could make them suitable when deploying servers in "remote, hard-to-service locations." However, they may not be able outdo the failure reductions seen with Project Natick.

With Project Natick, Microsoft pumped the undersea containers with nitrogen, which reduced corrosive effects.

"A key finding from Project Natick is that the servers on the seafloor experienced one-eighth the failure rate of replica servers in a land datacenter," Microsoft explained. "Preliminary analysis indicates that the lack of humidity and corrosive effects of oxygen were primarily responsible for the superior performance of the servers underwater."