Liquid Cooling Big Picture Overview

The stunningly artistic picture below illustrates how liquid cooling works in general. There are some variations, which we'll go into a little later, but this is a fantastic depiction of the components and process. (Note:  None of the images below are to scale or, in at least one case, even realistic. Hard to believe I did this myself in PowerPoint, right?)

Step 1:  Pumps circulate liquid through each node in the rack in a closed loop. With immersion cooling, this would be the immersion enclosure, we'll show some pictures later on. The liquid circulating inside the rack isn't strictly water, it's at least water with some additives to aid cooling, but in some cases a synthetic liquid with un-water-like properties.

Step 2:  This is a conceptual, and poorly executed, view of a CDU (Coolant Distribution Unit). It's a liquid-to-liquid heat exchanger where cool facility water flows through (the blue area), picks up heat from the server liquid loop, and transports it out of the data center. Important notes:

• First, one CDU can handle many racks, it's typically not a one rack to one CDU relationship.
• CDUs come in a wide variety of form factors and capacities. Some are at the end of rows with piping running under or over the racks. Others are entirely under a raised floor.
• CDUs all have instrumentation monitoring and controlling liquid flows and temperatures.

Step 3:  Now we're up on the roof or somewhere else outside the building. The now warm water leaves the CDU(s) and is pumped through a radiator where high velocity air cools the water, thus removing the collected heat from the systems. This unit is generally called a "dry cooler" or "cooling tower" or something along those lines. Important notes:

• This is 'free cooling' since the fans and pumps use far less power than chillers/condensers. When your outside air is colder than the temperature you need for your facility water loop, then a dry cooler is a great solution.
• The temperature of the liquid in the facility water cooling loop doesn't have to be all that cold. ASHRAE and others say it can be as warm as 89F (31C) and still provide enough cooling capability for systems. Unless your outside ambient temperatures are pretty high, you will get serious benefits from a dry cooler.
• Variable speed fans will increase dry cooler energy efficiency.
• However, the temperature you need for your facility water depends on the number of systems you're trying to cool and their configs. As TDPs on CPUs rise above 500 watts and on GPUs above 700 watts, the water coming out of the cooling tower and heading to the systems will need to be closer to 80F (26.6C).

Step 4:  In our simple (and bad) illustration above, the lawn sprinkler represents Adiabatic Cooling, which adds more cooling punch to your dry cooler. Imagine that the radiator in Step 3 above is surrounded by wet porous material. As the fan draw in air, it goes through this material and is pre-cooled and humidified. Cooler and more humid air will carry significantly more heat away as it is pushed through the facility water radiator. This is a great solution in hot, dry, climates and in places where the rooftop ambient temperatures are very high. Important notes:  

• Water is only used on the dry tower when it's needed, and it's not a lot of water, a little bit goes a long way when it comes to pre-cooling air and adding humidity.
• Adiabatic coolers can be smaller than dry coolers because of the performance they can provide. This makes them an easier fit and takes better advantage of roof top real estate.
• You have to factor in not only energy costs, but also water consumption and waste water costs, depending on climate, scale, usage and other factors, the water costs can be significant.

Step 5:  The refrigerator in the picture represents traditional water chillers, which have long been used to drop liquid temperatures for data centers and a wide variety of other industrial needs. They use compressors, condensers, and refrigerant to remove heat and return cooler liquid to the facility loop - which is a pretty energy intensive process when compared to a dry cooler and adiabatic cooling. Important notes:

• Chillers are quite a bit more complicated, and thus more expensive to purchase than dry coolers and dry coolers plus adiabatic coolers.
• Operating costs for chillers are higher than for the others too. The refrigeration is a much more energy intensive process.
• With dry coolers and, if needed, adiabatic add-ons, the load on your chillers will be much lower, meaning lower costs.
• However, even with all of the above, there is still a place for chillers, but as a failsafe, a last resort source of cooling.