LC-Opt: Benchmarking Reinforcement Learning and Agentic AI for End-to-End Liquid Cooling Optimization in Data Centers

AI summary

Overview

Research area: machine learning for data center liquid cooling, reinforcement learning, multi-agent RL, agentic AI, and high-performance computing digital twins. Technical level: Advanced. Scope: LC-Opt is a benchmark environment for developing and comparing RL and agentic AI controllers for end-to-end liquid cooling optimization in data centers.

What This Paper Is About

High-density AI workloads make liquid cooling essential, but controlling it efficiently across an entire data center is complex. The paper introduces LC-Opt, a benchmark built on a high-fidelity digital twin of Oak Ridge National Lab's Frontier Supercomputer cooling system, so RL agents can learn to adjust cooling controls while balancing thermal safety and energy use. The goal is to give researchers, operators, and vendors a shared environment for testing automated liquid cooling control strategies.

Key Contributions

1. LC-Opt benchmark: a high-fidelity Modelica-based digital twin of Frontier's liquid cooling system, spanning site-level cooling towers to data center cabinets and server blade groups, exposed through a Gymnasium interface.
2. End-to-end RL control formulation: agents optimize liquid supply temperature, flow rate, cabinet-level valve actuation, and cooling tower setpoints under dynamic workloads, with optional heat recovery unit support.
3. Multi-objective evaluation: the environment balances local thermal regulation against global energy efficiency in real time, supporting centralized and decentralized multi-agent RL benchmarks.
4. Interpretability and agentic AI: policy distillation into decision and regression trees, plus LLM-based natural-language explanations through an agentic mesh architecture for user trust and system management.

Main Findings

• Benchmark availability: LC-Opt provides a detailed, customizable liquid cooling environment that democratizes access to models otherwise tied to specific HPC facilities.
• Control hierarchy: The benchmark covers IT-level controls such as supply temperature, flow rate, and valve actuation, as well as facility-level cooling tower setpoints.
• Multi-agent comparison: Centralized and decentralized multi-agent RL approaches are benchmarked for the cooling control task.
• Interpretable policies: Learned RL policies can be distilled into decision and regression trees, making control decisions more understandable.
• LLM-based explanation: LLM methods within an agentic mesh architecture can explain control actions in natural language, aiming to increase user trust and simplify management.
• Heat recovery support: The environment can include a heat recovery unit, extending optimization beyond cooling to waste-heat reuse.
• Dynamic workloads: The benchmark accounts for changing computational workloads, making control more realistic for AI and HPC data centers.

Methodology in Plain English

The researchers start with a high-fidelity digital twin of a real supercomputer cooling system. They wrap that simulation in a standard RL interface so agents can observe conditions and take cooling actions. The agents learn to adjust temperatures, flow rates, valves, and cooling tower settings as workloads change. The team then tests centralized and decentralized multi-agent RL controllers, converts some learned policies into simple decision trees so humans can inspect them, and uses large language models to describe what the controllers are doing in plain language. The result is a benchmark for comparing different AI control strategies on the same realistic liquid cooling problem.

Why This Matters

Impact on research: LC-Opt gives the ML community a shared, realistic benchmark for RL and agentic AI in data center cooling, which can accelerate reproducible progress in energy-efficient control. It bridges machine learning with HPC facility operations and sustainability.

Real-world applications:

• AI and HPC data centers with high-density racks that require liquid cooling.
• Supercomputing facilities needing reliable thermal management under variable workloads.
• Data center operators pursuing lower energy consumption, cost, and carbon emissions.
• Cooling vendors and equipment manufacturers designing automated control and heat-recovery systems.

Industry relevance: AI workloads are increasing data center power and cooling demands. Liquid cooling is becoming necessary for high-density computing, but its control is complex across IT and facility systems. LC-Opt offers a way to develop, compare, and explain automated controllers before deployment, which can support reliability, efficiency, and operator trust.

Future Directions

• Validate LC-Opt controllers in live data centers and integrate with real building management and cooling systems.
• Extend benchmarks to more RL and multi-agent algorithms, safety constraints, robustness tests, and standardized energy, thermal, and reliability metrics.
• Improve the LLM agentic mesh for real-time operator decision support, anomaly explanation, and automated system management.
• Scale the environment to heterogeneous data centers, different cooling architectures, water use constraints, heat reuse, and grid-interactive energy optimization.

Target Audience

ML researchers working on reinforcement learning, multi-agent systems, and agentic AI; HPC and data center thermal engineers; facility operators and sustainability managers; cooling system vendors; and industry practitioners interested in digital twins, energy-efficient computing, and interpretable control.

Authors’ abstract

Liquid cooling is critical for thermal management in high-density data centers with the rising AI workloads. However, machine learning-based controllers are essential to unlock greater energy efficiency and reliability, promoting sustainability. We present LC-Opt, a Sustainable Liquid Cooling (LC) benchmark environment, for reinforcement learning (RL) control strategies in energy-efficient liquid cooling of high-performance computing (HPC) systems. Built on the baseline of a high-fidelity digital twin of Oak Ridge National Lab's Frontier Supercomputer cooling system, LC-Opt provides detailed Modelica-based end-to-end models spanning site-level cooling towers to data center cabinets and server blade groups. RL agents optimize critical thermal controls like liquid supply temperature, flow rate, and granular valve actuation at the IT cabinet level, as well as cooling tower (CT) setpoints through a Gymnasium interface, with dynamic changes in workloads. This environment creates a multi-objective real-time optimization challenge balancing local thermal regulation and global energy efficiency, and also supports additional components like a heat recovery unit (HRU). We benchmark centralized and decentralized multi-agent RL approaches, demonstrate policy distillation into decision and regression trees for interpretable control, and explore LLM-based methods that explain control actions in natural language through an agentic mesh architecture designed to foster user trust and simplify system management. LC-Opt democratizes access to detailed, customizable liquid cooling models, enabling the ML community, operators, and vendors to develop sustainable data center liquid cooling control solutions.