A Learning Agent for HVAC Thermostat Control

Computing & Wireless : Application Software

Available for licensing


  • Peter Stone, Ph.D. , Computer Science
  • Daniel Urieli , University of Texas at Austin

Background/unmet need

Heating, ventilation and air donditioning (HVAC) systems are some of the biggest energy consumers around the world. Heat pump-based HVAC systems have gained popularity due to their high efficiency and because they are powered by electricity rather than by oil or gas. However, one drawback is that is that their efficiency sharply declines when the outdoor temperature is around or below freezing. Therefore, they are backed up by an auxiliary heating system that is effective in cold weather but that consumes twice as much energy.

A popular way of saving energy in HVAC systems is setting back the thermostat, which means to relax the heating and cooling requirements when occupants are not at home. Although this practice leads to significant energy savings in many systems, it may increase the energy consumption in a heat-pump based system using existing control strategies, since it forces an excessive usage of the auxiliary heater.

Invention Description

Researchers at The University of Texas at Austin have designed a complete, adaptive reinforcement learning agent which applies a new control strategy for heat-pump HVAC systems. This learning agent can decrease energy consumption, adapt to the properties of a specific house, and learn to save energy while satisfying comfort settings. As proof of concept, a realistic simulator developed for the U.S. Department of Energy was used to test the learning agent. Results show that the learned control strategy leads to roughly 7% to 15% energy savings annually in weather conditions imitating those recorded in the New York City, Boston, and Chicago areas.


  • Provides savings for major energy consumers
  • Uses adaptive technology, can tailor services for a wide range of conditions
  • Designed to work in real-time inside a working thermostat


  • Coded in C++
  • Can work with heat-pump systems
  • Interfaces with the thermostat and a weather forecast source
  • Online re-planning using an efficient tree-search algorithm; responsive to changing conditions

Market potential/applications

Smart thermostats, thermostat control software

Development Stage

Lab/bench prototype