Should buildings learn our behavior? Case study of self-learning hot water systems

Smart Living Lunch

Should buildings learn our behavior? Doctoral researcher Amirreza Heidari (TEBEL and IPESE, EPFL) will present a residential case study in Switzerland. This study aims to develop a control framework for heat pump water heating systems based on Reinforcement Learning, which can learn and adapt to the occupant behavior. On 15 December 2021 from 11:00 to 12:00 at bluefactory Fribourg and online. Zoom link on demand.

Today occupant behavior is known as a key player in building energy use. Occupant behavior is complex, highly stochastic, varies over time, and is different from one building to another. Due to the stochasticity of occupant behavior, the control of building energy systems is detached from the occupant behavior and follows a conservative and energy-intensive approach to ensure occupants' comfort. Variation of occupant behavior over different buildings makes it challenging to develop a universal occupant behavior model that can be integrated into the control method of residential energy systems. Recent advances in the Internet of Things (IoT) on one hand and the development of Machine Learning models on the other hand have made it possible to provide the learning ability to the residential energy systems. Among different Machine Learning models, Reinforcement Learning is of specific interest for residential energy systems as it can learn the stochastic parameters such as occupant behavior or environmental conditions without any model from that specific building and provide energy-saving by continuously adapting the control policy to the stochastic parameters.

Most of the studies on Reinforcement Learning for the built environment have focused on thermal comfort and little attention has been given to the other residential energy systems. Hot water use behavior in buildings is also associated with a high level of stochasticity. Conventional operational strategies of hot water systems, therefore, follow a conservative and energy-intensive approach to meet the stochastic demand. Energy use for hot water production is a major draw in modern energy-efficient buildings.

This study aims to develop a control framework for heat pump water heating systems based on Reinforcement Learning, which can learn and adapt to the occupant behavior and variating heat pump COP and make a balance between energy use, comfort, and hygiene aspects.

This talk presents the primary results of a residential case study in Switzerland. The developed framework is now under development to include other components of the residential energy system and to be evaluated for further case studies.