Learning-Based Controller Design with Application to a Chiller Process

Sammanfattning: In this thesis, we present and study a few approaches for constructing controllers for uncertain systems, using a combination of classical control theory and modern machine learning methods. The thesis can be divided into two subtopics. The first, which is the focus of the first two papers, is dual control. The second, which is the focus of the third and last paper, is multiple-input multiple-output (MIMO) control of a chiller process. In dual control, the goal is to construct controllers for uncertain systems that in expectation minimize some cost over a certain time horizon. To achieve this, the controller must take into account the dual goals of accumulating more information about the process, by applying some probing input, and using the available information for controlling the system. This is referred to as the exploration-exploitation trade-off. Although optimal dual controllers in theory can be computed by solving a functional equation, this is usually intractable in practice, with only some simple special cases as exceptions. Therefore, it is interesting to examine methods for approximating optimal dual control. In the first paper, we take the approach of approximating the value function, which is the solution of the functional equation that can be used to deduce the optimal control, by using artificial neural networks. In the second paper, neural networks are used to represent and estimate hyperstates, which contain information about the conditional probability distributions of the system uncertainties. The optimal dual controller is a function of the hyperstate, and hence it should be useful to have a representation of this quantity when constructing an approximately optimal dual controller. The hyperstate transition model is used in combination with a reinforcement learning algorithm for constructing a dual controller from stochastic simulations of a system model that includes models of the system uncertainties. In the third paper, we suggest a simple reinforcement learning method that can be used to construct a decoupling matrix that allows MIMO control of a chiller process. Compared to the commonly used single-input single-output (SISO) structures, these controllers can decrease the variations in some system signals. This makes it possible to run the system at operating points closer to some constraints, which in turn can enable more energy-efficient operation.