Stochastic Optimal Control based on Gaussian Processes Regression

基于高斯过程回归的随机最优控制

基本信息

批准号：
349395379
负责人：
Professor Dr.-Ing. Uwe D. Hanebeck
金额：
--
依托单位：
Lehrstuhl für Intelligente Sensor-Aktor-Systeme (ISAS)
依托单位国家：
德国
项目类别：
Research Grants
财政年份：
2017
资助国家：
德国
起止时间：
2016-12-31 至 2020-12-31
项目状态：
已结题

来源：
https://gepris.dfg.de/gepris/projekt/349395379?language=en
关键词：
Stochastic Optimal Control based Gaussian

项目摘要

In stochastic control, optimal decision making in continuous domains under statistically modeled uncertainty is usually addressed via Dynamic Programming (DP). The goal consists in finding policies that map the information available to the controller to a control input in such a way that a performance criterion, often defined in terms of costs, is optimized. Usually, using nonlinear filtering methods, this information is condensed into a probability distribution that represents the state estimate of the system to be controlled, and the policies map these distributions to control inputs.Unfortunately, DP is intractable except in a few very special cases. Therefore, approximate but tractable approaches are of interest. One such approach is the point-based value iteration algorithm, where each point is a probability distribution. In this approach, the controller maintains the optimal costs for a set of representative state estimates instead of trying the impossible task of maintaining the costs for all state estimates as it would be required in classical DP. Then, it uses this information in order to obtain an approximation of the optimal costs at a state estimate that is needed for decision making. As we see, point-based value iteration requires approximation methods for functions defined over general probability distributions. However, state-of-the-art approaches either restrict the class of possible state estimates or assume finite sets of control inputs and measurements. Although workarounds for continuous control inputs and measurements exist, they usually require additional approximations. For this reason, we propose a novel approach to stochastic control of nonlinear dynamical systems with continuous states, control inputs, and measurements that is based on Gaussian Process (GP) regression. Classical GP regression only allows for deterministic vector-valued inputs. For this reason, we propose a novel extension of the GP framework to inputs given in form of probability distributions. By doing so, we extend the GP framework to infinite-dimensional inputs. Our approach is based on the idea to define the covariance functions that determine the GP in terms of the distance between the probability distributions provided as inputs to the GP.In the course of the project, we plan to develop a solid framework for GPs defined over general probability distributions and to derive stochastic control algorithms that use such GPs to compute the policy. We believe that the proposed project will substantially contribute to research on stochastic control. Furthermore, the presented idea for defining GPs with inputs given in terms of probability distributions can also be used in machine learning research in order to derive other non-parametric Bayesian regression and classification methods over probability distributions.

在随机控制中，通常通过动态编程（DP）来解决统计建模不确定性下连续域中的最佳决策。目标包括查找将可用的信息映射到控制器的控制输入的策略，以使通常根据成本定义的性能标准优化。通常，使用非线性过滤方法，将此信息凝结成代表要控制的系统的状态估计的概率分布，并且策略将这些分布映射到控制输入。因此，大约但可进行的方法是感兴趣的。一种方法是基于点的值迭代算法，其中每个点都是概率分布。在这种方法中，控制器维持一组代表性状态估计的最佳成本，而不是尝试不可能的任务来维持所有状态估计的成本，因为它在经典DP中需要。然后，它使用此信息来在决策所需的州估算中获得最佳成本的近似值。如我们所见，基于点的价值迭代需要针对一般概率分布定义的函数的近似方法。但是，最新的方法要么限制可能的状态估计等级，要么假设控制输入和测量值有限。尽管存在连续控制输入和测量值的解决方法，但它们通常需要额外的近似值。因此，我们提出了一种新的方法，可以对具有连续状态，控制输入和基于高斯过程（GP）回归的非线性动力学系统进行随机控制。经典的GP回归仅允许确定性矢量值输入。因此，我们提出了以概率分布形式给出的输入的新型GP框架的新扩展。通过这样做，我们将GP框架扩展到无限维输入。我们的方法是基于定义协方差函数的想法一般概率分布并得出使用此类GP来计算策略的随机控制算法。我们认为，拟议的项目将大大有助于对随机控制的研究。此外，在机器学习研究中，还可以使用以概率分布来定义输入的GP的想法，以便得出其他非参数贝叶斯回归和分类方法，而不是概率分布。

项目成果

期刊论文数量（3）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Stochastic Optimal Control Using Gaussian Process Regression over Probability Distributions

DOI：
10.23919/acc.2019.8814658
发表时间：
2019-07
期刊：
2019 American Control Conference (ACC)
影响因子：
0
作者：
Jana Mayer;Maxim Dolgov;Tobias Stickling;Selim Özgen;Florian Rosenthal;U. Hanebeck
通讯作者：
Jana Mayer;Maxim Dolgov;Tobias Stickling;Selim Özgen;Florian Rosenthal;U. Hanebeck

Position and Speed Estimation of PMSMs Using Gaussian Processes

使用高斯过程估计 PMSM 的位置和速度