Control of Markov Processes Subject to Qualitative Constraints

受定性约束的马尔可夫过程的控制

• 批准号: 0218207
• 负责人: Ari Arapostathis
• 金额: $ 33.4万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2002
• 资助国家: 美国
• 起止时间: 2002-09-01 至 2006-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0218207&HistoricalAwards=false
• 关键词: Control; Markov; Processes; Subject; Qualitative

Discrete event systems (DESs) are systems evolving according to the occurrence of certain discrete qualitative changes, called events. Examples of events include the arrival of a customer in a queue, the termination of an algorithm in a computer program, the loss of a message packet in a communication network, the breakdown of a machine in a manufacturing system.In this proposal we consider stochastic DESs modeled by Markov processes, and study control problems that satisfy qualitative properties. We outline a program of work aiming to develop a unified framework for qualitative control of stochastic DESs. In particular, we propose to study the control of stochastic systems with general qualitative constraints such as safety, non-blocking, recurrence, and stability; optimal control subject to qualitative constraints; control under complete or partial observations; effects of using deterministic versus randomized policies; and effects of using stationary versus non-stationary and asymptotically stationary control policies.Safety constraints are specified as unit-interval-valued vectors serving as an upper bound for the state probability distribution of the controlled Markov chain. Non-blocking specifications require that the probability of hitting a target set of states stays above a certain minimum value. Recurrence or liveness amounts to the probability of hitting a target set of states infinitely-often being bounded below by a positive constant, while convergence or stability demands that the state probability distribution enters and stays in a `safe' set within a finite number of steps.

离散事件系统 (DES) 是根据某些离散的质变（称为事件）的发生而演化的系统。 事件的示例包括队列中顾客的到达、计算机程序中算法的终止、通信网络中消息包的丢失、制造系统中机器的故障。在本提案中，我们考虑随机事件通过马尔可夫过程建模的 DES，并研究满足定性属性的控制问题。 我们概述了一个工作计划，旨在开发一个统一的随机 DES 定性控制框架。 特别是，我们建议研究具有安全性、非阻塞、递归性和稳定性等一般定性约束的随机系统的控制；受定性约束的最优控制；完全或部分观察下的控制；使用确定性策略与随机性策略的效果；以及使用平稳与非平稳和渐近平稳控制策略的效果。安全约束被指定为单位区间值向量，作为受控马尔可夫链状态概率分布的上限。 非阻塞规范要求达到目标状态集的概率保持在某个最小值以上。 重复性或活性相当于无限地达到目标状态集的概率——通常受正常数限制，而收敛或稳定性要求状态概率分布在有限步数内进入并保持在“安全”集中。