Markov Decision Processes

Markov decision processes (MDPs), also called stochastic dynamic programming, were born in 1960s. MDPs model and solve dynamic decision-making problems with multi-periods under stochastic circumstances. There are three basic branches in MDPs: discrete time MDPs, continuous time MDPs, and semi-Markov decision processes. Based on these branches, many generalized MDP models were presented to model various practical problems, such as partially observable MDPs, adaptive MDPs, MDPs in stochastic environments, and MDPs with multiple ob jectives, constraints, or imprecise parameters. MDPs have been applied in many areas, such as communications, signal processing, artificial intelligence, stochastic scheduling and manufacturing systems, discrete event systems, management, and economics. Examines several fundamentals concerning the manner in which Markov decision problems may be properly formulated and the determination of solutions or their properties. Coverage includes optimal equations, algorithms and their characteristics, probability distributions, modern development in the Markov decision process area, namely structural policy analysis, approximation modeling, multiple objectives and Markov games. Copiously illustrated with examples. jectives, constraints, or imprecise parameters. MDPs have been applied in many areas, such as communications, signal processing, artificial intelligence, stochastic scheduling and manufacturing systems, discrete event systems, management, and economics. Examines several fundamentals concerning the manner in which Markov decision problems may be properly formulated and the determination of solutions or their properties. Coverage includes optimal equations, algorithms and their characteristics, probability distributions, modern development in the Markov decision process area, namely structural policy analysis, approximation modeling, multiple objectives and Markov games. Copiously illustrated with examples.