Limiting Average Criteria for Nonstationary Markov Decision Processes

Peng Shi, Guo Xianping

Research output: Contribution to journalArticlepeer-review


This paper deals with the so-called limiting average criteria for nonstationary Markov decision processes with (possibly unbounded) rewards and Borel state space. A new set of conditions is provided, under which the existence of both a solution to the optimality equations and the limiting average e(= 0)-optimal Markov policies is derived. Also, a rolling horizon algorithm for computing limiting average e(andgt; 0)-optimal Markov policies is developed. Furthermore, the results in this paper are illustrated by several examples such as the water regulation problem.
Original languageEnglish
Pages (from-to)1037 - 1053
Number of pages16
JournalSIAM Journal of Control and Optimization
Issue number4
Publication statusPublished - 30 Jun 2001


  • nonstationary markov decision processes
  • limiting average criteria
  • optimality equations
  • rolling horizon algorithm


Dive into the research topics of 'Limiting Average Criteria for Nonstationary Markov Decision Processes'. Together they form a unique fingerprint.

Cite this