Limiting Average Criteria for Nonstationary Markov Decision Processes

This paper deals with the so-called limiting average criteria for nonstationary Markov decision processes with (possibly unbounded) rewards and Borel state space. A new set of conditions is provided, under which the existence of both a solution to the optimality equations and the limiting average e(= 0)-optimal Markov policies is derived. Also, a rolling horizon algorithm for computing limiting average e(andgt; 0)-optimal Markov policies is developed. Furthermore, the results in this paper are illustrated by several examples such as the water regulation problem.