Markov Decision Processes: Discrete Stochastic Dynamic Programming
"This text is unique in bringing together so many results
hitherto found only in part in other texts and papers. . . . The
text is fairly self-contained, inclusive of some basic mathematical
results needed, and provides a rich diet of examples, applications,
and exercises. The bibliographical material at the end of each
chapter is excellent, not only from a historical perspective, but
because it is valuable for researchers in acquiring a good
perspective of the MDP research potential."
Zentralblatt fur Mathematik
". . . it is of great value to advanced-level students,
researchers, and professional practitioners of this field to have
now a complete volume (with more than 600 pages) devoted to this
topic. . . . Markov Decision Processes: Discrete Stochastic Dynamic
Programming represents an up-to-date, unified, and rigorous
treatment of theoretical and computational aspects of discrete-time
Markov decision processes."
Journal of the American Statistical Association
2. Model Formulation.
4. Finite-Horizon Markov Decision Processes.
5. Infinite-Horizon Models: Foundations.
6. Discounted Markov Decision Problems.
7. The Expected Total-Reward. Criterion.
8. Average Reward and Related Criteria.
9. The Average Reward Criterion-Multichain and Communicating Models.
10. Sensitive Discount Optimality.
11. Continuous-Time Models.
Appendix A. Markov Chains.
Appendix B. Semicontinuous Functions.
Appendix C. Normed Linear Spaces.
Appendix D. Linear Programming.