Markov Chains: From Theory to Implementation and ExperimentationISBN: 9781119387558
240 pages
July 2017

Description
A fascinating and instructive guide to Markov chains for experienced users and newcomers alike
This unique guide to Markov chains approaches the subject along the four convergent lines of mathematics, implementation, simulation, and experimentation. It introduces readers to the art of stochastic modeling, shows how to design computer implementations, and provides extensive worked examples with case studies.
Markov Chains: From Theory to Implementation and Experimentation begins with a general introduction to the history of probability theory in which the author uses quantifiable examples to illustrate how probability theory arrived at the concept of discretetime and the Markov model from experiments involving independent variables. An introduction to simple stochastic matrices and transition probabilities is followed by a simulation of a twostate Markov chain. The notion of steady state is explored in connection with the longrun distribution behavior of the Markov chain. Predictions based on Markov chains with more than two states are examined, followed by a discussion of the notion of absorbing Markov chains. Also covered in detail are topics relating to the average time spent in a state, various chain configurations, and nstate Markov chain simulations used for verifying experiments involving various diagram configurations.
 Fascinating historical notes shed light on the key ideas that led to the development of the Markov model and its variants
 Various configurations of Markov Chains and their limitations are explored at length
 Numerous examples—from basic to complex—are presented in a comparative manner using a variety of color graphics
 All algorithms presented can be analyzed in either Visual Basic, Java Script, or PHP
 Designed to be useful to professional statisticians as well as readers without extensive knowledge of probability theory
Covering both the theory underlying the Markov model and an array of Markov chain implementations, within a common conceptual framework, Markov Chains: From Theory to Implementation and Experimentation is a stimulating introduction to and a valuable reference for those wishing to deepen their understanding of this extremely valuable statistical tool.
Table of Contents
Abstract
Dedication
Acknowledgements
Preface
Chapter 1: Historical Notes
1.1 Introduction
1.2 On The Wings Of Dependent Variables
1.3 From Bernoulli To Markov
Chapter 2: From Observation To Simulation
2.1 Introduction
2.2 Stochastic Matrices
2.3 Transition Probabilities
2.4 The Simulation Of A TwoState Markov Chain
Chapter 3: Building The Stochastic Matrix
3.1 Introduction
3.2 Building The Stochastic Matrix From Events
3.3 Building The Stochastic Matrix From Percentages
Chapter 4: Predictions By Using 2State Markov Chains
4.1 Introduction
4.2 Performing The Predictions By Using The Stochastic Matrix
4.3 The Steady State Of A Markov Chain
4.4 The LongRun Distribution Of A Markov Chain
Chapter 5: Predictions By Using NState Markov Chains
5.1 Introduction
5.2 Predictions By Using The 3State Markov Chain
5.3 Predictions By Using The 4State Markov Chain
5.4 Predictions By Using NState Markov Chains
5.5 Markov Chain Modeling On Measurements
Chapter 6: Absorbing Markov Chains
6.1 Introduction
6.2 The Absorbing State
Chapter 7: The Average Time Spent In Each State
7.1 Introduction
7.2 The Proportion Of Balls In The System
7.3 The Average Time Spent In A Particular State
7.4 Exemplification Of The Average Time And Proportions
Chapter 8: Discussions On Different Configurations Of Chains
8.1 Introduction
8.2 Examples Of Two State Diagrams
8.3 Examples Of Three State Diagrams
8.4 Examples Of Four State Diagrams
8.5 Examples Of State Diagrams Divided Into Classes
8.6 Examples Of State Diagrams With Absorbing States
8.7 The Gambler's Ruin
Chapter 9: The Simulation Of An NState Markov Chain
9.1 Introduction
9.2 The Simulation Of Behavior
9.3 Simulation Of Different Chain Configurations
Glossary
Bibliography
Appendices
Appendix A: Supporting Algorithms In Php
Appendix B: Supporting Algorithms In Javascript
Appendix C: Syntax Equivalence Between Languages
Author Information
Paul A. Gagniuc, PhD, is Associate Professor at Polytechnic University of Bucharest, Romania. He obtained his MS and his PhD in genetics at the University of Bucharest. Dr. Ganiuc’s work has been published in numerus high profile scientific journals, ranging from the Public Library of Science to BioMed Central and Nature journals. He is the recipient of several awards for exceptional scientific results and a highly active figure in the review process for different scientific areas.