Print this page Share

Understanding Computational Bayesian Statistics

ISBN: 978-0-470-04609-8
336 pages
December 2009
Understanding Computational Bayesian Statistics (0470046090) cover image


A hands-on introduction to computational statistics from a Bayesian point of view

Providing a solid grounding in statistics while uniquely covering the topics from a Bayesian perspective, Understanding Computational Bayesian Statistics successfully guides readers through this new, cutting-edge approach. With its hands-on treatment of the topic, the book shows how samples can be drawn from the posterior distribution when the formula giving its shape is all that is known, and how Bayesian inferences can be based on these samples from the posterior. These ideas are illustrated on common statistical models, including the multiple linear regression model, the hierarchical mean model, the logistic regression model, and the proportional hazards model.

The book begins with an outline of the similarities and differences between Bayesian and the likelihood approaches to statistics. Subsequent chapters present key techniques for using computer software to draw Monte Carlo samples from the incompletely known posterior distribution and performing the Bayesian inference calculated from these samples. Topics of coverage include:

  • Direct ways to draw a random sample from the posterior by reshaping a random sample drawn from an easily sampled starting distribution
  • The distributions from the one-dimensional exponential family
  • Markov chains and their long-run behavior
  • The Metropolis-Hastings algorithm
  • Gibbs sampling algorithm and methods for speeding up convergence
  • Markov chain Monte Carlo sampling

Using numerous graphs and diagrams, the author emphasizes a step-by-step approach to computational Bayesian statistics. At each step, important aspects of application are detailed, such as how to choose a prior for logistic regression model, the Poisson regression model, and the proportional hazards model. A related Web site houses R functions and Minitab macros for Bayesian analysis and Monte Carlo simulations, and detailed appendices in the book guide readers through the use of these software packages.

Understanding Computational Bayesian Statistics is an excellent book for courses on computational statistics at the upper-level undergraduate and graduate levels. It is also a valuable reference for researchers and practitioners who use computer programs to conduct statistical analyses of data and solve problems in their everyday work.

See More

Table of Contents


1 Introduction to Bayesian Statistics.

1.1 The Frequentist Approach to Statistics.

1.2 The Bayesian Approach to Statistics.

1.3 Comparing Likelihood and Bayesian Approaches to Statistics.

1.4 Computational Bayesian Statistics.

1.5 Purpose and Organization of This Book.

2 Monte Carlo Sampling from the posterior.

2.1 Acceptance-Rejection-Sampling.

2.2 Sampling-Importance-Resampling.

2.3 Adaptive-Rejection-Sampling from a Log-Concave Distribution.

2.4 Why Direct Methods are Inefficient for High-Dimension Parameter Space.

3 Bayesian Inference.

3.1 Bayesian Inference from the Numerical Posterior.

3.2 Bayesian Inference from Posterior Random Sample.

4 Bayesian Statistics using Conjugate Priors.

4.1 One-Dimensional Exponential Family of Densities.

4.2 Distributions for Count Data.

4.3 Distributions for Waiting Times.

4.4 Normally Distributed Observations with Known Variance.

4.5 Normally Distributed Observations with Known Mean.

4.6 Normally Distributed Observations with Unknown Mean and Variance.

4.7 Multivariate Normal Observations with Known Covariance Matrix.

4.8 Observations from Normal Linear Regression Model.

Appendix: Proof of Poisson Process Theorem.

5 Markov Chains.

5.1 Stochastic Processes.

5.2 Markov Chains.

5.3 Time-Invariant Markov Chains with Finite State Space.

5.4 Classification of States of a Markov Chain.

5.5 Sampling from a Markov Chain.

5.6 Time-Reversible Markov Chains and Detailed Balance.

5.7 Markov Chains with Continuous State Space.

6 Markov Chain Monte Carlo Sampling from Posterior.

6.1 Metropolis-Hastings Algorithm for a Single Parameter.

6.2 Metropolis-Hastings Algorithm for Multiple Parameters.

6.3 Blockwise Metropolis-Hastings Algorithm.

6.4 Gibbs Sampling .

6.5 Summary.

7 Statistical Inference from a Markov Chain Monte Carlo Sample.

7.1 Mixing Properties of the Chain.

7.2 Finding a Heavy-Tailed Matched Curvature Candidate Density.

7.3 Obtaining An Approximate Random Sample For Inference.

Appendix: Procedure for Finding the Matched Curvature Candidate Density for a Multivariate Parameter.

8 Logistic Regression.

8.1 Logistic Regression Model.

8.2 Computational Bayesian Approach to the Logistic Regression Model.

8.3 Modelling with the Multiple Logistic Regression Model.

9 Poisson Regression and Proportional Hazards Model.

9.1 Poisson Regression Model.

9.2 Computational Approach to Poisson Regression Model.

9.3 The Proportional Hazards Model.

9.4 Computational Bayesian Approach to Proportional Hazards Model.

10 Gibbs Sampling and Hierarchical Models.

10.1 Gibbs Sampling Procedure.

10.2 The Gibbs Sampler for the Normal Distribution.

10.3 Hierarchical Models and Gibbs Sampling.

10.4 Modelling Related Populations with Hierarchical Models.

Appendix: Proof that Improper Jeffrey’s Prior Distribution for the Hypervariance Can Lead to an Improper Posterior.

11 Going Forward with Markov Chain Monte Carlo.

A Using the Included Minitab Macros.

B Using the Included R Functions.


Topic Index.

See More

Author Information

William M. Bolstad, PhD, is Senior Lecturer in the Department of Statistics at The University of Waikato (New Zealand). Dr. Bolstad's research interests include Bayesian statistics, MCMC methods, recursive estimation techniques, multiprocess dynamic time series models, and forecasting. He is the author of Introduction to Bayesian Statistics, Second Edition, also published by Wiley.
See More

The Wiley Advantage

  • The coverage of topics is similar in range to a traditional statistics coursebook.
  • Exercises are provided throughout the book, along with selected answers.

  • Each summer concludes with a summary of main points, helping readers understand the key principles related to each discussion.

  • Bayesian statistics is now more appropriate than ever because computer programs enable the practitioner to work on problems that contain many parameters

  • The author supplies a calculus refresher anda summary on the use of statistical tables to provide the necessary background information.

  • A related Web site provides R functions and Minitab macros for Bayesian analysis and Monte Carlo simulations.

See More


"Understanding computational Bayesian statistics is an excellent book for courses on computational statistics at the advanced undergraduate and graduate levels. It is also a valuable reference for researchers and practitioners who use computer programs to conduct statistical analyses of data and solve problems in their everyday work." (Mathematical Reviews, 2011)
See More

Buy Both and Save 25%!


Understanding Computational Bayesian Statistics (US $142.00)

-and- Data Mining and Statistics for Decision Making (US $99.95)

Total List Price: US $241.95
Discounted Price: US $181.46 (Save: US $60.49)

Buy Both
Cannot be combined with any other offers. Learn more.
Back to Top