Textbook
Introduction to MetaAnalysisApril 2009, ©2008

 Outlines the role of metaanalysis in the research process
 Shows how to compute effects sizes and treatment effects
 Explains the fixedeffect and randomeffects models for synthesizing data
 Demonstrates how to assess and interpret variation in effect size across studies
 Clarifies concepts using text and figures, followed by formulas and examples
 Explains how to avoid common mistakes in metaanalysis
 Discusses controversies in metaanalysis
 Features a web site with additional material and exercises
A superb combination of lucid prose and informative graphics, written by four of the world’s leading experts on all aspects of metaanalysis. Borenstein, Hedges, Higgins, and Rothstein provide a refreshing departure from cookbook approaches with their clear explanations of the what and why of metaanalysis. The book is ideal as a course textbook or for selfstudy. My students, who used prepublication versions of some of the chapters, raved about the clarity of the explanations and examples. David Rindskopf, Distinguished Professor of Educational Psychology, City University of New York, Graduate School and University Center, & Editor of the Journal of Educational and Behavioral Statistics.
The approach taken by Introduction to Metaanalysis is intended to be primarily conceptual, and it is amazingly successful at achieving that goal. The reader can comfortably skip the formulas and still understand their application and underlying motivation. For the more statistically sophisticated reader, the relevant formulas and worked examples provide a superb practical guide to performing a metaanalysis. The book provides an eclectic mix of examples from education, social science, biomedical studies, and even ecology. For anyone considering leading a course in metaanalysis, or pursuing selfdirected study, Introduction to Metaanalysis would be a clear first choice. Jesse A. Berlin, ScD
Introduction to MetaAnalysis is an excellent resource for novices and experts alike. The book provides a clear and comprehensive presentation of all basic and most advanced approaches to metaanalysis. This book will be referenced for decades. Michael A. McDaniel, Professor of Human Resources and Organizational Behavior, Virginia Commonwealth University
List of Tables
Acknowledgements
Preface
PART 1: INTRODUCTION
1 HOW A METAANALYSIS WORKS
Introduction
Individual studies
The summary effect
Heterogeneity of effect sizes
Summary points
2 WHY PERFORM A METAANALYSIS
Introduction
The SKIV metaanalysis
Statistical significance
Clinical importance of the effect
Consistency of effects
Summary points
PART 2: EFFECT SIZE AND PRECISION
3 OVERVIEW
Treatment effects and effect sizes
Parameters and estimates
Outline
4 EFFECT SIZES BASED ON MEANS
Introduction
Raw (unstandardized) mean difference D
Standardized mean difference, D and G
Response ratios
Summary points
5 EFFECT SIZES BASED ON BINARY DATA (2×2 TABLES)
Introduction
Risk ratio
Odds ratio
Risk difference
Choosing an effect size index
Summary points
6 EFFECT SIZES BASED ON CORRELATIONS
Introduction
Computing R
Other approaches
Summary points
7 CONVERTING AMONG EFFECT SIZES
Introduction
Converting from the log odds ratio to D
Converting from D to the log odds ratio
Converting from R to D
Converting from D to R
Summary points
8 FACTORS THAT AFFECT PRECISION
Introduction
Factors that affect precision
Sample size
Study design
Summary points
9 CONCLUDING REMARKS
Further reading
PART 3: FIXEDEFFECT VERSUS RANDOMEFFECTS MODELS
10 OVERVIEW
Introduction
Nomenclature
11 FIXEDEFFECT MODEL
Introduction
The true effect size
Impact of sampling error
Performing a fixedeffect metaanalysis
Summary points
12 RANDOMEFFECTS MODEL
Introduction
The true effect sizes
Impact of sampling error
Performing a randomeffects metaanalysis
Summary points
13 FIXED EFFECT VERSUS RANDOMEFFECTS MODELS
Introduction
Definition of a summary effect
Estimating the summary effect
Extreme effect size in large study
Confidence interval
The null hypothesis
Which model should we use?
Model should not be based on the test for heterogeneity
Concluding remarks
Summary points
14 WORKED EXAMPLES (PART 1)
Introduction
Worked example for continuous data (Part 1)
Worked example for binary data (Part 1)
Worked example for correlational data (Part 1)
Summary points
PART 4: HETEROGENEITY
15 OVERVIEW
Introduction
16 IDENTIFYING AND QUANTIFYING HETEROGENEITY
Introduction
Isolating the variation in true effects
Computing Q
Estimating tausquared
The I 2 statistic
Comparing the measures of heterogeneity
Confidence intervals for T 2
Confidence intervals (or uncertainty intervals) for I 2
Summary points
17 PREDICTION INTERVALS
Introduction
Prediction intervals in primary studies
Prediction intervals in metaanalysis
Confidence intervals and prediction intervals
Comparing the confidence interval with the prediction interval
Summary points
18 WORKED EXAMPLES (PART 2)
Introduction
Worked example for continuous data (Part 2)
Worked example for binary data (Part 2)
Worked example for correlational data (Part 2)
Summary points
19 SUBGROUP ANALYSES
Introduction
Fixedeffect model within subgroups
Computational models
Random effects with separate estimates of T 2
Random effects with pooled estimate of T 2
The proportion of variance explained
Mixedeffect model
Obtaining an overall effect in the presence of subgroups
Summary points
20 METAREGRESSION
Introduction
Fixedeffect model
Fixed or random effects for unexplained heterogeneity
Randomeffects model
Statistical power for regression
Summary points
21 NOTES ON SUBGROUP ANALYSES AND METAREGRESSION
Introduction
Computational model
Multiple comparisons
Software
Analysis of subgroups and regression are observational
Statistical power for subgroup analyses and metaregression
Summary points
PART 5: COMPLEX DATA STRUCTURES
22 OVERVIEW
23 INDEPENDENT SUBGROUPS WITHIN A STUDY
Introduction
Combining across subgroups
Comparing subgroups
Summary points
24 MULTIPLE OUTCOMES OR TIME POINTS WITHIN A STUDY
Introduction
Combining across outcomes or timepoints
Comparing outcomes or timepoints within a study
Summary points
25 MULTIPLE COMPARISONS WITHIN A STUDY
Introduction
Combining across multiple comparisons within a study
Differences between treatments
Summary points
26 NOTES ON COMPLEX DATA STRUCTURES
Introduction
Combined effect
Differences in effect
PART 6: OTHER ISSUES
27 OVERVIEW
28 VOTE COUNTING – A NEW NAME FOR AN OLD PROBLEM
Introduction
Why vote counting is wrong
Votecounting is a pervasive problem
Summary points
29 POWER ANALYSIS FOR METAANALYSIS
Introduction
A conceptual approach
In context
When to use power analysis
Planning for precision rather than for power
Power analysis in primary studies
Power analysis for metaanalysis
Power analysis for a test of homogeneity
Summary points
30 PUBLICATION BIAS
Introduction
The problem of missing studies
Methods for addressing bias
Illustrative example
The model
Getting a sense of the data
Is the entire effect an artifact of bias
How much of an impact might the bias have?
Summary of the findings for the illustrative example
Small study effects
Concluding remarks
Summary points
PART 7: ISSUES RELATED TO EFFECT SIZE
31 OVERVIEW
32 EFFECT SIZES RATHER THAN P VALUES
Introduction
Relationship between pvalues and effect sizes
The distinction is important
The pvalue is often misinterpreted
Narrative reviews vs. metaanalyses
Summary points
33 SIMPSON’S PARADOX
Introduction
Circumcision and risk of HIV infection
An example of the paradox
Summary points
34 GENERALITY OF THE BASIC INVERSEVARIANCE METHOD
Introduction
Other effect sizes
Other methods for estimating effect sizes
Individual participant data metaanalyses
Bayesian approaches
Summary points
PART 8: FURTHER METHODS
35 OVERVIEW
36 METAANALYSIS METHODS BASED ON DIRECTION AND P VALUES
Introduction
Vote counting
The sign test
Combining pvalues
Summary points
37 FURTHER METHODS FOR DICHOTOMOUS DATA
Introduction
MantelHaenszel method
Onestep (Peto) formula for odds ratio
Summary points
38 PSYCHOMETRIC METAANALYSIS
Introduction
The attenuating effects of artifacts
Metaanalysis methods
Example of psychometric metaanalysis
Comparison of artifact correction with metaregression
Sources of information about artifact values
How heterogeneity is assessed
Reporting in psychometric metaanalysis
Concluding remarks
Summary points
PART 9: METAANALYSIS IN CONTEXT
39 OVERVIEW
40 WHEN DOES IT MAKE SENSE TO PERFORM A METAANALYSIS?
Introduction
Are the studies similar enough to combine?
Can I combine studies with different designs?
How many studies are enough to carry out a metaanalysis?
Summary points
41 REPORTING THE RESULTS OF A METAANALYSIS
Introduction
The computational model
Forest plots
Sensitivity analysis
Summary points
42 CUMULATIVE METAANALYSIS
Introduction
Why perform a cumulative metaanalysis?
Summary points
43 CRITICISMS OF METAANALYSIS
Introduction
One number cannot summarize a research field
The file drawer problem invalidates metaanalysis
Mixing apples and oranges
Garbage in, garbage out
Important studies are ignored
Metaanalysis can disagree with randomized trials
Metaanalyses are performed poorly
Is a narrative review better?
Concluding remarks
Summary points
PART 10: RESOURCES AND SOFTWARE
44 SOFTWARE
Introduction
Three examples of metaanalysis software
The software
Comprehensive metaanalysis (CMA) 2.0
Revman 5.0
StataTM macros with Stata 10.0
Summary points
45 BOOKS, WEB SITES AND PROFESSIONAL ORGANIZATIONS
Books on systematic review methods
Books on metaanalysis
Web sites
INDEX
 Provides worked examples throughout, with visual explanations, using screenshots from Excel spreadsheets and computer programs such as Comprehensive MetaAnalysis (CMA) or Strata.
 Details instructions for performing the analyses shown in the screenshots, and manipulating the variables to fully master the techniques.
 Accompanied by a free download of an instructional version of comprehensive metaanalysis to enable the readers to perform all the exercises from the book.
 Authored by four of the leading names in metaanalysis research.
 Wiley ETexts are powered by VitalSource technologies ebook software.
 With Wiley ETexts you can access your ebook how and where you want to study: Online, Download and Mobile.
 Wiley etexts are nonreturnable and nonrefundable.
 WileyPLUS registration codes are NOT included with the Wiley EText. For informationon WileyPLUS, click here .
 To learn more about Wiley etexts, please refer to our FAQ.
 Ebooks are offered as ePubs or PDFs. To download and read them, users must install Adobe Digital Editions (ADE) on their PC.
 Ebooks have DRM protection on them, which means only the person who purchases and downloads the ebook can access it.
 Ebooks are nonreturnable and nonrefundable.
 To learn more about our ebooks, please refer to our FAQ.