Improvement Science in Evaluation: Methods and Uses: New Directions for Evaluation, Number 153
While improvement science has experienced a surge of interest over the past 30 years, applications of it are rare in the evaluation literature.
This issue promotes the cross-fertilization of ideas, techniques, and tools between evaluation and improvement science. There are at least four areas where this cross-fertilization is particularly relevant: learning from error, examining variation, appreciating context, and focusing on systems change. This volume considers:
- the conceptual similarities and distinctions between improvement science and evaluation;
- the intellectual foundations, methods, and tools that collectively comprise improvement science; and
- case chapters that offer an inspiring review of state-of-the-art improvement science applications.
Cutting across all of these applications is a shared grounding in systems thinking, a determination to capture and better understand variation and contextual complexity, as well as a sustained commitment to generative learning about projects and programs—all issues of great concern to evaluators. The issue offers producers and users of evaluations the potential benefits of a closer engagement with improvement science.
This is the 153rd issue in the New Directions for Evaluation series from Jossey-Bass. It is an official publication ofthe American Evaluation Association.
EDITORS’ NOTES 7
Christina A. Christie, Moira Inkelas, Sebastian Lemire
1. Understanding the Similarities and Distinctions Between Improvement Science and Evaluation 11
Christina A. Christie, Sebastian Lemire, Moira Inkelas
This chapter considers the similarities and distinctions between improvement science and evaluation according to use, valuing, and methods.
2. The Methods and Tools of Improvement Science 23
Sebastian Lemire, Christina A. Christie, Moira Inkelas
This chapter introduces the intellectual foundation, core principles, and selected tools of improvement science.
3. Timely and Appropriate Healthcare Access for Newborns: A Neighborhood-Based, Improvement Science Approach 35
Courtney M. Brown, Robert S. Kahn, Neera K. Goyal
This chapter illustrates the application of quality improvement science techniques to iteratively address family- and system-level barriers to primary care.
4. Improvement for a Community Population: The Magnolia Community Initiative 51
Moira Inkelas, Patricia Bowie, Lila Guirguis
This chapter illustrates how a network of diverse organizations can use iterative learning cycles to come up with promising ideas, test and prototype these ideas, and spread and sustain what is found to work for a community population.
5. Breaking the “Adopt, Attack, Abandon” Cycle: A Case for Improvement Science in K–12 Education 65
This chapter describes the implementation of rapid cycles of evaluations (Plan–Do–Study–Act cycles) to adapt interventions to local school contexts.
6. Online Learning as a Wind Tunnel for Improving Teaching 79
James W. Stigler, Karen B. Givvin
This chapter considers the potential value of combining improvement science and online learning.
7. Value and Opportunity for Improvement Science in Evaluation 93
Moira Inkelas, Christina A. Christie, Sebastian Lemire
This concluding chapter reflects on the case examples comprising this volume and considers the major benefits and implications of integrating improvement science more firmly in evaluation.