Skip to main content

Experimental Methods in Survey Research: Techniques that Combine Random Sampling with Random Assignment





Experimental Methods in Survey Research: Techniques that Combine Random Sampling with Random Assignment



A thorough and comprehensive guide to the theoretical, practical, and methodological approaches used in survey experiments across disciplines such as political science, health sciences, sociology, economics, psychology, and marketing

This book explores and explains the broad range of experimental designs embedded in surveys that use both probability and non-probability samples. It approaches the usage of survey-based experiments with a Total Survey Error (TSE) perspective, which provides insight on the strengths and weaknesses of the techniques used.

Experimental Methods in Survey Research: Techniques that Combine Random Sampling with Random Assignment addresses experiments on within-unit coverage, reducing nonresponse, question and questionnaire design, minimizing interview measurement bias, using adaptive design, trend data, vignettes, the analysis of data from survey experiments, and other topics, across social, behavioral, and marketing science domains.

Each chapter begins with a description of the experimental method or application and its importance, followed by reference to relevant literature. At least one detailed original experimental case study then follows to illustrate the experimental method’s deployment, implementation, and analysis from a TSE perspective. The chapters conclude with theoretical and practical implications on the usage of the experimental method addressed. In summary, this book:

  • Fills a gap in the current literature by successfully combining the subjects of survey methodology and experimental methodology in an effort to maximize both internal validity and external validity
  • Offers a wide range of types of experimentation in survey research with in-depth attention to their various methodologies and applications
  • Is edited by internationally recognized experts in the field of survey research/methodology and in the usage of survey-based experimentation —featuring contributions from across a variety of disciplines in the social and behavioral sciences
  • Presents advances in the field of survey experiments, as well as relevant references in each chapter for further study
  • Includes more than 20 types of original experiments carried out within probability sample surveys
  • Addresses myriad practical and operational aspects for designing, implementing, and analyzing survey-based experiments by using a Total Survey Error perspective to address the strengths and weaknesses of each experimental technique and method

Experimental Methods in Survey Research: Techniques that Combine Random Sampling with Random Assignment is an ideal reference for survey researchers and practitioners in areas such political science, health sciences, sociology, economics, psychology, public policy, data collection, data science, and marketing. It is also a very useful textbook for graduate-level courses on survey experiments and survey methodology.

Paul J. Lavrakas, PhD, is Senior Fellow at the NORC at the University of Chicago, Adjunct Professor at University of Illinois-Chicago, Senior Methodologist at the Social Research Centre of Australian National University and at the Office for Survey Research at Michigan State University.

Michael W. Traugott, PhD, is Research Professor in the Institute for Social Research at the University of Michigan.

Courtney Kennedy, PhD, is Director of Survey Research at Pew Research Center in Washington, DC.

Allyson L. Holbrook, PhD, is Professor of Public Administration and Psychology at the University of Illinois-Chicago.

Edith D. de Leeuw, PhD, is Professor of Survey Methodology in the Department of Methodology and Statistics at Utrecht University.

Brady T. West, PhD, is Research Associate Professor in the Survey Research Center at the University of Michigan-Ann Arbor.

List of Contributors


About the companion Website

Chapter 1 Probability Survey-Based Experimentation and the Balancing of Internal and External Validity Concerns

Chapter 2 Within-household selection methods: A critical review and experimental examination

Chapter 3 Measuring Within-Household Contamination: The Challenge of Interviewing More Than One Member of a Household

Chapter 4 Survey Experiments on Interactions and Nonresponse: A Case Study of Incentives and Modes

Chapter 5 Experiments on the Effects of Advance Letters in Surveys

Chapter 6 Experiments on the Design and Evaluation of Complex Survey Questions

Chapter 7 Impact of response scale features on survey responses to behavioral questions

Chapter 8 Mode effects versus Question Format Effects: An Experimental Investigation of Measurement Error Implemented in a Probability-Based Online Panel

Chapter 9 Conflicting Cues: Item Nonresponse and Experimental Mortality

Chapter 10 Application of a List Experiment at the Population Level: The Case of Opposition to Immigration in the Netherlands

Chapter 11 Race- and Ethnicity-of-Interviewer Effects

Chapter 12 Investigating Interviewer Effects and Confounds in Survey-based Experimentation

Chapter 13 Using Experiments to Assess Interactive Feedback That Improves Response Quality in Web Surveys

Chapter 14 Randomized experiments for web-mail surveys conducted using address-based samples of the general population

Chapter 15 Mounting multiple experiments on longitudinal social surveys: Design and implementation considerations

Chapter 16 Obstacles and Opportunities for Experiments in Establishment Surveys Supporting Official Statistics

Chapter 17 Tracking Question-Wording Experiments across Time in the General Social Survey, 1984-2014

Chapter 18 Survey Experiments and Changes in Question Wording in Repeated Cross-Sectional Surveys

Chapter 19 Are Factorial Survey Experiments Prone to Survey Mode Effects?

Chapter 20 Experimental Methods in Survey Research

Chapter 21 Identities and Intersectionality: A Case for Purposive Sampling in Survey Experimental Research

Chapter 22 Designing Probability Samples to Study Treatment Effect Heterogeneity

Chapter 23 Design-based analysis of experiments embedded in probability samples

Chapter 24 Extending the within-persons experimental design: The multitrait-multierror (MTME) approach