Mark Himmelstein, Georgia Institute of Technology
Measuring Persuasion Without Measuring a Prior Belief: A New Application of Planned Missing Data Techniques
Research on advice taking frequently uses a Judge Advisor System (JAS) experimental design, in which a judge reports a prior belief, receives advice, and then revises their initial estimate. However, recent studies have shown that the mere act of measuring a judge’s prior belief has an unintended effect on how they utilize advice. This phenomenon is known as measurement reactivity, which is a potential confound in any research design with both pre- and post-treatment measurement. I show it is possible to use statistical imputation methods to model change as a function of a treatment without requiring a pre-treatment measurement by treating such measurements as planned missing data. I demonstrate the effectiveness of this method in two simulation studies and two empirical JAS studies. Simulation studies were used to demonstrate the conditions under which planned missing data imputation methods effectively recover parameters of interest, including test for and quantify measurement reactivity. The empirical studies demonstrate different research designs for deploying these methods and provide new theoretical insight about how people take advice when they aren’t required to explicitly report an independent prior belief.
2025 Dissertation Prize
About the speaker
Mark Himmelstein received his Ph.D. in Psychometrics and Quantitative Psychology at Fordham University. After graduating, he joined Georgia Tech as an Assistant Professor of Psychology where he teaches courses in statistics and manages the Subjective Uncertainty and Belief lab. His research lies at the intersection of human judgment and decision-making, normative models of decision theory, and quantitative methodology. His work has included adapting Item Response Theory models to assess the quality of probability forecasts for real world geopolitical events, developing computational models of how people update their beliefs under uncertainty, and novel implementations of planned missing data designs to facilitate counterfactual inference.