Leslie Rutkowski, Indiana University
Beyond Carrying Capacity: Have International Assessments Been Stretched Too Far?
Date & Time: Tuesday, July 20 at 12:00pm (Noon) EST
In this talk I borrow Henry Braun’s concept of carrying capacity of data (CCD) to describe several challenges to measuring achievement internationally and, importantly, to using the resultant data in increasingly dubious ways. The CCD, related to Messick’s (1989) notion of validity, “is an integrated, evaluative judgment of the credibility of specific data-based inferences, informed by quantitative (and qualitative) analyses, leavened by experience” (Braun, 2019, p. 5). The foundation of my argument is based on psychometric approaches to data analysis as well as a review of substantive literature that uses international assessment data as an evidentiary basis from which to make questionable claims. I especially highlight two emergent areas: (1) a resurgence in eugenic research, a portion of which uses PISA and related data; and (2) linking disparate studies to measure the UN’s Sustainable Development Goals. I conclude with an emphasis on the importance of developing measures that are appropriate for the target populations and, especially, associated inferences and claims.
About the Speaker
Leslie Rutkowski is Professor of Inquiry Methodology at Indiana University. She earned her PhD in Educational Psychology, specializing in Statistics and Measurement, from the University of Illinois at Urbana-Champaign. Leslie’s research is in the area of international large-scale assessment. Her interests include latent variable modeling and examining methods for comparing heterogeneous populations in international surveys. In addition to a recently completed Norwegian Research Council grant on developing international measurement methods, Leslie published the edited volume Handbook of International Large-Scale Assessment (Rutkowski, von Davier, and Rutkowski, 2014) with Chapman & Hall. She is also currently co-authoring a text book on large-scale assessment under the Guilford stamp. Leslie currently serves as associate editor for the Springer journal Large-scale Assessments in Education and as executive editor for the Springer book series, IEA Research for Education. Finally, Leslie serves as chair of the PISA Technical Advisory Group and as chair of the NAEP Questionnaire Standing Committee.