Leslie Rutkowski, Indiana University

Beyond Carrying Capacity: Have International Assessments Been Stretched Too Far?

Invited Speaker

In this talk I borrow Henry Braun’s concept of carrying capacity of data (CCD) to describe several challenges to measuring achievement internationally and, importantly, to using the resultant data in increasingly dubious ways. The CCD, related to Messick’s (1989) notion of validity, “is an integrated, evaluative judgment of the credibility of specific data-based inferences, informed by quantitative (and qualitative) analyses, leavened by experience” (Braun, 2019, p. 5). The foundation of my argument is based on psychometric approaches to data analysis as well as a review of substantive literature that uses international assessment data as an evidentiary basis from which to make questionable claims. I especially highlight two emergent areas: (1) a resurgence in eugenic research, a portion of which uses PISA and related data; and (2) linking disparate studies to measure the UN’s Sustainable Development Goals. I conclude with an emphasis on the importance of developing measures that are appropriate for the target populations and, especially, associated inferences and claims.  

Log in