Debate: Is the concept of equal Interval scales in educational testing attainable or just a myth?
Wim J. van der Linden, Pacific Metrics Corporation, Monterey, CA, USA
Dr. van der Linden is Distinguished Scientist and Director of Research Innovation, Pacific Metrics Corporation, Monterey, CA, and Professor Emeritus of Measurement and Data Analysis, University of Twente. He received his PhD in psychometrics from the University of Amsterdam. His research interests include item response theory, adaptive testing, optimal test assembly, parameter linking, statistical detection of cheating and response time modeling. He is the author of Linear Models for Optimal Test Design (Springer, 2005) and the editor of a new three-volume Handbook of Item Response Theory: Models, Statistical Tools, and Applications (Chapman & Hall/CRC, 2015). He is also a co-editor of Computerized Adaptive Testing: Theory and Applications (Kluwer, 2000; with C. A. W. Glas), and its sequel Elements of Adaptive Testing (Springer, 2010; with C. A. W. Glas). Dr. van der Linden has served on the editorial boards of nearly all major test-theory journals and is co-editor for the Chapman & Hal//CRC Series on Statistics for Social and Behavioral Sciences. He is also a former President of the National Council on Measurement in Education (NCME) and the Psychometric Society, Fellow of the Center for Advanced Study in the Behavioral Sciences, Stanford, CA, was awarded an Honorary Doctorate from Umea University in Sweden in 2008, and is a recipient of the ATP and NCME Career Achievement Awards for his work on educational measurement.
The Myth of Equal Units of Measurement
In this debate I will defend the position that the quest for equal measurement units in the social and behavioral sciences since N. D. Campbell’s Physics, the elements (1920) and S. S. Stevens’ On the theory of scales of measurement (1946) has been a waste of time. The belief in the existence of such units seems to have be restricted to these sciences only. Physics does not care about equal units, statistics does not, in our daily lives we do not, and psychometrics should not. The “permissible statistical operations” terror that has plagued us since Stevens’ introduction of scale levels is entirely due to misunderstanding of the nature of such operations. I will conclude my introduction to the debate by showing that measurements provided by the use of item response models, although on an entirely arbitrary scale, have absolute norm-referenced and criterion-reference interpretations, and thus give us all the theoretical and practical information we need.
Derek C. Briggs, University of Colorado, Boulder, CO, USA
Derek Briggs is Professor and Chair of the Research and Evaluation Methodology program in the University of Colorado’s School of Education, where he also serves as Director of the Center for Assessment, Design, Research and Evaluation (CADRE). His research agenda focuses upon building sound psychometric approaches for the measurement and evaluation of growth in student learning. He has a special interest methods that bridge the use of student assessments for both formative and summative purposes.
Psychometrics, Measurement and Obtainable Goals
Standardized tests are commonly used to evaluate student growth as well as group differences in student achievement. The validity of these sorts of uses depends upon the ability to make generalizable inferences about magnitudes. In the context of latent variable modeling, the foundation for these sorts of inferences has been a matter of ongoing debate, and much of this hinges upon distinctions between the meaning and practice of measurement in metrology and psychometrics. In this presentation I discuss some of these distinctions, illustrate how conventional psychometric practices (at least in the United States) provide a highly equivocal basis for interpretations about score magnitudes, and suggest that this is a source of considerable confusion in educational research. I conclude by outlining a constructive program of research that could begin to establish meaningful reference units for targeted cognitive attributes in the domains of mathematics, science and reading comprehension.
Some useful background reading
D. J. Hand (2010). Measurement theory and practice: The world through quantification. New York: Wiley.
D. Briggs ( 2013). Measuring growth with vertical scales. Journal of Educational Measurement, 50, 2, 204-226.
von Davier, M., Xu, X., & Carstensen, C. H. (2011). Measuring growth in a longitudinal large-scale assessment with a general latent variable model. Psychometrika, 76(2), 318-336.
Rock, D. A. (2012). Modeling change in large‐scale longitudinal studies of educational growth: Four decades of contributions to the assessment of educational growth. ETS Research Report Series, 2012(1), i-37.
Psychometrika Anniversary Sessions
Session 1- Chair: Lawrence Hubert
Willem Heiser on Carroll and Chang (1970)- Click here for article
Klaas Sijtsma on Cronbach (1951)- Click here for article
Hans Friedrich Koehn on Greenhouse and Geisser (1959)- Click here for article
Yoshio Takane on Akaike (1967) - Click here for article
Session 2- Chair: Willem Heiser
Jack McArdle on Horn (1965)- Click here for article
Lawrence Hubert on Johnson (1967)- Click here for article
Jacqueline Meulman on Kruskal (1964)- Click here for article
Charles Lewis on Tucker and Lewis (1973)- Click here for article
Session 3- Chair: Irini Moustaki
Presenters: Irini Moustaki, Lawrence Hubert, Willem Heiser, Ulf Böckenolt, Brian Junker
Symposium 2: Virtual International Collaboration
Chair: Luz Bay, The College Board
Symposium 5: Statistical Consulting
Chair: Carolin Strobl, Department of Psychology, University of Zurich, Switzerland
Many of us offer statistical or psychometric consulting to applied researchers or practitioners in- and outside of academia - whether informally or as a part of our job description. Consulting teaches us important lessons from practice and can be very rewarding, but it can also be very time consuming and expectations may differ between us and our clients. On the panel, five researchers with extensive experience in statistical consulting will give short presentations about how they organize their consulting services, what challenges they have encountered and how they are coping with them. After this round of introductions, the panel is open to questions and discussion with the audience.