Minjeong Jeon, University of California, Los Angeles
A network view on item response data and a latent space modeling approach to item response network
Conventional item response data analysis typically relies on several assumptions such as local item independence, respondent independence, and homogeneity. However, these assumptions are often violated in practice, and moreover difficult to verify. To weaken the reliance on these assumptions, I propose a new perspective on item response data - to view them as network data that represent relationships between two types of actors, respondents and items. In this network view on item response data, a tie (or connection) between the two types of actors is made when a correct response is given to the item by the respondent. The probability of a tie between a respondent and an item is then modeled as a function of a person attribute (ability), an item attribute (easiness), and a distance between the person and the item in a low-dimensional Euclidean space. In this latent space item response model, the probability of a tie (that is, the probability of a respondent’ giving a correct response to an item) is determined not only by the person’s ability and the item’ easiness level, but also by how closely or distantly the person is located from the item in latent space. I will explain how the conventional assumptions of local item independence, respondent independence, and homogeneity are relaxed in the proposed latent space item response model. Additional benefits of the proposed network perspective on item response data and the proposed modeling approach are discussed based on empirical data examples.
Maarten Marsman, University of Amsterdam
(Towards) A Grand Unified Theory of Psychometrics
In this talk, I unify several of the disparate approaches that exist in the psychometric universe. I will focus on the three distinct statistical realms of item response models, binary graphical models, and network models and show how these can be brought together in the Ising model, one of the flagship models in the new field of network psychometrics. I will discuss some of the lessons that can be learned from this synthesis, add some interesting historical notes, and use it to provide some useful new results. One important consequence of this synthesis is the consolidation of intraindividual and interindividual approaches in network psychometrics; a formal relation between idiographic topological structures and group-level associations. But there are many more important lessons that we could learn from this grand unified theory of psychometrics.
Leslie Rutkowski, Indiana University
The Ship Is Listing: Have We Reached Carrying Capacity of International Assessment Data?
In this talk I borrow Henry Braun’s concept of carrying capacity of data (CCD) to describe several challenges to measuring achievement internationally and, importantly, to using the resultant data in increasingly dubious ways. The CCD, related to Messick’s (1989) notion of validity, “is an integrated, evaluative judgment of the credibility of specific data-based inferences, informed by quantitative (and qualitative) analyses, leavened by experience” (Braun, 2019, p. 5). The foundation of my argument is based on psychometric approaches to data analysis as well as a review of substantive literature that uses international assessment data as an evidentiary basis from which to make questionable claims. I especially highlight a resurgence in eugenic research, a portion of which uses PISA and related data. I conclude with an emphasis on the importance of developing measures that are appropriate for the target populations.
Anna-Lena Schubert, Heidelberg University
Neurocognitive psychometric approaches to the measurement of individual differences in cognitive processes
The aim of neurocognitive psychometrics is to measure individual differences in parameters of cognitive processes. Often, these parameters are measured as performances indicators (e.g., response times or accuracies) in tasks supposedly engaging one specific cognitive process. This approach presumes that a specific task provides a process-pure measure of a single cognitive process—an assumption that is often violated as most cognitive tasks do not measure one specific cognitive process, but rather a combination of several cognitive processes. In this talk, I will discuss latent change and bifactor models as modeling approaches that may help to overcome this conceptual problem by modeling interindividual differences in intraindividual differences between experimental conditions. Special consideration will be given to recent studies suggesting negligible variance and substantial interindividual heterogeneity in these intraindividual experimental effects (Frischkorn, Schubert, & Hagemann, 2019; Gärtner & Strobel, 2019; Rey-Mermet, Gade, & Oberauer, 2018; Rouder & Haaf, 2019; Rouder, Kumar, & Haaf, 2019). Moreover, I will demonstrate how mathematical models of cognition and neural correlates of cognitive processes can be used to directly quantify individual differences in specific cognitive processes without relying on the assumption of pure insertion. I will highlight several cognitive models that may be of particular interest for cognitive psychometrics and discuss particularities of the psychometric modeling of model parameters and neural data, such as their hierarchical nature and their typically low-to-moderate consistencies.
Klaas Sijtsma, Tilburg University
Agenda setting for Psychometrics: Where are we and where will we go?
I will review one century of psychometric modeling and its meaning for the measurement of psychological attributes. Key topics will be (1) the sequencing of new and old knowledge; (2) the alignment of psychological theory and psychometric modeling or vice versa; (3) what data tell us about cognitive and affective processes; (4) the reliance of psychology on complex statistical models, and indexing as modern measurement; and (5) the blessings and curses of rich data sources. I will try to hypothesize what this all means for psychometrics of the future.
Zhiyong Johnny Zhang, University of Notre Dame
Psychometric Models for Social Network Data Analysis
Network analysis is becoming a popular interdisciplinary research area in computer science, statistics, sociology, political science, and psychology. To better analyze social network data for psychological research, we propose to combine psychometric models with social network techniques. In this talk, I will present three studies to show the advantages of such combinations. In the first study, we investigate the relationship between the personality factor space and a friendship network by combining a factor model and a latent space model. In the second study, we expand the traditional mediation model to study the mediation role of a friendship network in understanding sex differences in smoking. In the third study, we use a growth curve model to investigate the longitudinal change of friendship networks. I will end my talk by discussing a general psychometric modeling framework for social network data analysis.