Shiyu Wang, University of Georgia

Early Career Award

Cognitive diagnosis models (CDMs) are a class of restricted latent class models designed to provide fine-grained diagnostic information about examinees’ mastery of knowledge, skills, or attributes. A complete CDM framework involves multiple interconnected components: (1) identifying relevant skills or attributes, (2) characterizing structural relationships among attributes, (3) linking attributes to items through the Q matrix, and (4) specifying statistical models that relate examinees’ latent attributes and item characteristics to observed responses. To date, most methodological developments have concentrated on the fourth component, proposing alternative item response functions under different assumptions about how mastered attributes interact with item requirements. More recently, data-driven approaches have also emerged to infer attributes, attribute hierarchies, and Q-matrices directly from response data.

In this talk, I argue that accurate cognitive diagnosis is fundamentally a problem of reconstructing latent structure from incomplete and fragmented evidence. Response accuracy alone provides only a narrow projection of examinees’ cognitive and behavioral processes and is insufficient to recover the full diagnostic system. I first present a family of cognitive diagnosis models that jointly leverage response accuracy and response time to improve skill mastery inference, reveal heterogeneity in test-taking and learning behaviors, and enable the tracking of skill development over time. These models illustrate how incorporating process data expands the inferential scope of CDMs beyond static classification toward dynamic and behaviorally informed diagnosis.

Building on this work, I propose a unifying perspective in which cognitive diagnosis resembles assembling a complex jigsaw puzzle: each data modality—items, scores, and response processes—offers a partial and uncertain view of the underlying structure. The central challenge is not simply to add more data, but to extract structural information from each source, quantify its uncertainty, and integrate these complementary signals in a principled and coherent manner. Using several illustrative studies, I demonstrate how integrating multiple data modalities can inform different components of CDMs. I conclude by discussing open challenges and promising directions for future research in multimodal cognitive diagnosis.

About the Speaker

Dr. Shiyu Wang is an Associate Professor and Program coordinator of the Quantitative Methodology Program in the Department of Educational Psychology at the University of Georgia. She also directs the NewGen Psychometrics and Data Science Analytics (NPDA) Lab at UGA. Her methodological research spans computerized adaptive testing, cognitive diagnosis models, and next-generation psychometric and data science methods. Her recent work focuses on integrating multimodal data and advanced analytic techniques to expand the cognitive diagnosis modeling framework, improve adaptive assessment design, and strengthen validity evidence for learning and assessment. Dr. Wang currently serves as an Associate Editor for the Journal of Educational and Behavioral Statistics and AERA Open. She has published extensively in leading journals across psychometrics, educational measurement, and statistics. Her research contributions have been recognized with several honors, including the Jason Millman Promising Measurement Scholar Award from the National Council on Measurement in Education (2020), the Early Career Researcher Award from the International Association for Computerized Adaptive Testing (2019), and a NAEd/Spencer Postdoctoral Fellowship (2019).