Data-Driven Learning of Attribute Hierarchy and Q-Matrix for Cognitive Diagnostic Models

Half day short course (9:00am – 12:30pm)

Short course #3

Course Description

Cognitive Diagnostic Models (CDMs) provide detailed feedback on skill mastery based on examinees’ responses to assessment items. However, these models typically rely on expert-specified Q-matrices and attribute hierarchies. Because expert judgment can be subjective or unavailable in exploratory settings, data-driven approaches are needed to uncover these latent structures. This short course introduces a Bayesian framework for the simultaneous estimation of unknown Q-matrices and hierarchical prerequisite relationships directly from examinee response data.

Participants will learn Bayesian formulation for CDMs with hierarchical attribute structures and explore Markov Chain Monte Carlo methods for estimation of the latent structures and Q-matrices. The course will first introduce a Metropolis–Hastings within Gibbs (MH-Gibbs) algorithm for estimating attribute hierarchies as Directed Acyclic Graphs (DAGs) when the Q-matrix is assumed to be known, using data collected from both cross-sectional and dynamic settings. The course then introduces a second algorithm that learns the Q-matrix and attribute hierarchy simultaneously: a MH-Gibbs algorithm that employs a mini-batch strategy to improve computational efficiency and help the sampler escape local optima. Key methodological topics include the transitive reduction of DAGs for parsimonious modeling and the use of the Hungarian algorithm to address label-switching issues. Through hands-on demonstrations using a specialized R package optimized with Rcpp, participants will learn how to recover and interpret posterior inclusion probabilities using real-world datasets. This course is designed for psychometrics researchers and graduate students seeking advanced and reproducible tools for exploratory cognitive diagnosis.

About the Instructors

Yinghan Chen (University of Nevada, Reno)

 Dr. Yinghan Chen is an Associate Professor in the Department of Mathematics & Statistics at the University of Nevada, Reno. She received her Ph.D. in Statistics from the University of Illinois at Urbana-Champaign, Champaign, IL, in 2017. Her research interests include Bayesian statistics, computational statistics, latent variable models and psychometrics. She has published papers in leading journals including Psychometrika, Journal of Educational and Behavioral Statistics, IISE transactions, and Journal of Computational and Graphical Statistics, and has received grants from National Science Foundation (NSF) SES-2051198 and SES-1758688. She currently serves as an Associate Editor for the Journal of Educational and Behavioral Statistics. 

Xue Wang (Northeast Normal University)

 Xue Wang is a Postdoctoral researcher in the School of Mathematics
and Statistics at Northeast Normal University. She received her B.A. in Mathematics and Applied Mathematics from Northeast Normal University and earned her Ph.D. in Statistics from the same institution in 2025. From 2023 to 2024, she visited the Department of Educational Psychology at the University of Georgia as a Visiting Scholar for one year. Her research interests include cognitive diagnostic models, item response theory, variational inference, and Bayesian statistical methodology.

Shiyu Wang (University of Georgia)

 Dr. Shiyu Wang is an Associate Professor and Program coordinator of the Quantitative Methodology Program in the Department of Educational Psychology at the University of Georgia. She also directs the NewGen Psychometrics and Data Science Analytics (NPDA) Lab at UGA. Her methodological research spans computerized adaptive testing, cognitive diagnosis models, and next-generation psychometric and data science methods. Her recent work focuses on integrating multimodal data and advanced analytic techniques to expand the cognitive diagnosis modeling framework, improve adaptive assessment design, and strengthen validity evidence for learning and assessment. Dr. Wang currently serves as an Associate Editor for the Journal of Educational and Behavioral Statistics and AERA Open. She has published extensively in leading journals across psychometrics, educational measurement, and statistics. Her research contributions have been recognized with several honors, including the Jason Millman Promising Measurement Scholar Award from the National Council on Measurement in Education (2020), the Early Career Researcher Award from the International Association for Computerized Adaptive Testing (2019), and a NAEd/Spencer Postdoctoral Fellowship (2019).