Hong Jiao, University of Maryland, College Park
Integrating Item Product and Process Data to Enhance the Theory and Practices of Psychometric Analyses in Digital Assessments
Assessing learning outcomes in the digital era brings about both opportunities and challenges. Digital assessment facilitates the use of technology-enhanced innovative items in assessing new constructs. In the meantime, digital assessment platforms can readily collect rich process data including response time, click streaming data as well as other biometric data such as eye-tracking, body posture and movement, facial expression, and audio-video data. In general, process data improves ability parameter estimation, provides more fine-grained cognitive diagnosis, better estimates problem-solving strategies, and more accurately detects abnormal responding behaviors or cheating. However, the analyses of these structured or unstructured new data types can be challenging in regard to how to integrate multiple data types from multimode into a coherent framework for one unified psychometric analysis purpose and how to use the results to improve assessment practices. This talk will present extended joint modeling of both product and process data for cognitive diagnosis and cheating detection. Alternatively, machine learning algorithms are explored to achieve the same purposes for psychometric analyses. Some limitations of the methods will be discussed and potential future explorations will be highlighted.
about the speaker
Hong Jiao is a professor in the Department of Human Development and Quantitative Methodology at the University of Maryland, College Park (UMD), and Director of the Maryland Assessment Research Center (MARC). She earned her Ph.D. in Measurement, Statistics, and Evaluation at Florida State University. Prior to joining the faculty in Measurement, Statistics, and Evaluation at UMD, she worked as a psychometrician at Harcourt Assessment on different state assessment programs. Her research focuses on improving the theory and practices in educational and psychological measurement including methodological research on joint modeling of product and process data for cognitive diagnosis, modeling testlet effects, machine learning methods for cheating detection and cognitive diagnosis, Bayesian model parameter estimation, computer-based testing, and psychometric issues in large-scale tests.