Susan Embretson, Georgia Institute of Technology, Atlanta, Georgia, USA
Susan E. Embretson is Professor of Psychology at the Georgia Institute of Technology. Previously she was Professor of Psychology at University of Kansas and Director of the Quantitative Psychology Program there when she moved to Georgia Tech in 2004. She has served as President of the Psychometric Society (1999), the Society of Multivariate Psychology (1998) and American Psychological Association, Division of Measurement, Evaluation and Statistics (1991). She has received several awards for her research on interfacing cognitive psychology with psychometric models including the Saul Sells Award for Distinguished Multivariate Research (Society for Multivariate Experimental Psychology, 2019), Career Contribution Award (National Council on Measurement in Education, 2013), Distinguished Scientific Contribution Award (American Psychological Association, Division of Measurement, Evaluation and Statistics, 2001) and the Technical and Scientific Contribution Award (National Council on Measurement in Education ,1994-1996). Her current research interests include explanatory item response theory models, automatic item generation and dynamic measurement.
Modeling Cognitive Processes, Skills and Strategies in Item Responses: Implications for Test and Item Design
Interpretations of test scores typically involves the assumption that examinees are applying the same cognitive processes, skills and strategies in their item responses. If so, then test scores indicate examinee’s standing on a single latent trait. Results from several studies using mixture and explanatory item response models will be presented to show that this assumption is often not met. The implications of these results for test scoring, interpretations and external correlates, as well as for item and test design will be discussed.
Burr Settles, DuoLingo, Pittsburgh, Pennsylvania, USA
Burr Settles leads the research group at Duolingo, an award-winning website and mobile app offering free language education for the world. He also runs FAWM.ORG, a global annual songwriting experiment. He is the author of Active Learning — a text on machine learning algorithms that are adaptive, curious, and exploratory (if you will). His research has been published in Cognitive Science as well as all the major artificial intelligence venues such as NIPS, ICML, AAAI, ACL, and EMNLP. His work has been covered by The New York Times, Slate, Forbes, WIRED, and the BBC among others. In past lives, Burr was a postdoc at Carnegie Mellon and earned a PhD from UW-Madison. He currently lives in Pittsburgh, where he gets around by bike and plays guitar in the pop band delicious pastries.
Improving Language Learning and Assessment with Data
As scalable learning technologies become more ubiquitous, student data can and should be analyzed to develop new instructional technologies, such as personalized practice schedules and data-driven assessments. I will describe a few projects at Duolingo — the world's largest language education platform with more than 200 million students worldwide — where we combine learner data with machine learning, computational linguistics, and psychometrics to improve learning, testing, and engagement outcomes.
Eric-Jan Wagenmakers, University of Amsterdam, Amsterdam, Netherlands
Eric-Jan Wagenmakers is a mathematical psychologist and a dedicated Bayesian. He works for the Psychological Methods unit at the University of Amsterdam where he coordinates the team that develops JASP, an open-source software program for statistical analyses (www.jasp-stats.org). Wagenmakers is also a strong advocate of open science and the preregistration of analysis plans. For more information see ejwagenmakers.com.
Bayesian Multi-Model Inference for Practical and Impractical Problems
Whenever a set of models is applied to data, uncertainty surrounds both the selection of the best model and the estimation of the model parameters. The coherent Bayesian answer to the model selection question is to avoid all-or-none selection altogether and instead retain model uncertainty, employing it for purposes such as prediction and parameter estimation. Advantages of this multi-model approach include a reduction of overconfidence, improved predictive performance, and an increased robustness against model misspecification. Moreover, the multi-model framework can be seamlessly integrated with the recent open-science ideals of multi-team inference and multiverse analyses. We debunk several philosophical objections to Bayesian multi-model inference and demonstrate its practical use for problems ranging from the simple to the complex.