JoonHo Lee, University of California, Berkeley

Modeling site-level variation in treatment effects is a critical component in understanding the results of large-scale multisite trials. Combined with the proper setting, a given treatment might be far more effective than the average effect would propose. A convenient, often implicit, modeling assumption is that the distribution of site-level effects is Gaussian. Yet, this could yield misleading estimates for quantities such as the proportion of sites had an impact larger than some threshold, particularly when the true distribution is multi-modal or long-tailed, for instance. In this paper, we show that this deconvolution problem, which aims to recover an unknown prior density G from the data, has deep connections to the meta-analysis and specifying latent trait distributions in IRT models. We then demonstrate the results from a simulation study to examine the costs and benefits of flexible nonparametric and semiparametric alternatives for modeling the site-level effects distribution and investigate ways to incorporate site-level covariates. We focus on the empirical Bayes deconvolution approach by Efron (2016) where G is restricted to be in a parametric exponential family, and explicitly compare it to the classic methods of multilevel modeling, finite mixture models, and Dirichlet Processes. We seek to examine how the Efron’s approach can balance between flexibility and efficiency, especially in the context where the number of sites is generally small to moderate, reaching only to the hundreds.