Title
Measuring Reproducibility of High-Throughput Biological Experiments
Speaker
QunHua Li, Statistics Department UC-Berkeley
Abstract
Reproducibility is essential to reliable scientific discovery in large-scale high-throughput biological studies. In this talk, I will present a unified approach to measure reproducibility of findings identified from replicate experiments and select discoveries using reproducibility between replicates.
Unlike the usual scalar measures of reproducibility, our approach views reproducibility as when the findings are no longer consistent across replicates.
To measure the pairwise consistency between replicates, we develop a graphical statistic based on empirical copulas and a copula mixture model to quantitatively describe the change of consistency in the decreasing significance of findings. Based on the copula mixture procedure, we define a quantity, called “irreproducible discovery rate” analogous to the false discovery rate. This quantity, which describes the lack of reproducibility for the identifications selected at each threshold, provides a reproducibility criterion for selecting reliable signals and assessing the overall reproducibility of findings. Our approach can be applied to both probabilistic- and heuristic-based significance scores, and permits principled setting of selection thresholds.
