Seminar Series: Rishabh Dudeja

Title: Universality in High-dimensional Statistics

Abstract: It has been observed that the statistical properties of many high-dimensional regression problems empirically exhibit universality with respect to the underlying design matrices. Specifically, design matrices with very different constructions seem to lead to identical estimation performance if they share the same spectrum and have "generic" singular vectors. This general universality phenomenon appears in numerous applications: in random optimization problems arising in statistical physics, in statistical inference problems like compressed sensing, and in the performance of sketching algorithms in randomized numerical linear algebra. In the first part of this talk, I will show how these empirical observations of universality can be exploited to design and analyze information-theoretically optimal spectral estimators for the phase retrieval problem: a non-linear regression problem that arises in imaging applications like X-ray crystallography. In the second part of the talk, I will describe recent progress toward a mathematical understanding of this universality phenomenon. In the context of penalized linear regression with strongly convex regularizers, I will describe nearly deterministic conditions on the design matrix under which this universality phenomenon occurs. I will show that these conditions can be easily verified for highly structured design matrices constructed with limited randomness like randomly subsampled Hadamard transforms and signed incoherent tight frames. I will conclude this talk by describing other exciting applications where we can potentially exploit similar universality phenomena to design and analyze improved estimators under assumptions justified by practical considerations rather than mathematical convenience.