Title
Who is Afraid of Over-Fitting? An Interpolation Perspective on Modern Machine Learning
Speaker
Mikhail Belkin, Department of Computer Science and Engineering, The Ohio State University
Abstract
A striking feature of modern supervised machine learning is its consistent use of techniques that nearly interpolate the data. Deep networks often containing several orders of magnitude more parameters than data points, are trained to obtain near zero error on the training set. Yet, at odds with most theory, they show excellent test performance.
In this talk I will discuss and give some historical context for the phenomenon of interpolation (zero training loss). I will show how it provides a new perspective on machine learning and statistical inference forcing us to rethink some commonly held assumptions and point to the significant gaps in our understanding, even in the simplest settings, of why such classifiers generalize well.
I will also demonstrate the power of this point of view by describing its advantages for optimization and showing how its simplicity can be used to construct very efficient and theoretically sound methods for training large-scale kernel machines.
