Ohio State nav bar

Seminar: Brian Kulis

Statistics Seminar
September 11, 2014
All Day
209 W. Eighteenth Ave. (EA), Room 170

Title

Small-Variance Asymptotics for Large-Scale Learning

Speaker

Brian Kulis, The Ohio State University

Abstract

It is widely known that the Gaussian mixture model is related to k-means by “small-variance asymptotics”: as the covariances of the clusters shrink, the EM algorithm approaches the k-means algorithm and the negative log-likelihood approaches the k-means objective. Similar asymptotic connections exist for other machine learning models, including dimensionality reduction (probabilistic PCA becomes PCA), multiview learning (probabilistic CCA becomes CCA), and classification (a restricted Bayes optimal classifier becomes the SVM). The asymptotic non-probabilistic counterparts to the probabilistic models are almost always more scalable, and are typically easier to analyze, making them useful alternatives to the probabilistic models in many situations. I will explore how we can extend such asymptotics to a richer class of probabilistic models, with a focus on large-scale graphical models, Bayesian nonparametric models, and time-series data. I will develop the necessary mathematical tools needed for these extensions and will describe a framework for designing scalable optimization problems derived from the rich probabilistic models. Applications are diverse, and include topic modeling, network evolution and feature learning.