Ohio State is in the process of revising websites and program materials to accurately reflect compliance with the law. While this work occurs, language referencing protected class status or other activities prohibited by Ohio Senate Bill 1 may still appear in some places. However, all programs and activities are being administered in compliance with federal and state law.

Seminar Series: Kai Tan

Kai Tan
February 11, 2025
3:00 pm - 4:00 pm
EA 170

Speaker: Kai Tan

Title: Estimating Generalization Error for Iterative Algorithms in High-Dimensional Regression
 
Abstract: In the first part of the talk, I will investigate the generalization error of iterates from iterative algorithms in high-dimensional linear regression. The proposed estimators apply to Gradient Descent, Proximal Gradient Descent, and accelerated methods like FISTA. These estimators are consistent under Gaussian designs and enable the selection of the optimal iteration when the generalization error follows a U-shaped pattern. Simulations on synthetic data demonstrate the practical utility of these methods.
 
In the second part of the talk, I will focus on the generalization performance of iterates obtained by Stochastic Gradient Descent (SGD), and their proximal variants in high-dimensional robust regression problems. I will introduce estimators that can precisely track the generalization error of the iterates along the trajectory of the iterative algorithm. These estimators are shown to be consistent under mild conditions that allow the noise to have infinite variance. Extensive simulations confirm the effectiveness of the proposed generalization error estimators.