Zoom link: https://umd.zoom.us/j/
Password: 828w
In the past decade the mathematical theory of machine learning has lagged far behind the successes of deep neural networks on practical challenges. In this lecture I will outline why the practice of neural networks precipitated a crisis in the theory of Machine Learning and rethinking of certain basic assumptions. I will discuss how the concept of interpolation (fitting the data exactly) clarifies many of the underlying issues leading to new theoretical analyses. Finally, I will briefly mention some new results showing how interpolated predictors may relate to practical settings where the training loss, while small, is not usually driven to zero.
The main reference paper is the review: https://arxiv.org/abs/
I will also briefly discuss some of the results in https://arxiv.org/abs/2207.
Mikhail Belkin received his Ph.D. in 2003 from the Department of Mathematics at the University of Chicago. His research interests are in theory and applications of machine learning and data analysis. Some of his well-known work includes widely used Laplacian Eigenmaps, Graph Regularization and Manifold Regularization algorithms, which brought ideas from classical differential geometry and spectral analysis to data science. His recent work has been concerned with understanding remarkable mathematical and statistical phenomena observed in deep learning. This empirical evidence necessitated revisiting some of the basic concepts in statistics and optimization. One of his key recent findings is the “double descent” risk curve that extends the textbook U-shaped bias-variance trade-off curve beyond the point of interpolation.
Mikhail Belkin is a recipient of a NSF Career Award and a number of best paper and other awards. He has served on the editorial boards of the Journal of Machine Learning Research, IEEE Pattern Analysis and Machine Intelligence and SIAM Journal on Mathematics of Data Science.