log in  |  register  |  feedback?  |  help  |  web accessibility
Remarkable phenomena of deep learning through the prism of interpolation.
Mikhail Belkin
Tuesday, September 6, 2022, 3:30-4:30 pm Calendar
  • You are subscribed to this talk through .
  • You are watching this talk through .
  • You are subscribed to this talk. (unsubscribe, watch)
  • You are watching this talk. (unwatch, subscribe)
  • You are not subscribed to this talk. (watch, subscribe)

Zoom link: https://umd.zoom.us/j/95197245230?pwd=cDRlVWRVeXBHcURGQkptSHpIS0VGdz09
Password: 828w

In the past decade the mathematical theory of machine learning has lagged far behind the successes of deep neural networks on practical challenges.  In this lecture I will outline why the practice of neural networks precipitated a crisis in the theory of Machine Learning and  rethinking of certain basic assumptions. I will discuss how the concept of interpolation (fitting the data exactly) clarifies many of the underlying issues leading to new theoretical analyses. Finally, I will briefly mention some new results showing how interpolated predictors may relate to practical settings where the training loss, while small, is not usually driven to zero.

The main reference paper is the review: https://arxiv.org/abs/2105.14368

I will also briefly discuss some of the results in https://arxiv.org/abs/2207.11621 and https://arxiv.org/abs/2010.01851


Mikhail Belkin received his Ph.D. in 2003 from the Department of Mathematics at the University of Chicago. His research interests are in theory and  applications of machine learning and data analysis. Some of his well-known work includes widely used Laplacian Eigenmaps, Graph Regularization and Manifold Regularization algorithms, which brought ideas from classical differential geometry and spectral analysis to data science. His recent work has been concerned with understanding remarkable mathematical and statistical phenomena observed in deep learning. This empirical evidence necessitated revisiting some of the basic concepts in statistics and optimization.  One of his key recent findings is the “double descent” risk curve that extends the textbook U-shaped bias-variance trade-off curve beyond the point of interpolation.

Mikhail Belkin is a recipient of a NSF Career Award and a number of best paper and other awards. He has served on the editorial boards of the Journal of Machine Learning Research, IEEE Pattern Analysis and Machine Intelligence and SIAM Journal on Mathematics of Data Science.

This talk is organized by Richa Mathur