Optimization for ML and AI Seminar: A non-equilibrium phase transition with broken ergodicity leads to double descent and benign overfitting in machine learning

HDSI 123 and Virtual 3234 Matthews Ln, La Jolla, CA, United States

Nigel Goldenfeld, UC San Diego Department of Physics and HDSI Abstract: The remarkable ability of modern neural networks to generalize improves with increasing network capacity, even when the number of model parameters or effective degrees of freedom exceeds the number of training data points. This phenomenon is all the more surprising given that generalization error […]

TILOS-HDSI Seminar: Inference-Time Algorithms: A Theoretical Lens on Tractability and Error Propagation

HDSI 123 and Virtual 3234 Matthews Ln, La Jolla, CA, United States

Andrej Risteski, Carnegie Mellon University Abstract: Modern AI systems are increasingly built by placing trained models inside larger computational loops. Inference-time algorithms are a basic instance of this idea: they use one or more trained models at test time to incorporate new information, exploit pretrained models as priors, and trade computational effort for accuracy, sample […]