Xiang Cheng, TILOS Postdoctoral Scholar, MIT Abstract: Motivated by the in-context learning phenomenon, we investigate how the Transformer neural network can implement learning algorithms in its forward pass. We show that a linear Transformer naturally learns to implement gradient descent, which enables it to learn linear functions in-context. More generally, we show that a non-linear […]

Nicklas Hansen, UC San Diego Abstract: Recent progress in AI can be attributed to the emergence of large models trained on large datasets. However, teaching AI agents to reliably interact with our physical world has proven challenging, which is in part due to a lack of large and sufficiently diverse robot datasets. In this talk, […]

Robert Nowak, University of Wisconsin Abstract: This talk presents a theory characterizing the types of functions neural networks learn from data. Specifically, the function space generated by deep ReLU networks consists of compositions of functions from the Banach space of second-order bounded variation in the Radon transform domain. This Banach space includes functions with smooth […]

Krishna Kalyanam, NASA Ames Research Center Abstract: We introduce several Air Traffic Management (ATM) initiatives envisioned by NASA and FAA for a future airspace that combines conventional traffic and new entrants (e.g., drones) without sacrificing safety. In this framework, we demonstrate the use of state-of-the-art AI/ML modeling and prediction tools that will enable efficient and […]

Elchanan Mossel, Professor of Mathematics, MIT Abstract: Classical Machine Learning theory is largely built upon the assumption that data samples are independent and identically distributed (i.i.d.) from general distribution families. In this talk, I will present novel insights that emerge when we move beyond these traditional assumptions, exploring both dependent sampling scenarios and structured generative […]

Rebecca Willett, University of Chicago Abstract: Algorithmic stability holds when our conclusions, estimates, fitted models, predictions, or decisions are insensitive to small changes to the training data. Stability has emerged as a core principle for reliable data science, providing insights into generalization, cross-validation, uncertainty quantification, and more. Whereas prior literature has developed mathematical tools for […]

Jason Lee, Princeton University Abstract: The incredible success of transformers on sequence modeling tasks can be largely attributed to the self-attention mechanism, which allows information to be transferred between different parts of a sequence. Self-attention allows transformers to encode causal structure which makes them particularly suitable for sequence modeling. However, the process by which transformers […]