TILOS Seminar: Single location regression and attention-based models
2025-03-27 14:00 - 15:00 (America/Los_Angeles) | HDSI 123 and Virtual, 3234 Matthews Ln, La Jolla, CA, 92093, United States
Claire Boyer, Université Paris-Saclay
Abstract: Attention-based models, such as Transformer, excel across various tasks but lack a comprehensive theoretical understanding, especially regarding token-wise sparsity and internal linear representations. To address this gap, we introduce the single-location regression task, where only one token in a sequence determines the output, and its position is a latent random variable, retrievable via a linear projection of the input. To solve this task, we propose a dedicated predictor, which turns out to be a simplified version of a non-linear self-attention layer. We study its theoretical properties, by showing its asymptotic Bayes optimality and analyzing its training dynamics. In particular, despite the non-convex nature of the problem, the predictor effectively learns the underlying structure. This work highlights the capacity of attention mechanisms to handle sparse token information and internal linear structures.

Boston Symmetry Day 2025
2025-03-31 | CA

TILOS Seminar: Foundational Methods for Foundation Models for Scientific Machine Learning
2025-04-02 11:00 - 12:00 | HDSI 123 and Virtual, 3234 Matthews Ln, La Jolla, CA, 92093, United States
Michael W. Mahoney, ICSI, LBNL, and Department of Statistics, UC Berkeley
Abstract: The remarkable successes of ChatGPT in natural language processing (NLP) and related developments in computer vision (CV) motivate the question of what foundation models would look like and what new advances they would enable, when built on the rich, diverse, multimodal data that are available from large-scale experimental and simulational data in scientific computing (SC), broadly defined. Such models could provide a robust and principled foundation for scientific machine learning (SciML), going well beyond simply using ML tools developed for internet and social media applications to help solve future scientific problems. I will describe recent work demonstrating the potential of the “pre-train and fine-tune” paradigm, widely-used in CV and NLP, for SciML problems, demonstrating a clear path towards building SciML foundation models; as well as recent work highlighting multiple “failure modes” that arise when trying to interface data-driven ML methodologies with domain-driven SC methodologies, demonstrating clear obstacles to traversing that path successfully. I will also describe initial work on developing novel methods to address several of these challenges, as well as their implementations at scale, a general solution to which will be needed to build robust and reliable SciML models consisting of millions or billions or trillions of parameters.

Michael W. Mahoney is at the University of California at Berkeley in the Department of Statistics and at the International Computer Science Institute (ICSI). He is also an Amazon Scholar as well as head of the Machine Learning and Analytics Group at the Lawrence Berkeley National Laboratory. He works on algorithmic and statistical aspects of modern large-scale data analysis. Much of his recent research has focused on large-scale machine learning, including randomized matrix algorithms and randomized numerical linear algebra, scientific machine learning, scalable stochastic optimization, geometric network analysis tools for structure extraction in large informatics graphs, scalable implicit regularization methods, computational methods for neural network analysis, physics informed machine learning, and applications in genetics, astronomy, medical imaging, social network analysis, and internet data analysis. He received his PhD from Yale University with a dissertation in computational statistical mechanics, and he has worked and taught at Yale University in the mathematics department, at Yahoo Research, and at Stanford University in the mathematics department. Among other things, he was on the national advisory committee of the Statistical and Applied Mathematical Sciences Institute (SAMSI), he was on the National Research Council’s Committee on the Analysis of Massive Data, he co-organized the Simons Institute’s fall 2013 and 2018 programs on the foundations of data science, he ran the Park City Mathematics Institute’s 2016 PCMI Summer Session on The Mathematics of Data, he ran the biennial MMDS Workshops on Algorithms for Modern Massive Data Sets, and he was the Director of the NSF/TRIPODS-funded FODA (Foundations of Data Analysis) Institute at UC Berkeley. More information is available at https://www.stat.berkeley.edu/~mmahoney/.

TILOS Seminar: Amplifying human performance in combinatorial competitive programming
2025-04-16 11:00 - 12:00 | Virtual
Petar Veličković, Google DeepMind
Abstract: Recent years have seen a significant surge in complex AI systems for competitive programming, capable of performing at admirable levels against human competitors. While steady progress has been made, the highest percentiles still remain out of reach for these methods on standard competition platforms such as Codeforces. In this talk, I will describe and dive into our recent work, where we focussed on combinatorial competitive programming. In combinatorial challenges, the target is to find as-good-as-possible solutions to otherwise computationally intractable problems, over specific given inputs. We hypothesise that this scenario offers a unique testbed for human-AI synergy, as human programmers can write a backbone of a heuristic solution, after which AI can be used to optimise the scoring function used by the heuristic. We deploy our approach on previous iterations of Hash Code, a global team programming competition inspired by NP-hard software engineering problems at Google, and we leverage FunSearch to evolve our scoring functions. Our evolved solutions significantly improve the attained scores from their baseline, successfully breaking into the top percentile on all previous Hash Code online qualification rounds, and outperforming the top human teams on several. To the best of our knowledge, this is the first known AI-assisted top-tier result in competitive programming.

HOT-AI: Horizons for Optimization in AI Workshop
2025-04-17 - 2025-04-18 | HDSI 123, 3234 Matthews Ln, La Jolla, CA, 92093, United States

TILOS Seminar: Optimal Quantization for LLMs and Matrix Multiplication
2025-05-23 11:00 - 12:00 | HDSI 123 and Virtual, 3234 Matthews Ln, La Jolla, CA, 92093, United States
Yury Polyanskiy, MIT
Abstract: The main building block of large language models is matrix multiplication, which is often bottlenecked by the speed of loading these matrices from memory. A number of recent quantization algorithms (SmoothQuant, GPTQ, QuIP, SpinQuant etc) address this issue by storing matrices in lower precision. We derive optimal asymptotic information-theoretic tradeoff between accuracy of the matrix product and compression rate (number of bits per matrix entry). We also show that a non-asymptotic version of our construction (based on nested Gosset lattices and Conway-Sloan decoding), which we call NestQuant, reduces perplexity deterioration almost three-fold compared to the state-of-the-art algorithms (as measured on LLama-2, Llama-3 with 8B to 70B parameters). Based on a joint work with Or Ordentlich (HUJI), Eitan Porat and Semyon Savkin (MIT EECS).

Yury Polyanskiy is a Cutten Professor of Electrical Engineering and Computer Science, a member of IDSS and LIDS at MIT, and an IEEE Fellow (2024). Yury received M.S. degree in applied mathematics and physics from the Moscow Institute of Physics and Technology in 2005 and Ph.D. degree in electrical engineering from Princeton University in 2010. His research interests span information theory, machine learning and statistics. Dr. Polyanskiy won the 2020 IEEE Information Theory Society James Massey Award, 2013 NSF CAREER award and 2011 IEEE Information Theory Society Paper Award.

TILOS Industry Day 2025
2025-06-02 | HDSI 123, 3234 Matthews Ln, La Jolla, CA, 92093, United States

TILOS-HDSI Seminar: A New Paradigm for Learning with Distribution Shift
2025-10-01 11:00 - 12:00 | HDSI 123 and Virtual, 3234 Matthews Ln, La Jolla, CA, 92093, United States
Adam Klivans, The University of Texas at Austin
Abstract: We revisit the fundamental problem of learning with distribution shift, where a learner is given labeled samples from training distribution D, unlabeled samples from test distribution D′ and is asked to output a classifier with low test error. The standard approach in this setting is to prove a generalization bound in terms of some notion of distance between D and D′. These distances, however, are difficult to compute, and this has been the main stumbling block for efficient algorithm design over the last two decades.
We sidestep this issue and define a new model called TDS learning, where a learner runs a test on the training set and is allowed to reject if this test detects distribution shift relative to a fixed output classifier. This approach leads to the first set of efficient algorithms for learning with distribution shift that do not take any assumptions on the test distribution. Finally, we discuss how our techniques have recently been used to solve longstanding problems in supervised learning with contamination.

Adam Klivans is a Professor of Computer Science at the University of Texas at Austin and Director of the NSF AI Institute for Foundations of Machine Learning (IFML). His research interests lie in machine learning and theoretical computer science, in particular, Learning Theory, Computational Complexity, Pseudorandomness, Limit Theorems, and Gaussian Space. Dr. Klivans is a recipient of the NSF CAREER Award and serves on the editorial board for the Theory of Computing and Machine Learning Journal.

Optimization for ML and AI Seminar: High-dimensional Optimization with Applications to Compute-Optimal Neural Scaling Laws
2025-10-24 11:00 - 12:00 | CSE 1242 and Virtual, 3235 Voigt Dr, La Jolla, CA, 92093, United States
Courtney Paquette, McGill University
Abstract: Given the massive scale of modern ML models, we now only get a single shot to train them effectively. This restricts our ability to test multiple architectures and hyper-parameter configurations. Instead, we need to understand how these models scale, allowing us to experiment with smaller problems and then apply those insights to larger-scale models. In this talk, I will present a framework for analyzing scaling laws in stochastic learning algorithms using a power-law random features model (PLRF), leveraging high-dimensional probability and random matrix theory. I will then use this scaling law to address the compute-optimal question: How should we choose model size and hyper-parameters to achieve the best possible performance in the most compute-efficient manner? Then using this PLRF model, I will devise a new momentum-based algorithm that (provably) improves the scaling law exponent. Finally, I will present some numerical experiments on LSTMs that show how this new stochastic algorithm can be applied to real data to improve the compute-optimal exponent.

Courtney Paquette is an assistant professor at McGill University in the Mathematics and Statistics department, a CIFAR AI Chair (MILA), and an active member of the Montreal Machine Learning Optimization Group (MTL MLOpt) at MILA. Her research broadly focuses on designing and analyzing algorithms for large-scale optimization problems, motivated by applications in data science, and using techniques that draw from a variety of fields, including probability, complexity theory, and convex and nonsmooth analysis. Dr. Paquette is a lead organizer of the OPT-ML Workshop at NeurIPS since 2020, and a lead organizer (and original creator) of the High-dimensional Learning Dynamics (HiLD) Workshop at ICML.

Student and Postdoc Lunch at Zanzibar Cafe
2025-10-24 12:00 - 13:30 | Zanzibar Cafe at UC San Diego
Join fellow TILOS students and postdoctoral researchers for an informal lunch at Zanzibar Cafe, located on the second floor of Price Center.

TILOS-HDSI Seminar: AI safety theory: the missing middle ground
2025-11-12 11:00 - 12:00 | HDSI 123 and Virtual, 3234 Matthews Ln, La Jolla, CA, 92093, United States
Adam Oberman, McGill University
Abstract: Over the past few years, the capabilities of generative artificial intelligence (AI) systems have advanced rapidly. Along with the benefits of AI, there is also a risk of harm. In order to benefit from AI while mitigating the risks, we need a grounded theoretical framework.
The current AI safety theory, which predates generative AI, is insufficient. Most theoretical AI safety results tend to reason absolutely: a system is a system is “aligned” or “mis-aligned”, “honest” or “dishonest”. But in practice safety is probabilistic, not absolute. The missing middle ground is a quantitative or relative theory of safety — a way to reason formally about degrees of safety. Such a theory is required for defining safety and harms, and is essential for technical solutions as well as for making good policy decisions.
In this talk I will:

Review current AI risks (from misuse, from lack of reliability, and systemic risks to the economy) as well as important future risks (lack of control).
Review theoretical predictions of bad AI behavior and discuss experiments which demonstrate that they can occur in current LLMs.
Explain why technical and theoretical safety solutions are valuable, even by contributors outside of the major labs.
Discuss some gaps in the theory and present some open problems which could address the gaps.

Adam Oberman is a Full Professor of Mathematics and Statistics at McGill University, a Canada CIFAR AI Chair, and an Associate Member of Mila. He is a research collaborator at LawZero, Yoshua Bengio’s AI Safety Institute. He has been researching AI safety since 2024. His research spans generative models, reinforcement learning, optimization, calibration, and robustness. Earlier in his career, he made significant contributions to optimal transport and nonlinear partial differential equations. He earned degrees from the University of Toronto and the University of Chicago, and previously held faculty and postdoctoral positions at Simon Fraser University and the University of Texas at Austin.

TILOS-SDSU Seminar: Certifiably Correct Machine Perception
2025-11-19 11:00 - 12:00 | Lamden Hall 341 (SDSU) and Virtual, San Diego, CA, 92182, United States
David Rosen, Northeastern University
Abstract: Many fundamental machine perception and state estimation tasks require the solution of a high-dimensional nonconvex estimation problem; this class includes (for example) the fundamental problems of simultaneous localization and mapping (in robotics), 3D reconstruction (in computer vision), and sensor network localization (in distributed sensing). Such problems are known to be computationally hard in general, with many local minima that can entrap the smooth local optimization methods commonly applied to solve them. The result is that standard machine perception algorithms (based upon local optimization) can be surprisingly brittle, often returning egregiously wrong answers even when the problem to which they are applied is well-posed.
In this talk, we present a novel class of certifiably correct estimation algorithms that are capable of efficiently recovering provably good (often globally optimal) solutions of generally-intractable machine perception problems in many practical settings. Our approach directly tackles the problem of nonconvexity by employing convex relaxations whose minimizers provide provably good approximate solutions to the original estimation problem under moderate measurement noise. We illustrate the design of this class of methods using the fundamental problem of pose-graph optimization (a mathematical abstraction of robotic mapping) as a running example. We conclude with a brief discussion of open questions and future research directions.

David M. Rosen is an Assistant Professor in the Departments of Electrical & Computer Engineering and Mathematics and the Khoury College of Computer Sciences (by courtesy) at Northeastern University, where he leads the Robust Autonomy Laboratory (NEURAL). Prior to joining Northeastern, he was a Research Scientist at Oculus Research (now Meta Reality Labs) from 2016 to 2018, and a Postdoctoral Associate at MIT’s Laboratory for Information and Decision Systems (LIDS) from 2018 to 2021. He holds the degrees of B.S. in Mathematics from the California Institute of Technology (2008), M.A. in Mathematics from the University of Texas at Austin (2010), and ScD in Computer Science from the Massachusetts Institute of Technology (2016).

He is broadly interested in the mathematical and algorithmic foundations of trustworthy machine perception, learning, and control. His work has been recognized with the IEEE Transactions on Robotics Best Paper Award (2024), an Honorable Mention for the IEEE Transactions on Robotics Best Paper Award (2021), a Best Student Paper Award at Robotics: Science and Systems (2020), a Best Paper Award at the International Workshop on the Algorithmic Foundations of Robotics (2016), and selection as an RSS Pioneer (2019).

Workshop on Topology, Algebra, and Geometry in Data Science (co-located with NeurIPS 2025)
2025-12-01 - 2025-12-02 | UC San Diego, La Jolla, CA, 92093, United States
We are thrilled to announce the first official TAG-DS Stand-Alone Event–TAG… We’re it! This will be a two day event, December 1 & 2, 2025, featuring keynotes, poster sessions, spotlight talks, collaboration activities, and community development. The dates and location were selected to align with NeurIPS 2025–twice the fun! The event will be hosted on the University of California San Diego campus both days and is readily accessible by public transit from downtown for those already planning to attend NeurIPS. There will be an associated Proceedings of Machine Learning Research volume for papers submitted to the archival track.

TILOS-SDSU Seminar: 95 Percent: Bridging the Gap Between Prototype and Product
2025-12-03 11:00 - 12:00 | Lamden Hall 341 (SDSU) and Virtual, San Diego, CA, 92182, United States
Jeremy Schwartz, Zoox
Abstract: When transitioning from the academic world to the professional world of engineering, one of the most common pitfalls is failing to understand the difference between a compelling prototype and a successful product. This talk will focus on that distinction. We will discuss the differences between them, and the work required to evolve a good prototype into a real product. We will also discuss some common pitfalls encountered in product development, and some of the practical software design considerations to keep in mind for development of robust, mature code. The talk will include examples from my background developing robotic systems for air, space, and ground.

Jeremy Schwartz is a robotics engineer at Zoox with expertise in a wide variety of areas of mechanical and electrical engineering and computer science. His primary professional expertise is in autonomy and behavioral algorithms, and he has worked in the aerospace industry as well as ground robotics, specializing in autonomous systems of all kinds.

Optimization for AI and ML Seminar: Training Neural Networks at Any Scale
2025-12-03 13:00 - 14:00 | HDSI 123 and Virtual, 3234 Matthews Ln, La Jolla, CA, 92093, United States
Volkan Cevher, École Polytechnique Fédérale de Lausanne
Abstract: At the heart of deep learning’s transformative impact lies the concept of scale–encompassing both data and computational resources, as well as their interaction with neural network architectures. Scale, however, presents critical challenges, such as increased instability during training and prohibitively expensive model-specific tuning. Given the substantial resources required to train such models, formulating high-confidence scaling hypotheses backed by rigorous theoretical research has become paramount.
To bridge theory and practice, the talk explores a key mathematical ingredient of scaling in tandem with scaling theory: the numerical solution algorithms commonly employed in deep learning, spanning domains from vision to language models. We unify these algorithms under a common master template, making their foundational principles transparent. In doing so, we reveal the interplay between adaptation to smoothness structures via online learning and the exploitation of optimization geometry through non-Euclidean norms. Our exposition moves beyond simply building larger models–it emphasizes strategic scaling, offering insights that promise to advance the field while economizing on resources.

Volkan Cevher received the B.Sc. (valedictorian) in electrical engineering from Bilkent University in Ankara, Turkey, in 1999 and the Ph.D. in electrical and computer engineering from the Georgia Institute of Technology in Atlanta, GA in 2005. He was a Research Scientist with the University of Maryland, College Park from 2006-2007 and also with Rice University in Houston, TX, from 2008-2009. Currently, he is an Associate Professor at the Swiss Federal Institute of Technology Lausanne and a Faculty Fellow in the Electrical and Computer Engineering Department at Rice University. His research interests include machine learning, signal processing theory, optimization theory and methods, and information theory. Dr. Cevher is an ELLIS fellow and was the recipient of the Google Faculty Research award in 2018, the IEEE Signal Processing Society Best Paper Award in 2016, a Best Paper Award at CAMSAP in 2015, a Best Paper Award at SPARS in 2009, and an ERC CG in 2016 as well as an ERC StG in 2011.

Networking Lunch Reception at NeurIPS 2025
2025-12-04 12:00 - 14:00 | Mezé Greek Fusion, San Diego, CA, United States
TILOS will host a networking lunch reception during NeurIPS 2025 at Mezé Greek Fusion from 12:00-2:00pm on Thursday, December 4, 2025. This event is open to all NeurIPS attendees affiliated with any of the NSF AI Research Institutes, as well as invited industry partners. Join us to connect with colleagues across the network of NSF AI Institutes, share research interests, and explore opportunities for collaboration.
Registration has closed. Please contact tilos@ucsd.edu with any questions.
? Date: Thursday, December 4, 2025? Time: 12:00 – 2:00pm PST? Location: Mezé Greek Fusion (3 blocks from the conference venue)

Optimization for ML and AI Seminar: Stochastic-Gradient and Diagonal-Scaling Algorithms for Constrained Optimization and Learning
2025-12-05 11:00 - 12:00 | HDSI 123 and Virtual, 3234 Matthews Ln, La Jolla, CA, 92093, United States
Frank E. Curtis, Lehigh University
Abstract: I will motivate and provide an overview of recent efforts in my research group on the design and analysis of stochastic-gradient-based algorithms for solving constrained optimization problems. I will focus in particular on our motivation for informed supervised learning, where constraints in the training problem can be used to impose prior knowledge on the properties that should be possessed by a trained prediction model. In addition, I will provide a detailed look at our newest extensions of heavy-ball and Adam schemes from the unconstrained to the equality-constrained setting, for which we have shown state-of-the-art convergence guarantees. I will demonstrate the impressive practical performance of our methods using a few informed supervised learning problems.

Frank E. Curtis is a Professor in the Department of Industrial and Systems Engineering at Lehigh University, where he has been employed since 2009. He received a bachelor’s degree from the College of William and Mary in 2003 with a double major in Computer Science and Mathematics, received a master’s degree in 2004 and Ph.D. degree in 2007 from the Department of Industrial Engineering and Management Science at Northwestern University, and spent two years as a Postdoctoral Researcher in the Courant Institute of Mathematical Sciences at New York University from 2007 until 2009. His research focuses on the design, analysis, and implementation of numerical methods for solving large-scale nonlinear optimization problems. He received an Early Career Award from the Advanced Scientific Computing Research (ASCR) program of the U.S. Department of Energy (DoE), and has received funding from various programs of the U.S. National Science Foundation (NSF), including through a TRIPODS Phase I grant awarded to him and his collaborators at Lehigh, Northwestern, and Boston University. He has also received funding from the U.S. Office of Naval Research (ONR) and DoE’s Advanced Research Projects Agency-Energy (ARPA-E). He received, along with Leon Bottou (Meta AI) and Jorge Nocedal (Northwestern), the 2021 SIAM/MOS Lagrange Prize in Continuous Optimization. He was awarded, with James V. Burke (U. of Washington), Adrian Lewis (Cornell), and Michael Overton (NYU), the 2018 INFORMS Computing Society Prize. He and team members Daniel Molzahn (Georgia Tech), Andreas Waechter (Northwestern), Ermin Wei (Northwestern), and Elizabeth Wong (UC San Diego) were awarded second place in the ARPA-E Grid Optimization Competition in 2020. He currently serves as Area Editor for Continuous Optimization for Mathematics of Operations Research and serves as an Associate Editor for Mathematical Programming, SIAM Journal on Optimization, Operations Research, IMA Journal of Numerical Analysis, and Mathematical Programming Computation. He previously served as the Vice Chair for Nonlinear Programming for the INFORMS Optimization Society, and is currently very active in professional societies and groups related to mathematical optimization, including INFORMS, the Mathematics Optimization Society, and the SIAM Activity Group on Optimization.

NeurIPS 2025 Workshop on Differentiable Learning of Combinatorial Algorithms
2025-12-06 | San Diego Convention Center, San Diego, CA, United States
Combinatorial algorithms are fundamental across a wide range of domains, owing to their ability to model optimization and decision-making tasks under complex constraints. These algorithms underpin practical applications such as vehicle routing, network and chip design, clustering and information retrieval. Combinatorial problems are also prominent in various areas of machine learning such as natural language processing and robotics. Recent research has focused on leveraging neural networks to design novel combinatorial algorithms and to come up with techniques that allow seamless integration of classic combinatorial algorithms in differentiable neural network architectures. Developments in this field, commonly referred to as neural combinatorial optimization, have raised several fundamental questions that span both theory and practice.
In this workshop, we take a broad perspective on designing differentiable algorithms for combinatorial optimization. The goal of the workshop is to explore novel ideas in the design and applications of neural combinatorial optimization, as well as to improve our theoretical understanding of existing methods.

NeurIPS 2025 Workshop on Optimization for Machine Learning
2025-12-06 | San Diego Convention Center, San Diego, CA, United States
Optimization lies at the heart of many machine learning algorithms and enjoys great interest in our community. Indeed, this intimate relation of optimization with ML is the key motivation for the OPT series of workshops. We aim to foster discussion, discovery, and dissemination of state-of-the-art research in optimization relevant to ML.
The focus of OPT 2025 is on “Statistics Meets Optimization”. Since its inception, stochastic optimization has been grounded in statistical principles. Today, many of the most pressing challenges in machine learning—such as generalization bounds, the training dynamics of overparameterized models, and the development of generative models—are directly inspired by statistical thinking. At the same time, the scale and complexity of modern datasets, along with the increasingly rich model classes used to represent them, pose new questions about how optimization algorithms interact with these structures—both computationally and statistically. For example, what role do data symmetries play in shaping optimization trajectories? How do statistical properties of the data affect the adaptivity and efficiency of learning algorithms? And how can optimization approaches be designed to scale with data while still preserving desirable statistical behavior? OPT 2025 will explore these questions with the goal of building bridges between the statistics and optimization communities, and highlighting their shared impact on the theory and practice of machine learning.

NeurIPS 2025 Workshop on Imageomics: Discovering Biological Knowledge from Images Using AI
2025-12-06 | San Diego Convention Center, San Diego, CA, United States
Imageomics is an emerging interdisciplinary field at the crossroads of machine learning (ML), computer vision (CV), and biological sciences. It leverages visual data—from microscopic images of single-cell species to videos of megafauna—to extract and analyze biological information, specifically traits. By grounding ML models in existing scientific knowledge, Imageomics aims to make traits computable from images, facilitating insights into the evolution and function of living organisms. Imageomics poses research problems that resonate with the broad machine-learning community: multimodal representation learning, object detection and tracking, few-shot learning, imbalanced-class learning, video understanding, 3D modeling, hierarchical learning, etc. When people leverage ML tools to solve biological questions, the foundational bridges between ML and biological sciences also provide opportunities to address key challenges in ML, creating a virtuous cycle between the two fields.
We welcome participation from anyone interested in learning about the field of Imageomics, including:

Biological Scientists who are interested in applying ML and CV to their research, or who want to learn how to use ML tools to analyze biological images.
Machine Learning Researchers who are interested in applying their expertise to biological image data, or who want to learn about the unique challenges and opportunities in this domain.

The workshop will feature keynote talks, paper presentations, and discussions on the latest research in Imageomics. We encourage participants to submit papers and demos related to the topics outlined in the Call For Papers. The workshop will also provide opportunities for networking and collaboration among researchers from diverse backgrounds.

NeurIPS 2025 Workshop on New Perspectives in Advancing Graph Machine Learning
2025-12-07 | San Diego Convention Center, San Diego, CA, United States
Graphs serve as a powerful representational framework for machine learning, and their integration has substantially advanced the field. Indeed, extensive studies have pushed forward graph machine learning (GML) in both theory and applications. Recently, new perspectives have been emerging in the machine learning community, including algebraic–topological analyses, foundation models, generative models, and large models in applications. Leveraging these ideas for core graph machine learning holds a lot of promise, including the dual benefit of deeper theoretical insight, new capabilities and more powerful, application-aligned algorithms and models. The aim of this workshop is to explore and connect these new perspectives on GML, and to identify overarching challenges and tools – in terms of theory, methodology, and modeling.

TILOS-HDSI Seminar: Incentivizing Emergent Behaviors for LLMs via Reinforcement Learning
2025-12-08 10:00 - 11:00 | Qualcomm Conference Center Room B (Jacobs Hall first floor) and Virtual, 9736 Engineers Ln, La Jolla, CA, 92093, United States
Yi Wu, Tsinghua University
Abstract: Reinforcement Learning (RL) has become a powerful post-training method for eliciting advanced behaviors in large language models (LLMs). This talk presents recent results showing how RL can incentivize the emergence of LLM capabilities across three domains: (1) multi-player deduction game, Werewolf, where RL-trained LLM agents develop strategic behaviors and outperform strong human players; (2) agentic search, where large-scale RL enables a 32B model to run multi-step search to answer non-trivial questions beyond commercial baselines; and (3) efficient reasoning, where RL mitigates over-thinking and improves both reliability and compute efficiency.
The papers can be found at

Werewolf: https://arxiv.org/abs/2310.18940 (ICML24), https://arxiv.org/abs/2502.04686 (ICML25)
ASearcher: https://arxiv.org/abs/2508.07976
Thinking Efficiency: https://www.arxiv.org/abs/2506.07104 (NeurIPS25)

All the projects are trained using our large-scale agentic RL system, AReaL, which is open-source at https://github.com/inclusionAI/AReaL with its paper at https://arxiv.org/abs/2505.24298 (NeurIPS25).

Yi Wu is an assistant professor at the Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University. He obtained his Ph.D. from UC Berkeley and was a researcher at OpenAI from 2019 to 2020. His research focuses on reinforcement learning, multi-agent learning, and LLM agents. His representative works include the value iteration network, the MADDPG/MAPPO algorithm, OpenAI’s hide-and-seek project, and the AReaL project. He received the best paper award at NIPS 2016, the best demo award finalist at ICRA 2024, and MIT TR35 Asia Pacific 2025 award.

Optimization for ML and AI Seminar: Randomized linear algebra with subspace injections
2026-01-09 11:00 - 12:00 | HDSI 123 and Virtual, 3234 Matthews Ln, La Jolla, CA, 92093, United States
Joel Tropp, Caltech
Abstract: To achieve the greatest possible speed, practitioners regularly implement randomized algorithms for low-rank approximation and least-squares regression with structured dimension reduction maps. This talk outlines a new perspective on structured dimension reduction, based on the injectivity properties of the dimension reduction map. This approach provides sharper bounds for sparse dimension reduction maps, and it leads to exponential improvements for tensor-product dimension reduction. Empirical evidence confirms that these types of structured random matrices offer exemplary performance for a range of synthetic problems and contemporary scientific applications.
Joint work with Chris Camaño, Ethan Epperly, and Raphael Meyer; available at arXiv:2508.21189.

Joel A. Tropp is Steele Family Professor of Applied & Computational Mathematics at the California Institute of Technology. His research centers on applied mathematics, machine learning, data science, numerical algorithms, and random matrix theory. Some of his best-known contributions include matching pursuit algorithms, randomized SVD algorithms, matrix concentration inequalities, and statistical phase transitions. Prof. Tropp attained the Ph.D. degree in Computational Applied Mathematics at the University of Texas at Austin in 2004, and he joined Caltech in 2007. He won the PECASE in 2008, and he was recognized as a Highly Cited Researcher in Computer Science each year from 2014–2018. He is co-founder of the SIAM Journal on Mathematics of Data Science (SIMODS), and he was co-chair of the inaugural 2020 SIAM Conference on the Mathematics of Data Science. Prof. Tropp was elected SIAM Fellow in 2019, IEEE Fellow in 2020, and IMS Fellow in 2024. He received the 2025 Richard P. Feynman Prize for Excellence in Teaching at Caltech. He is an invited speaker at the 2026 International Congress of Mathematicians (ICM).

Gordon Research Conference on Embodied Intelligence
2026-01-11 - 2026-01-16 | Four Points Sheraton / Holiday Inn Express, 1050 Schooner Drive, Ventura, CA, 93001, United States
The Robotics GRC is a premier, international scientific conference focused on advancing the frontiers of science through the presentation of cutting-edge and unpublished research, prioritizing time for discussion after each talk and fostering informal interactions among scientists of all career stages. The conference program includes an array of speakers and discussion leaders from institutions and organizations worldwide, concentrating on the latest developments in the field. The conference is five days long and held in a remote location to increase the sense of camaraderie and create scientific communities, with lasting collaborations and friendships. In addition to premier talks, the conference has designated time for poster sessions from individuals of all career stages, and afternoon free time and communal meals allow for informal networking opportunities with leaders in the field.
This year’s conference will focus on adaptive behavior and learning in animals and robots. We will explore how biological inspiration drives advancements in robotics, from simple reactive behaviors to complex planning and learning systems. Insights from biomechanics, neuroscience, and animal studies are increasingly shaping the design and control of robots, making them more robust and adaptable. A key focus will be on embodied intelligence, enabling robots to excel in locomotion, manipulation, and interactions with other agents.
Conversely, robotics research is also contributing to biology. Studies of perception and action in robotic systems are leading to new mathematical models that help integrative biologists understand locomotion, manipulation, and collective behavior in animals.
By bringing together experts from robotics, biomechanics, and neuroscience, this conference aims to foster cross-disciplinary insights that will push the boundaries of both fields.
Join us for a dynamic exchange of ideas at the intersection of robotics and biology, where engineering meets evolution.

TILOS-HDSI Seminar: Safety, Representations, and Generative Learning in Dynamical Systems
2026-01-28 11:00 - 12:00 | HDSI 123 and Virtual, 3234 Matthews Ln, La Jolla, CA, 92093, United States
Koushil Sreenath, UC Berkeley
Abstract: This talk explores the interplay between model-based guarantees and learning-based flexibility in the control of dynamical systems. I begin with safety-critical control using control barrier functions (CBFs), highlighting that while CBFs enforce state constraints, they may induce unstable internal dynamics. I introduce conditions under which CBF-based safety filters ensure boundedness of the full system state. I then transition to learning representations of hybrid dynamical systems. I present a framework that learns continuous neural representations by exploiting the geometric structure induced by guards and resets, enabling accurate flow prediction without explicit mode switching. Finally, I discuss generative learning approaches for control, emphasizing guided diffusion models that jointly represent states and actions. Through applications to agile humanoid locomotion, motion synthesis, and dynamic manipulation, I demonstrate how generative models can produce versatile, long-horizon behaviors while respecting physical constraints. Together, these results highlight how structure, geometry, and learning can bridge safety guarantees and expressive control in complex dynamical systems.

Koushil Sreenath is an Associate Professor of Mechanical Engineering, at UC Berkeley. He received a Ph.D. degree in Electrical Engineering and Computer Science and a M.S. degree in Applied Mathematics from the University of Michigan at Ann Arbor, MI, in 2011. He was a Postdoctoral Scholar at the GRASP Lab at University of Pennsylvania from 2011 to 2013 and an Assistant Professor at Carnegie Mellon University from 2013 to 2017. His research interest lies at the intersection of highly dynamic robotics and applied nonlinear control. His work on dynamic legged locomotion was featured on The Discovery Channel, CNN, ESPN, FOX, and CBS. His work on dynamic aerial manipulation was featured on the IEEE Spectrum, New Scientist, and Huffington Post. His work on adaptive sampling with mobile sensor networks was published as a book. He received the NSF CAREER, Hellman Fellow, Google Faculty Research Award in Robotics, and Best Paper Awards at Learning for Dynamics and Control (L4DC) and Robotics: Science and Systems (RSS).

[CANCELED] Optimization for ML and AI Seminar: Fantastic Pretraining Optimizers and Where to Find Them
2026-01-30 11:00 - 12:00 | HDSI 123 and Virtual, 3234 Matthews Ln, La Jolla, CA, 92093, United States
Tengyu Ma, Stanford
Abstract: AdamW has long been the dominant optimizer in language model pretraining, despite numerous claims that alternative optimizers offer 1.4 to 2x speedup. We posit that two methodological shortcomings have obscured fair comparisons and hindered practical adoption: (i) unequal hyperparameter tuning and (ii) limited or misleading evaluation setups. To address these two issues, we conduct a systematic study of ten deep learning optimizers across four model scales (0.1B-1.2B parameters) and data-to-model ratios (1-8x the Chinchilla optimum). We find that fair and informative comparisons require rigorous hyperparameter tuning and evaluations across a range of model scales and data-to-model ratios, performed at the end of training. First, optimal hyperparameters for one optimizer may be suboptimal for another, making blind hyperparameter transfer unfair. Second, the actual speedup of many proposed optimizers over well-tuned baselines is lower than claimed and decreases with model size to only 1.1x for 1.2B parameter models. Thirdly, comparing intermediate checkpoints before reaching the target training budgets can be misleading, as rankings between two optimizers can flip during training due to learning rate decay. Through our thorough investigation, we find that all the fastest optimizers such as Muon and Soap, use matrices as preconditioners—multiplying gradients with matrices rather than entry-wise scalars. However, the speedup of matrix-based optimizers is inversely proportional to model scale, decreasing from 1.4x over AdamW for 0.1B parameter models to merely 1.1x for 1.2B parameter models.

Tengyu Ma is an assistant professor of computer science at Stanford University. His research interests broadly include topics in machine learning, algorithms and their theory, such as deep learning, (deep) reinforcement learning, pre-training / foundation models, robustness, non-convex optimization, distributed optimization, and high-dimensional statistics.
Zoom: https://bit.ly/Opt-AI-ML

Optimization for ML and AI Seminar: Extended Convex Lifting for Policy Optimization in Control
2026-02-06 11:00 - 12:00 | HDSI 123 and Virtual, 3234 Matthews Ln, La Jolla, CA, 92093, United States
Yang Zheng, UC San Diego
Abstract: Direct policy search has achieved great empirical success in reinforcement learning. Many recent studies have revisited its theoretical foundation for continuous control, which reveals elegant nonconvex geometry in various benchmark problems. In this talk, we introduce an Extended Convex Lifting (ECL) framework, which reveals hidden convexity in classical optimal and robust control problems from a modern optimization perspective. Our ECL offers a bridge between nonconvex policy optimization and convex reformulations. Despite non-convexity and non-smoothness, the existence of an ECL not only reveals that minimizing the original function is equivalent to a convex problem, but also certifies a class of first-order non-degenerate stationary points to be globally optimal. This ECL framework encompasses many benchmark control problems, including LQR, LQG, state-feedback, and output-feedback H-infinity robust control. We believe that the ECL framework may be of independent interest for analyzing nonconvex problems beyond control.

Yang Zheng is an Assistant Professor in the ECE Department at UC San Diego. His research focuses on control theory, convex and nonconvex optimization, and their applications to autonomous vehicles and traffic systems. He received his DPhil (Ph.D.) in Engineering Science from the University of Oxford in 2019, and his B.E. and M.S. degrees from Tsinghua University in 2013 and 2015, respectively. His work has been recognized with several awards, including the 2019 European Ph.D. Award on Control for Complex and Heterogeneous Systems, the 2022 Best Paper Award from IEEE Transactions on Control of Network Systems, the 2023 Best Graduate Teacher Award from UC San Diego’s ECE Department, the 2024 NSF CAREER Award, and the 2025 Donald P. Eckman Award from the American Automatic Control Council.

TILOS-MICS Seminar: AI-Driven Design Automation for Multi-Chip Integration in AI Chips
2026-02-09 11:00 - 12:00 | HDSI 123 and Virtual, 3234 Matthews Ln, La Jolla, CA, 92093, United States
Sung-Kyu Lim, University of Southern California
Abstract: Multi-chip integration has become a standard approach in AI training and is rapidly gaining traction in edge learning applications. Leveraging 2.5D and 3D IC architecture enables substantial improvements in energy efficiency and latency by optimizing inter chip data transfer. At the core of this transformation lies the automation of design and simulation for heterogeneous AI chips, shifting from manual engineering to algorithm driven methodologies. This evolution is being accelerated by advanced electronic design automation (EDA) tools powered by AI. My research group develops novel AI driven algorithms that enhance or replace traditional design automation techniques, with a focus on enabling next generation heterogeneous AI systems. In this talk, I will present our recent innovations and explore the critical challenges that lie ahead in applying AI algorithms to EDA for high performance AI chip design.

Dr. Sung Kyu Lim is Dean’s Professor of Electrical and Computer Engineering at the University of Southern California, joining in Fall 2025 after over two decades at Georgia Tech. He received his B.S., M.S., and Ph.D. in Computer Science from UCLA. His research focuses on the architecture, design, and electronic design automation (EDA) of 2.5D and 3D integrated circuits, with over 450 publications. Dr. Lim is an IEEE Fellow and recipient of major awards including multiple Best Paper Awards (DAC 2023, TCAD 2022), and several Georgia Tech teaching honors. From 2022 to 2024, he served as a Program Manager at DARPA’s Microsystems Technology Office.

TILOS-HDSI Seminar: Kinetic Theory Perspective of Foundation Models for Physics
2026-02-11 11:00 - 12:00 | HDSI 123 and Virtual, 3234 Matthews Ln, La Jolla, CA, 92093, United States
Maarten de Hoop, Rice University
Abstract: We present a kinetic theory perspective of foundation models for physics. We begin with providing a mathematical framework for analyzing transformers. To uniformly address their expressivity, we consider the case that the mappings are conditioned on a context represented by a probability distribution of tokens. That is, transformers become mappings between probability measures. The relevant notion of smoothness then corresponds to continuity in terms of the Wasserstein distance between such contexts. We demonstrate that deep transformers are universal and can approximate continuous in-context mappings to arbitrary precision, uniformly over compact token domains. We then characterize the conditions on mappings between measures that enable these to be represented in terms of in-context mappings as transformers. The solution map of the Vlasov equation, which is of nonlocal transport type, for interacting particle systems in the mean-field regime for the Cauchy problem satisfies the conditions; conversely, we prove that the measure-theoretic self-attention has the properties that ensure that the infinite depth, mean-field transformer can be identified with a Vlasov flow. Extending this framework from interactions to collisions leads to a further development of structured architectures inspired by Lattice Boltzmann Models, while flow motivates a design based on self-warping.

Professor Maarten V. de Hoop, Simons Chair in Computational and Applied Mathematics and Earth Science at Rice University, is internationally recognized for his contributions to the mathematical foundations of seismology, wave propagation, and inverse problems. His research bridges microlocal and harmonic analysis, scattering theory, and structured numerical methods with applications to seismic imaging, geophysical inversion, and large-scale computational modeling of acoustic, elastic, and electromagnetic phenomena. De Hoop has been a pioneer in developing techniques to extract subtle information from massive, complex seismic datasets, advancing our ability to probe the Earth’s interior with unprecedented resolution, and more recently has integrated deep learning and data-driven discovery with rigorous mathematical frameworks to open new frontiers in the analysis of multiscale wave phenomena and inverse spectral problems. He is the recipient of the J. Clarence Karcher Award from the Society of Exploration Geophysicists and the Young Scientists Award from the International Society for Analysis, its Applications and Computation, has been elected a Fellow of the Institute of Physics and an External Member of the Finnish Academy of Science and Letters, and has served as associate editor for Inverse Problems, Inverse Problems and Imaging, and the International Journal on Geomathematics.

TILOS-HDSI Seminar: Neuromorphic LLMs
2026-02-20 10:00 - 11:00 | HDSI 123 and Virtual, 3234 Matthews Ln, La Jolla, CA, 92093, United States
Jason Eshraghian, UC Santa Cruz
Abstract: This talk will show you what neuromorphic computing can do when an academic lab accidentally pulls $2-million of GPU-hours. We will showcase a series of frontier reasoning LLMs developed out of an academic lab, from data curation and pre-training to post-training and alignment. These models surpass leading LLMs from Meta, Google, and other heavily-resourced labs in the ~10-billion parameter regime, despite being 5x smaller.
We have deployed several models on neuromorphic hardware at just 2 watts, bringing state-of-the-art reasoning from the datacenter to the edge. Along the way, we dispel a series of widely-held assumptions about large-scale neuromorphic computation, revealing how it fundamentally differs from conventional deep learning, and why that difference matters.

Jason Eshraghian is an Assistant Professor and Fulbright Scholar in the Department of Electrical and Computer Engineering at the University of California, Santa Cruz. He is the developer of snnTorch, a Python library with over 500,000 downloads for training spiking neural networks. He is a dual-appointed IEEE CAS and EMBS Distinguished Lecturer, an Associate Editor of APL Machine Learning, the Chair of the IEEE Neural Systems and Applications Technical Committee, has been the recipient of seven IEEE Best Paper Awards, a Scientific Advisory Board Member of BrainChip and leads the Neuromorphic Agents Team at Conscium.
