BEGIN:VCALENDAR
VERSION:2.0
PRODID:-// - ECPv6.15.20//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-ORIGINAL-URL:https://tilos.ai
X-WR-CALDESC:Events for 
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20250309T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20251102T090000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20260308T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20261101T090000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20270314T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20271107T090000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/Los_Angeles:20260508T100000
DTEND;TZID=America/Los_Angeles:20260508T110000
DTSTAMP:20260423T114318
CREATED:20260408T183052Z
LAST-MODIFIED:20260408T183052Z
UID:8257-1778234400-1778238000@tilos.ai
SUMMARY:Optimization for ML and AI Seminar: Fantastic Pretraining Optimizers and Where to Find Them
DESCRIPTION:Tengyu Ma\, Stanford \nAbstract: AdamW has long been the dominant optimizer in language model pretraining\, despite numerous claims that alternative optimizers offer 1.4 to 2x speedup. We posit that two methodological shortcomings have obscured fair comparisons and hindered practical adoption: (i) unequal hyperparameter tuning and (ii) limited or misleading evaluation setups. To address these two issues\, we conduct a systematic study of ten deep learning optimizers across four model scales (0.1B-1.2B parameters) and data-to-model ratios (1-8x the Chinchilla optimum). We find that fair and informative comparisons require rigorous hyperparameter tuning and evaluations across a range of model scales and data-to-model ratios\, performed at the end of training. First\, optimal hyperparameters for one optimizer may be suboptimal for another\, making blind hyperparameter transfer unfair. Second\, the actual speedup of many proposed optimizers over well-tuned baselines is lower than claimed and decreases with model size to only 1.1x for 1.2B parameter models. Thirdly\, comparing intermediate checkpoints before reaching the target training budgets can be misleading\, as rankings between two optimizers can flip during training due to learning rate decay. Through our thorough investigation\, we find that all the fastest optimizers such as Muon and Soap\, use matrices as preconditioners—multiplying gradients with matrices rather than entry-wise scalars. However\, the speedup of matrix-based optimizers is inversely proportional to model scale\, decreasing from 1.4x over AdamW for 0.1B parameter models to merely 1.1x for 1.2B parameter models. \n\nTengyu Ma is an assistant professor of computer science at Stanford University. His research interests broadly include topics in machine learning\, algorithms and their theory\, such as deep learning\, (deep) reinforcement learning\, pre-training / foundation models\, robustness\, non-convex optimization\, distributed optimization\, and high-dimensional statistics. \nZoom: https://bit.ly/Opt-AI-ML
URL:https://tilos.ai/event/optimization-for-ml-and-ai-seminar-fantastic-pretraining-optimizers-and-where-to-find-them/
LOCATION:HDSI 123 and Virtual\, 3234 Matthews Ln\, La Jolla\, CA\, 92093\, United States
CATEGORIES:TILOS Seminar Series,TILOS Sponsored Event
ATTACH;FMTTYPE=image/jpeg:https://tilos.ai/wp-content/uploads/2025/10/ma-tengyu-e1760473083457.jpg
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/Los_Angeles:20260515T100000
DTEND;TZID=America/Los_Angeles:20260515T110000
DTSTAMP:20260423T114318
CREATED:20260413T175443Z
LAST-MODIFIED:20260413T175503Z
UID:8269-1778839200-1778842800@tilos.ai
SUMMARY:Optimization for ML and AI Seminar with Nigel Goldenfeld (UC San Diego)
DESCRIPTION:Nigel Goldenfeld\, UC San Diego \nAbstract: TBA \n\nNigel Goldenfeld holds the Chancellor’s Distinguished Professorship in Physics at UC San Diego\, which he joined in Fall 2021 after 36 years at the University of Illinois at Urbana-Champaign (UIUC). Nigel’s research spans condensed matter theory\, the theory of living systems\, hydrodynamics and non-equilibrium statistical physics.  \nNigel received his PhD in theoretical physics from the University of Cambridge (UK) in 1982\, and from 1982-1985 was a postdoctoral fellow at the Institute for Theoretical Physics at UC Santa Barbara\, where his work on the dynamics of snowflake growth helped launch the modern theory of pattern formation in nature. He joined the condensed matter theory group at the Department of Physics at UIUC in 1985\, where his work was instrumental to the discovery of d-wave pairing in high temperature superconductors. Nigel’s interests in biology include microbial ecology\, evolution and systems biology. He was a founding member of the Institute for Genomic Biology at UIUC\, where he led the Biocomplexity Group and directed the NASA Astrobiology Institute for Universal Biology. During the COVID-19 pandemic\, he pivoted from his experience in mathematical modeling of bacteria and viruses to computational epidemiology\, advising the Governor of Illinois\, and helping devise\, set up and run the COVID saliva testing system at UIUC\, which provided ~12 hour turnaround of PCR tests to the 50\,000 people in the campus community and eventually to over 1700 schools and other institutions in Illinois and beyond. Nigel has served on the editorial boards of several journals\, including The Philosophical Transactions of the Royal Society\, Physical Biology and the International Journal of Theoretical and Applied Finance. Selected honors include: Alfred P. Sloan Foundation Fellow\, University Scholar of the University of Illinois\, the Xerox Award for research\, the A. Nordsieck award for excellence in graduate teaching and the American Physical Society’s Leo P. Kadanoff Prize 2020. Nigel is a Fellow of the American Physical Society\, a Fellow of the American Academy of Arts and Sciences\, a Fellow of the Royal Society (UK) and a Member of the US National Academy of Sciences. \nZoom: https://bit.ly/Opt-AI-ML
URL:https://tilos.ai/event/optimization-for-ml-and-ai-seminar-with-nigel-goldenfeld-uc-san-diego/
LOCATION:HDSI 123 and Virtual\, 3234 Matthews Ln\, La Jolla\, CA\, 92093\, United States
CATEGORIES:TILOS Seminar Series,TILOS Sponsored Event
ATTACH;FMTTYPE=image/jpeg:https://tilos.ai/wp-content/uploads/2026/04/goldenfeld-nigel-e1776102861254.jpg
END:VEVENT
END:VCALENDAR