BEGIN:VCALENDAR
VERSION:2.0
PRODID:-// - ECPv6.15.18//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-ORIGINAL-URL:https://tilos.ai
X-WR-CALDESC:Events for 
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20240310T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20241103T090000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20250309T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20251102T090000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20260308T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20261101T090000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/Los_Angeles:20250312T110000
DTEND;TZID=America/Los_Angeles:20250312T120000
DTSTAMP:20260403T205627
CREATED:20250828T192527Z
LAST-MODIFIED:20250828T192602Z
UID:7295-1741777200-1741780800@tilos.ai
SUMMARY:TILOS Seminar: Synthetic Tasks as Testbeds for Attributing Model Behavior
DESCRIPTION:Surbhi Goel\, University of Pennsylvania \nAbstract: Understanding how different components of the machine learning pipeline—spanning data composition\, architectural choices\, and optimization dynamics—shape model behavior remains a fundamental challenge. In this talk\, I will argue that synthetic tasks\, which enable precise control over data distribution and task complexity\, serve as powerful testbeds for analyzing and attributing behaviors in deep learning. Focusing on the sparse parity learning problem\, a canonical task in learning theory\, I will present insights into: (1) the phenomenon of “hidden progress” in gradient-based optimization\, where models exhibit consistent advancement despite stagnating loss curves; (2) nuanced trade-offs between data\, compute\, model width\, and initialization that govern learning success; and (3) the role of progressive distillation in implicitly structuring curricula to accelerate feature learning. These findings highlight the utility of synthetic tasks in uncovering empirical insights into the mechanisms driving deep learning\, without the cost of training expensive models. This talk is based on joint work with a lot of amazing collaborators: Boaz Barak\, Ben Edelman\, Sham Kakade\, Bingbin Liu\, Eran Malach\, Sadhika Malladi\, Abhishek Panigrahi\, Andrej Risteski\, and Cyril Zhang. \n\nSurbhi Goel is the Magerman Term Assistant Professor of Computer and Information Science at the University of Pennsylvania. She is associated with the theory group\, the ASSET Center on safe\, explainable\, and trustworthy AI systems\, and the Warren Center for Network and Data Sciences. Surbhi’s research focuses on theoretical foundations of modern machine learning paradigms\, particularly deep learning\, and is supported by Microsoft Research and OpenAI. Previously\, she was a postdoctoral researcher at Microsoft Research NYC and completed her Ph.D. at the University of Texas at Austin under Adam Klivans\, receiving the UTCS Bert Kay Dissertation Award. She has also been a visiting researcher at IAS\, Princeton\, and the Simons Institute at UC Berkeley. Surbhi co-founded the Learning Theory Alliance (LeT‐All) and holds several leadership roles\, including Office Hours co-chair for ICLR 2024 and co-treasurer for the Association for Computational Learning Theory.
URL:https://tilos.ai/event/tilos-seminar-synthetic-tasks-as-testbeds-for-attributing-model-behavior/
LOCATION:HDSI 123 and Virtual\, 3234 Matthews Ln\, La Jolla\, CA\, 92093\, United States
CATEGORIES:TILOS Seminar Series
ATTACH;FMTTYPE=image/jpeg:https://tilos.ai/wp-content/uploads/2025/08/goel-surbhi-e1727126779765-U5P80t.jpg
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=America/Los_Angeles:20250327T140000
DTEND;TZID=America/Los_Angeles:20250327T150000
DTSTAMP:20260403T205627
CREATED:20250828T192427Z
LAST-MODIFIED:20250828T192653Z
UID:7273-1743084000-1743087600@tilos.ai
SUMMARY:TILOS Seminar: Single location regression and attention-based models
DESCRIPTION:Claire Boyer\, Université Paris-Saclay \nAbstract: Attention-based models\, such as Transformer\, excel across various tasks but lack a comprehensive theoretical understanding\, especially regarding token-wise sparsity and internal linear representations. To address this gap\, we introduce the single-location regression task\, where only one token in a sequence determines the output\, and its position is a latent random variable\, retrievable via a linear projection of the input. To solve this task\, we propose a dedicated predictor\, which turns out to be a simplified version of a non-linear self-attention layer. We study its theoretical properties\, by showing its asymptotic Bayes optimality and analyzing its training dynamics. In particular\, despite the non-convex nature of the problem\, the predictor effectively learns the underlying structure. This work highlights the capacity of attention mechanisms to handle sparse token information and internal linear structures.
URL:https://tilos.ai/event/tilos-seminar-single-location-regression-and-attention-based-models/
LOCATION:HDSI 123 and Virtual\, 3234 Matthews Ln\, La Jolla\, CA\, 92093\, United States
CATEGORIES:TILOS Seminar Series
ATTACH;FMTTYPE=image/jpeg:https://tilos.ai/wp-content/uploads/2025/08/boyer-claire-e1742860147959-s8d3nW.jpg
END:VEVENT
END:VCALENDAR