BEGIN:VCALENDAR
VERSION:2.0
PRODID:-// - ECPv6.15.18//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-ORIGINAL-URL:https://tilos.ai
X-WR-CALDESC:Events for 
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20240310T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20241103T090000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20250309T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20251102T090000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20260308T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20261101T090000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/Los_Angeles:20250327T140000
DTEND;TZID=America/Los_Angeles:20250327T150000
DTSTAMP:20260403T222047
CREATED:20250828T192427Z
LAST-MODIFIED:20250828T192653Z
UID:7273-1743084000-1743087600@tilos.ai
SUMMARY:TILOS Seminar: Single location regression and attention-based models
DESCRIPTION:Claire Boyer\, Université Paris-Saclay \nAbstract: Attention-based models\, such as Transformer\, excel across various tasks but lack a comprehensive theoretical understanding\, especially regarding token-wise sparsity and internal linear representations. To address this gap\, we introduce the single-location regression task\, where only one token in a sequence determines the output\, and its position is a latent random variable\, retrievable via a linear projection of the input. To solve this task\, we propose a dedicated predictor\, which turns out to be a simplified version of a non-linear self-attention layer. We study its theoretical properties\, by showing its asymptotic Bayes optimality and analyzing its training dynamics. In particular\, despite the non-convex nature of the problem\, the predictor effectively learns the underlying structure. This work highlights the capacity of attention mechanisms to handle sparse token information and internal linear structures.
URL:https://tilos.ai/event/tilos-seminar-single-location-regression-and-attention-based-models/
LOCATION:HDSI 123 and Virtual\, 3234 Matthews Ln\, La Jolla\, CA\, 92093\, United States
CATEGORIES:TILOS Seminar Series
ATTACH;FMTTYPE=image/jpeg:https://tilos.ai/wp-content/uploads/2025/08/boyer-claire-e1742860147959-s8d3nW.jpg
END:VEVENT
END:VCALENDAR