BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Halıcıoğlu Data Science Institute - UC San Diego - ECPv6.17.1//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Halıcıoğlu Data Science Institute - UC San Diego
X-ORIGINAL-URL:https://datascience.ucsd.edu
X-WR-CALDESC:Events for Halıcıoğlu Data Science Institute - UC San Diego
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20250309T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20251102T090000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20260308T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20261101T090000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20270314T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20271107T090000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/Los_Angeles:20260908T110000
DTEND;TZID=America/Los_Angeles:20260908T230000
DTSTAMP:20260827T212231Z
CREATED:20260827T212214Z
LAST-MODIFIED:20260827T212231Z
UID:10000579-1788865200-1788908400@datascience.ucsd.edu
SUMMARY:HDSI Seminar - Ofir Lindenbaum - Efficient Training\, Fast Inference: Reducing Memory Requirements and Inference Time of Foundation Models
DESCRIPTION:Speaker: Ofir Lindenbaum\nTime/Date; Tuesday Sept 8th\, 11:00am\nLocation: HDSI Multipurpose Room 123\nZoom Link: http://bit.ly/HDSI-Seminars\n\nTitle: Efficient Training\, Fast Inference: Reducing Memory Requirements and Inference Time of Foundation Models.\nAbstract: Foundation models present a paradigm shift in machine learning\, where it often appears that “scale is all you need” to achieve strong performance across tasks. However\, the exponential growth of these models demands substantial memory and computational resources\, limiting their usability in small research environments. In this talk\, I will discuss our ongoing efforts to make the training and adaptation of large foundation models more efficient and accessible.\nI will first introduce AdaRankGrad\, which exploits the low-rank structure of gradients to enable memory-efficient full-parameter fine-tuning. I will then present SUMO\, a subspace-aware optimizer that performs exact moment orthogonalization to accelerate convergence and enhance generalization while further reducing memory requirements. Finally\, I will discuss a complementary approach that compresses models during fine-tuning through stochastic gating\, yielding compact networks that maintain accuracy while reducing inference time.The talk is based on:\n[1] Yehonathan Refael\, Jonathan Svirsky\, Boris Shustin\, Wasim Huleihel\, and Ofir Lindenbaum. “AdaRankGrad: Adaptive Gradient Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning.” ICLR\, 2025.\n[2] Yehonathan Refael\, Guy Smorodinsky\, Tom Tirer\, and Ofir Lindenbaum. “SUMO: Subspace-Aware Moment-Orthogonalization for Accelerating Memory-Efficient LLM Training.” NeurIPS\, 2025.\n[3] Jonathan Svirsky\, Refael\, Yehonathan\, and Ofir Lindenbaum. “Train Less\, Infer Faster: Efficient Model Finetuning and Compression via Structured Sparsity.” AISTATS\, 2026. \nBio: Ofir Lindenbaum is a senior lecturer (assistant professor) in the Faculty of Engineering at Bar-Ilan University. He completed a postdoctoral fellowship at Yale University in the Applied Mathematics Program\, working with Prof. Ronald Coifman and Prof. Yuval Kluger\, and earned his Ph.D. in Electrical Engineering at Tel Aviv University under the supervision of Prof. Arie Yeredor and Prof. Amir Averbuch.\nHis research focuses on developing machine learning methods to advance scientific discovery. He works on interpretable and efficient models for high-dimensional tabular data\, multimodal learning\, and addresses questions in sparsification\, optimization\, and representation learning. His work aims to develop principled\, reliable\, and data-efficient algorithms to extract meaningful structure from real-world scientific measurements.
URL:https://datascience.ucsd.edu/event/hdsi-seminar-ofir-lindenbaum-efficient-training-fast-inference-reducing-memory-requirements-and-inference-time-of-foundation-models/
LOCATION:Halıcıoğlu Data Science Institute (HDSI)\, Room 123\, 3234 Matthews Ln\, La Jolla\, CA\, 92093\, United States
CATEGORIES:Seminar
END:VEVENT
END:VCALENDAR