Events

Events Search and Views Navigation

Event Views Navigation

Today

Filters

Changing any of the form inputs will cause the list of events to refresh with the filtered results.

  • Uncovering the algorithmic foundations of language learning and processing

    Special Seminar Series
    SDSC, The Synthesis Center 9500 Gilman Drive, La Jolla, CA, United States

    Human language is extraordinarily complex. Nevertheless, we readily acquire language as children, when we are most cognitively limited, and we comprehend language as adults with striking efficiency. My research seeks to understand the mental algorithms that allow us to accomplish this feat, with particular focus on how memory and prediction mechanisms are recruited to overcome the bottlenecks of real-time language processing. In this talk, I will review results from three of my lines of inquiry into this question. First, using diverse naturalistic reading datasets, I will show evidence that prediction is a central concern of the human language processing system. Second, using fMRI measures of naturalistic story listening, I will show evidence that memory and prediction processes are dissociable in the brain's response to language, that syntactic structure building plays a major role in ordinary language comprehension, and that the neural resources that are responsible for structure building are largely specialized for language. Third, I will show evidence from computational modeling that memory and prediction pressures independently encourage discovery of phonological regularities from natural speech. Together, these results support an intricate coordination of memory and prediction abilities for language learning and comprehension. I will conclude by outlining planned directions for my future lab, integrating neuroimaging, behavioral methods, natural language processing, and computational modeling to study language learning and processing.

  • Structured Transformer Models for NLP

    Colloquia Lecture Series
    SDSC, The Synthesis Center 9500 Gilman Drive, La Jolla, CA, United States

    The field of natural language processing has recently unlocked a wide range of new capabilities through the use of large language models, such as GPT-4. The growing application of these models motivates developing a more thorough understanding of how and why they work, as well as further improvements in both quality and efficiency.

    In this talk, I will present my work on analyzing and improving the Transformer architecture underlying today's language models through the study of how information is routed between multiple words in an input. I will show that such models can predict the syntactic structure of text in a variety of languages, and discuss how syntax can inform our understanding of how the networks operate. I will also present my work on structuring information flow to build radically more efficient models, including models that can process text of up to one million words, which enables new possibilities for NLP with book-length text.

  • PhD Open House

    SDSC, The Auditorium 9836 Hopkins Dr, La Jolla, San Diego, CA, United States

    HDSI will host the first ever in person DSC Inaugural Open House for Prospective PhD Students (Mar 23 + 24).  The event will include Graduate Student Poster Session and will showcase HDSI diverse research […]

  • Distance-Estimation in Modern Graphs: Algorithms and Impossibility | Nicole Wein

    Special Seminar Series
    SDSC, The Auditorium 9836 Hopkins Dr, La Jolla, San Diego, CA, United States

    The size and complexity of today's graphs present challenges that necessitate the discovery of new algorithms. One central area of research in this endeavor is computing and estimating distances in graphs. In this talk I will discuss two fundamental families of distance problems in the context of modern graphs: Diameter/Radius/Eccentricities and Hopsets/Shortcut Sets. The best-known algorithm for computing the diameter (largest distance) of a graph is the naive algorithm of computing all-pairs shortest paths and returning the largest distance. Unfortunately, this can be prohibitively slow for massive graphs. Thus, it is important to understand how fast and how accurately the diameter of a graph can be approximated. I will present tight bounds for this problem via conditional lower bounds from fine-grained complexity. Secondly, for a number of settings relevant to modern graphs (e.g. parallel algorithms, streaming algorithms, dynamic algorithms), distance computation is more efficient when the input graph has low hop-diameter. Thus, a useful preprocessing step is to add a set of edges (a hopset) to the graph that reduces the hop-diameter of the graph, while preserving important distance information. I will present progress on upper and lower bounds for hopsets.

  • Frameworks for High Dimensional Optimization

    Special Seminar Series
    SDSC, The Auditorium 9836 Hopkins Dr, La Jolla, San Diego, CA, United States

    I present frameworks for solving extremely large, prohibitively massive optimization problems. Today, practical applications require optimization solvers to work at extreme scales, but existing solvers do not often scale as desired. I present black-box acceleration algorithms for speeding up optimization solvers, in both distributed and parallel settings. Given a huge problem, I develop dimension reduction techniques that allow the problem to be solved in a fraction of the original time, and simultaneously make the computation amenable to distributed computation. Efficient, dependable and secure distributed computing is increasingly fundamental to a wide range of core applications including distributed data centers, decentralized power grid, coordination of autonomous devices, and scheduling and routing problems.

  • Constrained, Casual, and Logical Reasoning for Neural Language Generation

    Special Seminar Series
    SDSC, The Auditorium 9836 Hopkins Dr, La Jolla, San Diego, CA, United States

    Today’s language models (LMs) can produce human-like fluent text. However, they generate words with no grounding in the world and cannot flexibly reason about everyday situations and events, such as counterfactual (“what if?”) and abductive (“what might explain these observations?”) reasoning that are important forms of human cognition activities. In this talk, I will present my research on connecting reasoning with language generation. Reasoning for language generation poses several key challenges, including incorporating diverse contextual constraints on the fly, understanding cause and effect when events unfold, and grounding on logic structures for consistent reasoning. I will first discuss COLD decoding, a unified energy-based framework for any off-the-shelf LMs to reason with arbitrary constraints. It also introduces differentiable reasoning over discrete symbolic text for improved efficiency. Secondly, I will focus on a particularly important form of reasoning, counterfactual reasoning, including its first formulation in language generation and our algorithm, DeLorean, that enables off-the-shelf LMs to capture causal invariance. Thirdly, I will present Maieutic prompting, which improves the logical consistency of neural reasoning by integrating with logic structures. I will conclude with future research toward more general, grounded, and trustworthy reasoning with language.

  • Acceleration in Optimization, Sampling, and Machine Learning

    Special Seminar Series
    SDSC, The Auditorium 9836 Hopkins Dr, La Jolla, San Diego, CA, United States

    Optimization, sampling, and machine learning are essential components of data science. In this talk, I will cover my work on accelerated methods in these fields and highlight some connections between them.

    In optimization, I will present optimization as a two-player zero-sum game, which is a modular approach for designing and analyzing convex optimization algorithms by pitting a pair of no-regret learning strategies against each other. This approach not only recovers several existing algorithms but also gives rise to new ones. I will also discuss the use of Heavy Ball in non-convex optimization, which is a popular momentum method in deep learning. Despite its success in practice, Heavy Ball currently lacks theoretical evidence for its acceleration in non-convex optimization. To bridge this gap, I will present some non-convex problems where Heavy Ball exhibits provable acceleration guarantees.

    In sampling, I will describe how to accelerate a classical sampling method called Hamiltonian Monte Carlo by setting its integration time appropriately, which builds on a connection between sampling and optimization. In machine learning, I will talk about Gradient Descent with pseudo-labels for fast test-time adaptation under the context of tackling distribution shifts.

  • Intelligent mobile systems for equitable healthcare

    Special Seminar Series
    SDSC, The Auditorium 9836 Hopkins Dr, La Jolla, San Diego, CA, United States

    Access to even basic medical resources is greatly influenced by factors like an individual's birth country and zip code. In this talk, I will present my work on designing AI-based mobile systems for equitable healthcare. I will showcase three systems that are not only interesting from an AI standpoint but are also having real-world medical impact. The first system can detect ear infections using only a smartphone and a paper cone. The second system enables low-cost newborn hearing screening using inexpensive earphones. Lastly, I will present an ambient sensing system that employs smart devices to detect emergent and life-threatening medical events such as cardiac arrest. Through these examples, I will demonstrate how new applied machine learning and sensing approaches that generalize across hardware and work in real-world environments can help to address pressing societal problems.

  • Language Models and Human Language Acquisition

    Special Seminar Series
    SDSC, The Auditorium 9836 Hopkins Dr, La Jolla, San Diego, CA, United States

    Abstract: Children have a remarkable ability to acquire language. This propensity has been an object of fascination in science for millennia, but in just the last few years, neural language models (LMs) have also proven to be incredibly adept at learning human language. In this talk, I discuss scientific progress that uses recent developments in natural language processing to advance linguistics—and vice-versa. My research explores this intersection from three angles: evaluation, experimentation, and engineering. Using linguistically motivated benchmarks, I provide evidence that LMs share many aspects of human grammatical knowledge and probe how this knowledge varies across training regimes. I further argue that—under the right circumstances—we can use LMs to test hypotheses that have been difficult or impossible to evaluate with human subjects. Such experiments have the potential to transform debates about the roles of nature and nurture in human language learning. As a proof of concept, I describe a controlled experiment examining how the distribution of linguistic phenomena in the input affects syntactic generalization. While the results suggest that the linguistic stimulus may be richer than often thought, there is no avoiding the fact that current LMs and humans learn language in vastly different ways. I describe ongoing work to engineer learning environments and objectives for LM pretraining inspired by human development, with the goal of making LMs more data efficient and more plausible models of human learning.