Contact Us
Give us a call or drop by anytime, we endeavor to answer all inquiries within 24 hours.
Find us
PO Box 16122 Collins Street West Victoria, Australia
Email us
info@domain.com / example@domain.com
Phone support
Phone: + (066) 0760 0260 / + (057) 0760 0560
- This event has passed.
Event Series: Special Seminar Series
Language Models and Human Language Acquisition
April 10, 2023 @ 12:00 pm - 1:00 pm
Abstract: Children have a remarkable ability to acquire language. This propensity has been an object of fascination in science for millennia, but in just the last few years, neural language models (LMs) have also proven to be incredibly adept at learning human language. In this talk, I discuss scientific progress that uses recent developments in natural language processing to advance linguistics—and vice-versa. My research explores this intersection from three angles: evaluation, experimentation, and engineering. Using linguistically motivated benchmarks, I provide evidence that LMs share many aspects of human grammatical knowledge and probe how this knowledge varies across training regimes. I further argue that—under the right circumstances—we can use LMs to test hypotheses that have been difficult or impossible to evaluate with human subjects. Such experiments have the potential to transform debates about the roles of nature and nurture in human language learning. As a proof of concept, I describe a controlled experiment examining how the distribution of linguistic phenomena in the input affects syntactic generalization. While the results suggest that the linguistic stimulus may be richer than often thought, there is no avoiding the fact that current LMs and humans learn language in vastly different ways. I describe ongoing work to engineer learning environments and objectives for LM pretraining inspired by human development, with the goal of making LMs more data efficient and more plausible models of human learning.
Bio: Alex Warstadt is a postdoc in the computer science department at ETH Zürich working with Ryan Cotterell. In 2022, he completed a PhD in linguistics at New York University supervised by Sam Bowman. Alex works on a variety of topics at the intersection of natural language processing and linguistics, including language model pretraining, evaluation and interpretability, language acquisition, and pragmatics.