Contact Us

Give us a call or drop by anytime, we endeavor to answer all inquiries within 24 hours.

map

Find us

PO Box 16122 Collins Street West Victoria, Australia

Email us

info@domain.com / example@domain.com

Phone support

Phone: + (066) 0760 0260 / + (057) 0760 0560

Neural Language Models

  • April 6, 2023
  • Kaleigh O'Merry

Language Models and Human Language Acquisition

Abstract: Children have a remarkable ability to acquire language. This propensity has been an object of fascination in science for millennia, but in just the last few years, neural language models (LMs) have also proven to be incredibly adept at learning human language. In this talk, I discuss scientific progress that uses recent developments in natural language processing to advance linguistics—and vice-versa. My research explores this intersection from three angles: evaluation, experimentation, and engineering. Using linguistically motivated benchmarks, I provide evidence that LMs share many aspects of human grammatical knowledge and probe how this knowledge varies across training regimes. I further argue that—under the right circumstances—we can use LMs to test hypotheses that have been difficult or impossible to evaluate with human subjects. Such experiments have the potential to transform debates about the roles of nature and nurture in human language learning. As a proof of concept, I describe a controlled experiment examining how the distribution of linguistic phenomena in the input affects syntactic generalization. While the results suggest that the linguistic stimulus may be richer than often thought, there is no avoiding the fact that current LMs and humans learn language in vastly different ways. I describe ongoing work to engineer learning environments and objectives for LM pretraining inspired by human development, with the goal of making LMs more data efficient and more plausible models of human learning.

Read More