Contact Us

Give us a call or drop by anytime, we endeavor to answer all inquiries within 24 hours.


Find us

PO Box 16122 Collins Street West Victoria, Australia

Email us /

Phone support

Phone: + (066) 0760 0260 / + (057) 0760 0560

Loading Events

« All Events

  • This event has passed.
Event Series Event Series: Colloquia Lecture Series

Structured Transformer Models for NLP

March 22, 2023 @ 2:00 pm - 3:30 pm

Abstract: The field of natural language processing has recently unlocked a wide range of new capabilities through the use of large language models, such as GPT-4. The growing application of these models motivates developing a more thorough understanding of how and why they work, as well as further improvements in both quality and efficiency.

In this talk, I will present my work on analyzing and improving the Transformer architecture underlying today’s language models through the study of how information is routed between multiple words in an input. I will show that such models can predict the syntactic structure of text in a variety of languages, and discuss how syntax can inform our understanding of how the networks operate. I will also present my work on structuring information flow to build radically more efficient models, including models that can process text of up to one million words, which enables new possibilities for NLP with book-length text.

Bio: Nikita Kitaev is a final-year Ph.D. student in Computer Science at UC Berkeley, advised by Dan Klein. His research spans natural language processing and machine learning, with a focus on understanding the structure and operation of large language models, including leveraging insights from the field of syntax. He is the recipient of an NSF Graduate Research Fellowship and his work has been recognized through a best paper award at ACL. Previously, Nikita has received a B.S. in Electrical Engineering and Computer Science from UC Berkeley.


March 22, 2023
2:00 pm - 3:30 pm
Event Category:
Event Tags:
, ,


SDSC, The Synthesis Center
9500 Gilman Drive
La Jolla, CA 92093 United States
+ Google Map


Nikita Kitaev