Date & Time:
May 2, 2022 3:00 pm – 4:00 pm
Location:
Crerar 390, 5730 S. Ellis Ave., Chicago, IL,
05/02/2022 03:00 PM 05/02/2022 04:00 PM America/Chicago Allyson Ettinger (UChicago) – “Understanding” and prediction: Controlled examinations of meaning sensitivity in pre-trained models Crerar 390, 5730 S. Ellis Ave., Chicago, IL,

In recent years, NLP has made what appears to be incredible progress, with performance even surpassing human performance on some benchmarks. How should we interpret these advances? Have these models achieved language “understanding”? Operating on the premise that “understanding” will necessarily involve the capacity to extract and deploy the information conveyed in language inputs — the “meaning” — in this talk I will discuss a series of projects leveraging targeted tests to examine NLP models’ ability to capture input meaning in a systematic fashion. I will first discuss work probing model representations for compositional phrase and sentence meaning, with a particular focus on disentangling compositional information from encoding of word-level properties. I’ll then explore models’ ability to extract and use meaning information when executing the basic pre-training task of word prediction in context. In all cases, these investigations apply tests that prioritize control of unwanted cues, so as to target the desired model capabilities with greater precision. The results of these studies suggest that although models show a good deal of sensitivity to word-level information, and to certain semantic and syntactic distinctions, they show little sign of representing higher-level compositional meaning, or of being able to retain and deploy such information robustly during word prediction. I will discuss potential implications of these findings with respect to the goals of achieving “understanding” with currently dominant pre-training paradigms.

Speakers

Allyson Ettinger

Assistant Professor, Department of Linguistics, University of Chicago

Dr. Allyson Ettinger’s research is focused on language processing in humans and in artificial intelligence systems, motivated by a combination of scientific and engineering goals. For studying humans, her research uses computational methods to model and test hypotheses about mechanisms underlying the brain’s processing of language in real time. In the engineering domain, her research uses insights and methods from cognitive science, linguistics, and neuroscience in order to analyze, evaluate, and improve natural language understanding capacities in artificial intelligence systems. In both of these threads of research, the primary focus is on the processing and representation of linguistic meaning.

Related News & Events

UChicago CS News

UChicago London Colloquium Features Data Science, Quantum Research

Jul 01, 2022
Video

Is it Ethical to Use Facial Imaging in Decision-Making?

Jun 28, 2022
UChicago CS News

Faculty Bill Fefferman and Chenhao Tan Receive Google Research Scholar Awards

Jun 21, 2022
UChicago CS News

Two Incoming UChicago CS PhD Students Receive Department of Energy Fellowship

Jun 16, 2022
Video

Data Science Institute Summit

Jun 15, 2022
UChicago CS News

DSI Summer Lab Returns In-Person With 49 Students From Across the U.S.

Jun 14, 2022
In the News

Nick Feamster Talks Internet Equity on Light Reading Podcast

Jun 09, 2022
UChicago CS News

UChicago CS Labs Join Museum of Science & Industry For Robot Block Party

Apr 20, 2022
UChicago CS News

Three UChicago CS Security Projects Receive Funding From c3.ai Institute

Mar 25, 2022
UChicago CS News

Junchen Jiang Wins CAREER Award to Study Using ML to Optimize Video Experiences

Mar 24, 2022
UChicago CS News

Professor Andrew Chien Authors New Textbook, “Computer Architecture for Scientists”

Mar 16, 2022
UChicago CS News

New Data & Democracy Research Initiative Launched at University of Chicago

Mar 09, 2022
arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube