Date & Time:
October 18, 2021 11:00 am – 12:00 pm
10/18/2021 11:00 AM 10/18/2021 12:00 PM America/Chicago Kristina Toutanova (Google) – Advances and Limitations in Generalization via Self-Supervised Pretrained Representations TTIC Zoom

Advances and Limitations in Generalization via Self-Supervised Pretrained Representations

Pretrained neural representations, learned from unlabeled text, have recently led to substantial improvements across many natural language problems. Yet some components of these models are still brittle and heuristic, and sizable human labeled data is typically needed to obtain competitive performance on end tasks.  

I will first talk about recent advances from our team leading to (i) improved multi-lingual generalization and ease of use through tokenization-free pretrained representations and (ii) better few-shot generalization for underrepresented task categories via neural language model-based example extrapolation. I will then point to limitations of generic pre-trained representations when tasked with handling both language variation and out-of-distribution compositional generalization, and the relative performance of induced symbolic representations.

This talk is part of the TTIC Colloquium and will be presented on Zoom, register here for details.

Host: Toyota Technology Institute at Chicago

Kristina Toutanova

Research Scientist, Google Research

Kristina Toutanova is a research scientist at Google Research in Seattle and an affiliate faculty at the University of Washington. She obtained her Ph.D. from the Computer Science Department at Stanford University with Christopher Manning, and her MSc in Computer Science from Sofia University, Bulgaria. Prior to joining Google in 2017, she was a researcher at Microsoft Research, Redmond. Kristina focuses on modeling the structure of natural language using machine learning, most recently in the areas of representation learning, question answering, information retrieval and semantic parsing. Kristina is a past co-editor in chief of TACL, a program co-chair for ACL 2014, and a general chair for NAACL 2021.

Related News & Events

UChicago CS News

Non-Unital Noise Adds a New Wrinkle to the Quantum Supremacy Debate

Apr 05, 2024
In the News

The Science of Computer Security: An Interview with Grant Ho, Assistant Professor in Computer Science

Apr 02, 2024
UChicago CS News

Four Students Receive Honorable Mention in CRA Undergraduate Research Awards

Mar 25, 2024
UChicago CS News

Navigating the Intersection of Technology and Public Policy: The Journey of Ranya Sharma at UChicago

Mar 21, 2024
UChicago CS News

Assistant Professor Aloni Cohen Receives Prestigious Award for Groundbreaking Research in Machine Learning Complexity

Mar 20, 2024
UChicago CS News

Haifeng Xu named a AI2050 Early Career Fellow

Mar 11, 2024
UChicago CS News

FabRobotics: The Fusion of 3D Printing and Mobile Robots

Feb 27, 2024
UChicago CS News

Professor Andrew A. Chien on the Environmental Impacts of Technology

Feb 26, 2024
UChicago CS News

Assistant Professor Yanjing Li Awarded NSF CAREER Grant for Innovative Computer Architecture and Deep Learning Research

Feb 26, 2024
UChicago CS News

Prof. Rebecca Willett awarded the SIAG DATA Career prize

Feb 21, 2024
UChicago CS News

Argonne scientists use AI to identify new materials for carbon capture

Feb 19, 2024
UChicago CS News

Alumni Spotlight: Dixin Tang, Assistant Professor of Computer Science at UT Austin

Feb 05, 2024
arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube