Date & Time:
October 18, 2021 11:00 am – 12:00 pm
10/18/2021 11:00 AM 10/18/2021 12:00 PM America/Chicago Kristina Toutanova (Google) – Advances and Limitations in Generalization via Self-Supervised Pretrained Representations TTIC Zoom

Advances and Limitations in Generalization via Self-Supervised Pretrained Representations

Pretrained neural representations, learned from unlabeled text, have recently led to substantial improvements across many natural language problems. Yet some components of these models are still brittle and heuristic, and sizable human labeled data is typically needed to obtain competitive performance on end tasks.  

I will first talk about recent advances from our team leading to (i) improved multi-lingual generalization and ease of use through tokenization-free pretrained representations and (ii) better few-shot generalization for underrepresented task categories via neural language model-based example extrapolation. I will then point to limitations of generic pre-trained representations when tasked with handling both language variation and out-of-distribution compositional generalization, and the relative performance of induced symbolic representations.

This talk is part of the TTIC Colloquium and will be presented on Zoom, register here for details.

Host: Toyota Technology Institute at Chicago

Kristina Toutanova

Research Scientist, Google Research

Kristina Toutanova is a research scientist at Google Research in Seattle and an affiliate faculty at the University of Washington. She obtained her Ph.D. from the Computer Science Department at Stanford University with Christopher Manning, and her MSc in Computer Science from Sofia University, Bulgaria. Prior to joining Google in 2017, she was a researcher at Microsoft Research, Redmond. Kristina focuses on modeling the structure of natural language using machine learning, most recently in the areas of representation learning, question answering, information retrieval and semantic parsing. Kristina is a past co-editor in chief of TACL, a program co-chair for ACL 2014, and a general chair for NAACL 2021.

Related News & Events

No Name

Prof. Rebecca Willett awarded the SIAG DATA Career prize

Feb 21, 2024
No Name

Argonne scientists use AI to identify new materials for carbon capture

Feb 19, 2024
No Name

Alumni Spotlight: Dixin Tang, Assistant Professor of Computer Science at UT Austin

Feb 05, 2024
No Name

NetMicroscope Uses AI to Improve Network Monitoring for a Better Internet Experience

Feb 01, 2024
No Name

NeurIPS 2023 Award-winning paper by DSI Faculty Bo Li, DecodingTrust, provides a comprehensive framework for assessing trustworthiness of GPT models

Feb 01, 2024
No Name

New research unites quantum engineering and artificial intelligence

Jan 29, 2024

“Machine Learning Foundations Accelerate Innovation and Promote Trustworthiness” by Rebecca Willett

Jan 26, 2024

Nightshade: Data Poisoning to Fight Generative AI with Ben Zhao

Jan 23, 2024
No Name

Exploring 3D Paintbrush: An AI That Colors with Words

Jan 22, 2024
No Name

Group From UChicago CS To Present Four Papers at Most Prestigious International Quantum Conference

Jan 09, 2024
No Name

Alumni Spotlight: Get To Know Emily Wenger, a 2023 CS Graduate Who Was Just Named To The Forbes 30 Under 30 List

Nov 29, 2023
No Name

Three UChicago PhD Students From The Department of Computer Science Named To Forbes 30 Under 30 List

Nov 28, 2023
arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube