Date & Time:
November 3, 2023 11:00 am – 12:00 pm
Location:
TTIC, 6045 S. Kenwood Ave., Chicago, IL,
Register
11/03/2023 11:00 AM 11/03/2023 12:00 PM America/Chicago Alan Ritter (Georgia Tech)- Towards Cost Efficient Use of Pre-trained Models TTIC, 6045 S. Kenwood Ave., Chicago, IL,

Large language models are leading to many exciting breakthroughs, but this comes at a significant cost in terms of both computational and data labeling expenses. Training state-of-the-art models requires access to high-end GPUs for pre-training and inference, in addition to labeled data for fine-tuning. In this talk I will examine the tradeoff between these costs, with the goal supporting better decisions. Conventional wisdom holds that annotating data is expensive, so computational methods that use unlabeled data to improve performance can present an economical alternative. I will examine this assumption in the context of pretraining-based adaptation, which requires significant computation for each new domain. As a second example where the tradeoff between computation and annotation arises, I will show that training and then distilling large models can be an economical strategy for improving performance. Finally, I will discuss applications on chemical synthesis protocols, and show a demo of a system that can help chemists to more efficiently find experimental conditions described in the literature. I will also present a new approach to extracting data from tables in scientific articles where the only supervision provided to the model is a database schema, eliminating the need for labeled data or custom data extraction pipelines.

Speakers

Card Image 8105225

Alan Ritter

Associate Professor

Alan Ritter is an associate professor in the College of Computing at Georgia Tech. His research on natural language processing aims to solve technical challenges that help machines read the web and engage in safe and helpful dialogue with people. In a recent project, covered by WIRED (https://www.wired.com/story/machine-learning-tweets-critical-security-flaws/), Alan’s group built a system that reads millions of online messages for mentions of new software vulnerabilities. He completed his Ph.D. at the University of Washington and was a postdoctoral fellow in the Machine Learning Department at Carnegie Mellon. Alan is the recipient of an NSF CAREER award and an Amazon Research Award.

Registration

Alan will have some time to meet with faculty and students virtually after the talk. Please sign up here if you are interested in meeting him:

Alan Ritter UChicago/TTIC Seminar Schedule

Please join us in TTIC 501 from 11-12 on Friday (you can also join on zoom).

 

Register
11/03/2023 11:00 AM 11/03/2023 12:00 PM America/Chicago Alan Ritter (Georgia Tech)- Towards Cost Efficient Use of Pre-trained Models TTIC, 6045 S. Kenwood Ave., Chicago, IL,

Related News & Events

Card Image eaa570d
Card Image
UChicago CS News

UChicago Undergrad Analyzes Machine Learning Models Used By CPD, Uncovers Lack of Transparency About Data Usage

Card Image ed7c202
Card Image
UChicago CS News

UChicago Team Wins The NIH Long COVID Computational Challenge

Card Image bb6361c
Card Image
UChicago CS News

UChicago Assistant Professor Raul Castro Fernandez Receives 2023 ACM SIGMOD Test-of-Time Award

Card Image 0001b7c
Card Image
UChicago CS News

Mike Franklin, Dan Nicolae Receive 2023 Arthur L. Kelly Faculty Prize

Card Image eec77e4
Card Image
UChicago CS News

PhD Student Kevin Bryson Receives NSF Graduate Research Fellowship to Create Equitable Algorithmic Data Tools

Card Image b03a065
Card Image
UChicago CS News

Computer Science Displays Catch Attention at MSI’s Annual Robot Block Party

Card Image 272e5c6
Card Image
UChicago CS News

UChicago / School of the Art Institute Class Uses Art to Highlight Data Privacy Dangers

Card Image 3989496
Card Image
UChicago CS News

UChicago Undergrad Team Places Second Overall In Regionals For World’s Largest Programming Competition

Card Image 19a688c
Card Image
UChicago CS News

Postdoc Alum John Paparrizos Named ICDE Rising Star

Card Image d64bd94
Card Image
UChicago CS News

UChicago and NYU Research Team Finds Edtech Tools Could Pose Privacy Risks For Students

Card Image f76e6b4
Card Image
UChicago CS News

Assistant Professor Chenhao Tan Receives Sloan Research Fellowship

Card Image 6bb1a0b
Card Image
UChicago CS News

Student Spotlight: Gabi Garcia’s Bridge Between CS and Classics

arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube