Date & Time:
October 20, 2021 10:30 am – 11:30 am
Location:
TTIC, 6045 S. Kenwood Ave., Chicago, IL,
10/20/2021 10:30 AM 10/20/2021 11:30 AM America/Chicago Cong Ma (UChicago) – Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism TTIC TTIC, 6045 S. Kenwood Ave., Chicago, IL,

Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism

In this talk, I will focus on offline reinforcement learning (RL) problems where one aims to learn an optimal policy from a fixed dataset without active data collection. Depending on the composition of the offline dataset, two categories of methods are used: imitation learning which is suitable for expert datasets and vanilla offline RL which often requires uniform coverage datasets. However in practice, datasets often deviate from these two extremes and the exact data composition is usually unknown a priori.

To bridge this gap, I will present a new offline RL framework that smoothly interpolates between the two extremes of data composition, hence unifying imitation learning and vanilla offline RL. The new framework is centered around a weak version of the concentrability coefficient that measures the deviation from the behavior policy to the expert policy alone. Under this new framework, we show that a lower confidence bound algorithm based on pessimism is adaptively optimal for solving offline contextual bandit problems over the entire data composition range. Extensions to Markov decision processes will also be discussed.

Contact Denise Howard (denise.howard@ttic.edu) for Zoom and in-person attendance details.

Host: Toyota Technological Institute of Chicago

Cong Ma

Assistant Professor of Statistics, University of Chicago

Cong Ma is an assistant professor in the Department of Statistics at the University of Chicago. Prior to joining UChicago, he was a postdoctoral researcher at UC Berkeley, advised by Professor Martin Wainwright. He obtained his Ph.D. at Princeton University in May 2020, advised by Professor Yuxin Chen and Professor Jianqing Fan. He is broadly interested in the mathematics of data science, with a focus on the interplay between statistics and optimization.

Related News & Events

Students posing at competition
No Name

UChicago Undergrad Team Places Second Overall In Regionals For World’s Largest Programming Competition

Mar 17, 2023
Haifeng Xu
No Name

New CS and DSI Faculty Haifeng Xu Brings Strategic Intelligence to NeurIPS 2022

Nov 28, 2022
No Name

UChicago CS Research Finds New Angle on Database Query Processing with Geometry

Nov 08, 2022
No Name

Asst. Prof. Aloni Cohen Receives Award For Revealing Flaws in Deidentifying Data

Sep 09, 2022
No Name

UChicago Hosts NSF Workshop on Frontiers of Quantum Advantage

Aug 15, 2022
No Name

New 2022-23 Faculty Add Expertise in Linguistics, Visualization, Economics, and Data Science Education

Aug 11, 2022
No Name

UChicago Co-Leads $10 Million NSF Institute on Foundations of Data Science

Aug 09, 2022
No Name

Bill Fefferman Comments on New Standards for Quantum-Proof Cryptography

Jul 07, 2022
No Name

UChicago London Colloquium Features Data Science, Quantum Research

Jul 01, 2022
No Name

Faculty Bill Fefferman and Chenhao Tan Receive Google Research Scholar Awards

Jun 21, 2022
No Name

First-Year PhD Student Co-Authors Outstanding Paper Award Winner at TQC 2022

Apr 28, 2022
No Name

Quanta Magazine Features Prof. Bill Fefferman’s Work on Quantum Algorithms

Jan 20, 2022
arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube