Date & Time:
December 6, 2019 10:30 am – 11:30 am
Location:
TTIC 526, 6045 S. Kenwood Ave., Chicago, IL,
12/06/2019 10:30 AM 12/06/2019 11:30 AM America/Chicago Zhaoran Wang (Northwestern) – Computational and Statistical Efficiency of Policy Optimization UChicago CS/Toyota Technological Institute of Chicago Machine Learning Seminar Series TTIC 526, 6045 S. Kenwood Ave., Chicago, IL,

On the Computational and Statistical Efficiency of Policy Optimization in (Deep) Reinforcements Learning

Coupled with powerful function approximators such as neural networks, policy optimization plays a key role in the tremendous empirical successes of deep reinforcement learning. In sharp contrast, the theoretical understandings of policy optimization remain rather limited from both the computational and statistical perspectives. From the perspective of computational efficiency, it remains unclear whether policy optimization converges to the globally optimal policy in a finite number of iterations, even given infinite data. From the perspective of statistical efficiency, it remains unclear how to attain the globally optimal policy with a finite regret or sample complexity.

To address the computational question, I will show that, under suitable conditions, natural policy gradient/proximal policy optimization/trust-region policy optimization (NPG/PPO/TRPO) converges to the globally optimal policy at a sublinear rate, even when it is coupled with neural networks. To address the statistical question, I will present an optimistic variant of NPG/PPO/TRPO, namely OPPO, which incorporates exploration in a principled manner and attains a sqrt{T}-regret. 

(Joint work with Qi Cai, Chi Jin, Jason Lee, Boyi Liu, Zhuoran Yang) 

Host: Mladen Kolar

Zhaoran Wang

Assistant Professor of Industrial Engineering and Management Sciences, Northwestern University

Zhaoran Wang is an assistant professor at Northwestern University, working at the interface of machine learning, statistics, and optimization. He is the recipient of the AISTATS (Artificial Intelligence and Statistics Conference) notable paper award, ASA (American Statistical Association) best student paper in statistical learning and data mining, INFORMS (Institute for Operations Research and the Management Sciences) best student paper finalist in data mining, and the Microsoft fellowship.

Related News & Events

UChicago CS News

UChicago CS Researchers Shine at UIST 2024 with Papers, Posters, Workshops and Demonstrations

Oct 10, 2024
UChicago CS News

UChicago Scientists Receive Grant to Expand Global Data Management Platform, Globus

Oct 03, 2024
UChicago CS News

UChicago Researchers Demonstrate the Quantifiable Uniqueness of Former President Donald Trump’s Language Use

Sep 30, 2024
UChicago CS News

Five UChicago CS students named to Siebel Scholars class of 2025

Sep 20, 2024
UChicago CS News

NSF and Simons Foundation launch $20 million National AI Research Institute in Astronomy

Sep 18, 2024
In the News

Data Ecology: A Socio-Technical Approach to Controlling Dataflows

Sep 18, 2024
UChicago CS News

Ph.D. Student Shawn Shan Named MIT Technology Review’s 35 Innovators Under 35 and Innovator of the Year

Sep 16, 2024
UChicago CS News

Ben Zhao Named to TIME Magazine’s TIME100 AI List

Sep 05, 2024
UChicago CS News

Ian Foster and Rick Stevens Named to HPCwire’s 35 Legends List

Aug 28, 2024
UChicago CS News

University of Chicago to Develop Software for Effort to Create a National Quantum Virtual Laboratory

Aug 28, 2024
UChicago CS News

New Classical Algorithm Enhances Understanding of Quantum Computing’s Future

Aug 27, 2024
UChicago CS News

Decoding Content Moderation: Analyzing Policy Variations Across Top Online Platforms

Aug 26, 2024
arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube