Date & Time:
February 12, 2020 10:30 am – 11:30 am
Location:
Crerar 390, 5730 S. Ellis Ave., Chicago, IL,
02/12/2020 10:30 AM 02/12/2020 11:30 AM America/Chicago Simon Du (IAS/Princeton) – Foundations of Learning Systems with (Deep) Function Approximators Crerar 390, 5730 S. Ellis Ave., Chicago, IL,

Foundations of Learning Systems with (Deep) Function Approximators

Function approximators, such as deep neural networks, play a crucial role in building learning systems that make predictions and decisions. In this talk, I will discuss my work on understanding, designing, and applying function approximators.

First, I will focus on understanding deep neural networks. The main result is that the over-parameterized neural network is equivalent to a new kernel, Neural Tangent Kernel. This equivalence implies two surprising phenomena: 1) the simple algorithm gradient descent provably finds the global optimum of the highly non-convex empirical risk, and 2) the learned neural network generalizes well despite being highly over-parameterized.  Furthermore, this equivalence helps us design a new class of function approximators: we transform (fully-connected and graph) neural networks to (fully-connected and graph) Neural Tangent Kernels, which achieve superior performance on standard benchmarks. 

In the second part of the talk, I will focus on applying function approximators to decision-making, aka reinforcement learning, problems. In sharp contrast to the (simpler) supervised prediction problems, solving reinforcement learning problems requires an exponential number of samples, even if one applies function approximators.  I will then discuss what additional structures that permit statistically efficient algorithms.

Host: Michael Maire

Simon Du

Postdoctoral Researcher, Institute for Advanced Study, Princeton

Simon S. Du is a postdoc at the Institute for Advanced Study of Princeton, hosted by Sanjeev Arora. He completed his Ph.D. in Machine Learning at Carnegie Mellon University, where he was co-advised by Aarti Singh and Barnabás Póczos. Previously, he studied EECS and EMS at UC Berkeley. He has also spent time at Simons Institute and research labs of Facebook, Google, and Microsoft. His research interests are broadly in machine learning, with a focus on the foundations of deep learning and reinforcement learning.

Related News & Events

UChicago CS News

Super.tech/EPiQC Research Informs New Suite of Benchmarks for Quantum Computers

Feb 24, 2022
In the News

Quanta Magazine Features Prof. Bill Fefferman’s Work on Quantum Algorithms

Jan 20, 2022
UChicago CS News

UChicago CS Prof. Ben Zhao Named ACM Fellow

Jan 19, 2022
UChicago CS News

CS 4th Year Sophie Veys Receives CRA Undergraduate Research Award

Jan 14, 2022
UChicago CS News

In-Fridge Controller Could Scale Up Quantum Computers, Award-Winning UChicago Research Finds

Jan 10, 2022
UChicago CS News

Prof. Rebecca Willett Named IEEE Fellow

Nov 29, 2021
UChicago CS News

Aaron Elmore Promoted to Associate Professor at UChicago Computer Science

Nov 24, 2021
UChicago CS News

ScaleStuds Project Receives $5 Million to Build Foundations for Massive Computation

Nov 19, 2021
UChicago CS News

EPiQC Research Receives Best Paper Award at IEEE Quantum Week

Oct 22, 2021
UChicago CS News

New Wearable Device Controls Individual Fingers for Sign Language, Music Applications

Oct 11, 2021
UChicago CS News

UChicago and EPiQC Alum Yongshan Ding Joins Yale in Faculty Position

Oct 08, 2021
UChicago CS News

$3.25m DOE Grant Funds UChicago/Argonne Research on AI Models of Physics Simulations

Sep 27, 2021
arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube