Date & Time:
February 1, 2021 3:00 pm – 4:00 pm
Live Stream
02/01/2021 03:00 PM 02/01/2021 04:00 PM America/Chicago Mukund Sundararajan (Google) – Using Attribution to Understand Deep Neural Networks CDAC Center for Data and Computing Distinguished Speaker Series Live Stream

Using Attribution to Understand Deep Neural Networks

Zoom (RSVP for login) or YouTube (no registration required)

There was a neural model for predicting cancer from XRays. It had good accuracy on held out training data. But when we attributed its predictions back to the pixels of the XRays, we found that the network relied on barely visible pen marks that the doctors had made on the training data, and not the pathology of cancer. Naturally, the model was not deployed!

I work on techniques to perform prediction attribution of this kind. The target of the attribution can be input features (pixels in the example above), or interactions between its input features, or neurons. or training data examples. Attributions are reductive; i.e, they abstract away most of the interactions and a lot of the non-linearity of neural networks. However, attributions, done systematically, are effective at uncovering bugs as in the anecdote above.

We will briefly discuss the theory (e.g connections to the Taylor series, Shapley values, and Stochastic Gradient Descent) and philosophy of attribution, and other amusing examples of bugs.

If you are a deep learning practitioner, you can easily apply attribution to your own models; all the techniques can be implemented with less than ten lines of code.

Mukund Sundararajan

Principal Research Scientist/Director, Google

I am a principal research scientist/director at Google. These days, I analyze complex machine learning models. I have also worked on question-answering systems, ad auctions, security protocol analysis, privacy, and computational biology.

There once was a RS called MS,
He studies models that are a mess,
A director at Google.
Accurate and frugal,
Explanations are what he likes best.

Related News & Events


“Machine Learning Foundations Accelerate Innovation and Promote Trustworthiness” by Rebecca Willett

Jan 26, 2024

Nightshade: Data Poisoning to Fight Generative AI with Ben Zhao

Jan 23, 2024
No Name

In The News: U.N. Officials Urge Regulation of Artificial Intelligence

"Security Council members said they feared that a new technology might prove a major threat to world peace."
Jul 27, 2023
No Name

UChicago Computer Scientists Bring in Generative Neural Networks to Stop Real-Time Video From Lagging

Jun 29, 2023
No Name

Computer Science Displays Catch Attention at MSI’s Annual Robot Block Party

Apr 07, 2023
No Name

UChicago, Stanford Researchers Explore How Robots and Computers Can Help Strangers Have Meaningful In-Person Conversations

Mar 29, 2023
Students posing at competition
No Name

UChicago Undergrad Team Places Second Overall In Regionals For World’s Largest Programming Competition

Mar 17, 2023
No Name

Postdoc Alum John Paparrizos Named ICDE Rising Star

Mar 15, 2023
No Name

New EAGER Grant to Asst. Prof. Eric Jonas Will Explore ML for Quantum Spectrometry

Mar 03, 2023
No Name

Assistant Professor Chenhao Tan Receives Sloan Research Fellowship

Feb 15, 2023
No Name

UChicago Scientists Develop New Tool to Protect Artists from AI Mimicry

Feb 13, 2023
No Name

Professors Rebecca Willett and Ben Zhao Discuss the Future of AI on Public Radio

Jan 26, 2023
arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube