Artificial intelligence has gotten better and better at understanding the visual world, classifying two-dimensional images and video without human intervention. But the application of neural networks, deep learning and other AI approaches to create new graphics and work with three-dimensional data is still in its earliest stages. The production of 3D graphics remains accessible only to experts, and the application of AI to analyze 3D data and produce new images is still limited in its scope and possibilities.

Rana Hanocka, a new assistant professor who joined UChicago CS in the summer of 2021, aims to democratize this technology, building AI capabilities for 3D data that transform aspiration into reality. Her research creates new frameworks and applications that allow users to more easily and expansively work with 3D geometric data: transforming two-dimensional images into 3D structures, changing features with descriptive text, and generating realistic synthetic data for the training of computer vision models.

Her research group 3DL — an acronym for “3D Deep Learning,” pronounced “threedle” — produces new tools that can help artists, designers, architects, and hobbyists create new 3D objects with limited technical expertise. For example, a user could take a 3D model of a table, chair or any other object and manipulate its appearance and texture through text inputs: “crochet,” “wicker,” “brick,” or more fantastical choices such as “cactus,” or “astronaut.”

“We’re interested in how to make things that work automatically, while being more intuitive and easier to understand and use,” Hanocka said. “We design tools for manipulating 3d objects, creating theatrics, and animating them. And we want to make our research available, enabling people to use these tools so that people can build off of and extend them.”

Hanocka joined the University of Chicago after completing her Ph.D. at Tel Aviv University in Israel, where she studied with Daniel Cohen-Or and Raja Giryes. She gravitated to studying computer graphics as a means of combining her interests in mathematics and visual art, seeing the potential of new AI approaches in this area.

“I think we have seen an AI explosion in structured data such as text and images, but we haven’t really seen that same success yet in unstructured 3D data and shape analysis,” Hanocka said. “So my hope is to be able to achieve that same success in this domain where we don’t yet see the same kind of amazing results.”

Point2Mesh is a technique for reconstructing a surface mesh from an input point cloud. This approach “learns” from a single object, by optimizing the weights of a CNN to deform some initial mesh to shrink-wrap the input point cloud.

One obstacle is the relative shortage of 3D data compared to the deep well of 2D images that can be harvested from the Internet. Training neural networks requires lots of data, and several different research groups have developed methods to convert 2D imagery into 3D models. Hanocka’s research has focused on transforming point clouds into polygonal meshes – an efficient representation of 3D shapes.

By then designing neural networks that work with these mesh objects, Hanocka enables automated analyses and manipulations of 3D objects that previously required difficult and time-consuming software. These edits range from simple changes in characteristics such as color and lighting to 3D-specific alterations of texture and geometry. The networks also enable users to generate new data from limited examples, such as a particular motion sequence, opening up new graphics capabilities for virtual reality, animation, and other media. These new datasets could also help alleviate the shortage of 3D training data for computer vision models necessary for autonomous vehicles and other technologies.

In addition to her research, Hanocka brings new courses in computer graphics to UChicago CS. In Fall 2021, she taught a topics course on “Machine Learning: 3D Geometry Processing and Computer Vision with Deep Neural Networks.” This spring, she will revive and reinvent Introduction to Computer Graphics, a subject of high demand among UChicago students.

And it’s the opportunity to work with those students that attracted Hanocka to the University of Chicago, in addition to the rapidly-growing faculty, the university’s ties to national laboratories Argonne and Fermilab, and the experience of living in Chicago.

“The undergraduate students at the University are really top of the line,” Hanocka said. “I think that they can bring a different perspective and it’s a big strength that can be leveraged in research. When you have such talent, it’s a real asset.”

Related News

More UChicago CS stories from this research area.
Video

“Machine Learning Foundations Accelerate Innovation and Promote Trustworthiness” by Rebecca Willett

Jan 26, 2024
Video

Nightshade: Data Poisoning to Fight Generative AI with Ben Zhao

Jan 23, 2024
No Name

In The News: U.N. Officials Urge Regulation of Artificial Intelligence

"Security Council members said they feared that a new technology might prove a major threat to world peace."
Jul 27, 2023
No Name

UChicago Computer Scientists Bring in Generative Neural Networks to Stop Real-Time Video From Lagging

Jun 29, 2023
No Name

Computer Science Displays Catch Attention at MSI’s Annual Robot Block Party

Apr 07, 2023
No Name

UChicago, Stanford Researchers Explore How Robots and Computers Can Help Strangers Have Meaningful In-Person Conversations

Mar 29, 2023
No Name

Postdoc Alum John Paparrizos Named ICDE Rising Star

Mar 15, 2023
No Name

New EAGER Grant to Asst. Prof. Eric Jonas Will Explore ML for Quantum Spectrometry

Mar 03, 2023
No Name

Asst. Prof. Rana Hanocka Receives NSF Grant to Develop New AI-Driven 3D Modeling Tools

Feb 28, 2023
No Name

Assistant Professor Chenhao Tan Receives Sloan Research Fellowship

Feb 15, 2023
No Name

UChicago Scientists Develop New Tool to Protect Artists from AI Mimicry

Feb 13, 2023
No Name

Professors Rebecca Willett and Ben Zhao Discuss the Future of AI on Public Radio

Jan 26, 2023
arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube