Artificial intelligence has gotten better and better at understanding the visual world, classifying two-dimensional images and video without human intervention. But the application of neural networks, deep learning and other AI approaches to create new graphics and work with three-dimensional data is still in its earliest stages. The production of 3D graphics remains accessible only to experts, and the application of AI to analyze 3D data and produce new images is still limited in its scope and possibilities.

Rana Hanocka, a new assistant professor who joined UChicago CS in the summer of 2021, aims to democratize this technology, building AI capabilities for 3D data that transform aspiration into reality. Her research creates new frameworks and applications that allow users to more easily and expansively work with 3D geometric data: transforming two-dimensional images into 3D structures, changing features with descriptive text, and generating realistic synthetic data for the training of computer vision models.

Her research group 3DL — an acronym for “3D Deep Learning,” pronounced “threedle” — produces new tools that can help artists, designers, architects, and hobbyists create new 3D objects with limited technical expertise. For example, a user could take a 3D model of a table, chair or any other object and manipulate its appearance and texture through text inputs: “crochet,” “wicker,” “brick,” or more fantastical choices such as “cactus,” or “astronaut.”

“We’re interested in how to make things that work automatically, while being more intuitive and easier to understand and use,” Hanocka said. “We design tools for manipulating 3d objects, creating theatrics, and animating them. And we want to make our research available, enabling people to use these tools so that people can build off of and extend them.”

Hanocka joined the University of Chicago after completing her Ph.D. at Tel Aviv University in Israel, where she studied with Daniel Cohen-Or and Raja Giryes. She gravitated to studying computer graphics as a means of combining her interests in mathematics and visual art, seeing the potential of new AI approaches in this area.

“I think we have seen an AI explosion in structured data such as text and images, but we haven’t really seen that same success yet in unstructured 3D data and shape analysis,” Hanocka said. “So my hope is to be able to achieve that same success in this domain where we don’t yet see the same kind of amazing results.”

Point2Mesh is a technique for reconstructing a surface mesh from an input point cloud. This approach “learns” from a single object, by optimizing the weights of a CNN to deform some initial mesh to shrink-wrap the input point cloud.

One obstacle is the relative shortage of 3D data compared to the deep well of 2D images that can be harvested from the Internet. Training neural networks requires lots of data, and several different research groups have developed methods to convert 2D imagery into 3D models. Hanocka’s research has focused on transforming point clouds into polygonal meshes – an efficient representation of 3D shapes.

By then designing neural networks that work with these mesh objects, Hanocka enables automated analyses and manipulations of 3D objects that previously required difficult and time-consuming software. These edits range from simple changes in characteristics such as color and lighting to 3D-specific alterations of texture and geometry. The networks also enable users to generate new data from limited examples, such as a particular motion sequence, opening up new graphics capabilities for virtual reality, animation, and other media. These new datasets could also help alleviate the shortage of 3D training data for computer vision models necessary for autonomous vehicles and other technologies.

In addition to her research, Hanocka brings new courses in computer graphics to UChicago CS. In Fall 2021, she taught a topics course on “Machine Learning: 3D Geometry Processing and Computer Vision with Deep Neural Networks.” This spring, she will revive and reinvent Introduction to Computer Graphics, a subject of high demand among UChicago students.

And it’s the opportunity to work with those students that attracted Hanocka to the University of Chicago, in addition to the rapidly-growing faculty, the university’s ties to national laboratories Argonne and Fermilab, and the experience of living in Chicago.

“The undergraduate students at the University are really top of the line,” Hanocka said. “I think that they can bring a different perspective and it’s a big strength that can be leveraged in research. When you have such talent, it’s a real asset.”

Related News

More UChicago CS stories from this research area.
In the News

UChicago CS Researchers Share in Special Prize on COVID-19 Research

Dec 01, 2022
Haifeng Xu
UChicago CS News

New CS and DSI Faculty Haifeng Xu Brings Strategic Intelligence to NeurIPS 2022

Nov 28, 2022
UChicago CS News

UChicago AI Summit Examines Promise and Concerns for Science and Society

Nov 01, 2022
UChicago CS News

New Schmidt Futures Fellowship at UChicago to Foster Next Generation of AI-Driven Scientists

Oct 26, 2022
UChicago CS News

New UpDown Project Uses “Intelligent Data Movement” to Accelerate Graph Analytics

Oct 21, 2022
UChicago CS News

Five UChicago CS Students Named to Siebel Scholars 2023 Class

Sep 22, 2022
UChicago CS News

UChicago CS Students Emily Wenger and Xu Zhang Receive Harper Fellowships

Sep 14, 2022
UChicago CS News

First In-Person Robotics Class Lets Students See Code Come To (Artificial) Life

Sep 06, 2022
UChicago CS News

UChicago/Argonne Researchers Will Cultivate AI Model “Gardens” With $3.5M NSF Grant

Aug 30, 2022
UChicago CS News

High School Students in College Prep Program Visit UChicago CS

Aug 23, 2022
UChicago CS News

New 2022-23 Faculty Add Expertise in Linguistics, Visualization, Economics, and Data Science Education

Aug 11, 2022
In the News

UChicago Co-Leads $10 Million NSF Institute on Foundations of Data Science

Aug 09, 2022
arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube