Artificial intelligence has gotten better and better at understanding the visual world, classifying two-dimensional images and video without human intervention. But the application of neural networks, deep learning and other AI approaches to create new graphics and work with three-dimensional data is still in its earliest stages. The production of 3D graphics remains accessible only to experts, and the application of AI to analyze 3D data and produce new images is still limited in its scope and possibilities.

Rana Hanocka, a new assistant professor who joined UChicago CS in the summer of 2021, aims to democratize this technology, building AI capabilities for 3D data that transform aspiration into reality. Her research creates new frameworks and applications that allow users to more easily and expansively work with 3D geometric data: transforming two-dimensional images into 3D structures, changing features with descriptive text, and generating realistic synthetic data for the training of computer vision models.

Her research group 3DL — an acronym for “3D Deep Learning,” pronounced “threedle” — produces new tools that can help artists, designers, architects, and hobbyists create new 3D objects with limited technical expertise. For example, a user could take a 3D model of a table, chair or any other object and manipulate its appearance and texture through text inputs: “crochet,” “wicker,” “brick,” or more fantastical choices such as “cactus,” or “astronaut.”

“We’re interested in how to make things that work automatically, while being more intuitive and easier to understand and use,” Hanocka said. “We design tools for manipulating 3d objects, creating theatrics, and animating them. And we want to make our research available, enabling people to use these tools so that people can build off of and extend them.”

Hanocka joined the University of Chicago after completing her Ph.D. at Tel Aviv University in Israel, where she studied with Daniel Cohen-Or and Raja Giryes. She gravitated to studying computer graphics as a means of combining her interests in mathematics and visual art, seeing the potential of new AI approaches in this area.

“I think we have seen an AI explosion in structured data such as text and images, but we haven’t really seen that same success yet in unstructured 3D data and shape analysis,” Hanocka said. “So my hope is to be able to achieve that same success in this domain where we don’t yet see the same kind of amazing results.”

Point2Mesh is a technique for reconstructing a surface mesh from an input point cloud. This approach “learns” from a single object, by optimizing the weights of a CNN to deform some initial mesh to shrink-wrap the input point cloud.

One obstacle is the relative shortage of 3D data compared to the deep well of 2D images that can be harvested from the Internet. Training neural networks requires lots of data, and several different research groups have developed methods to convert 2D imagery into 3D models. Hanocka’s research has focused on transforming point clouds into polygonal meshes – an efficient representation of 3D shapes.

By then designing neural networks that work with these mesh objects, Hanocka enables automated analyses and manipulations of 3D objects that previously required difficult and time-consuming software. These edits range from simple changes in characteristics such as color and lighting to 3D-specific alterations of texture and geometry. The networks also enable users to generate new data from limited examples, such as a particular motion sequence, opening up new graphics capabilities for virtual reality, animation, and other media. These new datasets could also help alleviate the shortage of 3D training data for computer vision models necessary for autonomous vehicles and other technologies.

In addition to her research, Hanocka brings new courses in computer graphics to UChicago CS. In Fall 2021, she taught a topics course on “Machine Learning: 3D Geometry Processing and Computer Vision with Deep Neural Networks.” This spring, she will revive and reinvent Introduction to Computer Graphics, a subject of high demand among UChicago students.

And it’s the opportunity to work with those students that attracted Hanocka to the University of Chicago, in addition to the rapidly-growing faculty, the university’s ties to national laboratories Argonne and Fermilab, and the experience of living in Chicago.

“The undergraduate students at the University are really top of the line,” Hanocka said. “I think that they can bring a different perspective and it’s a big strength that can be leveraged in research. When you have such talent, it’s a real asset.”

Related News

More UChicago CS stories from this research area.
UChicago CS News

New 2022-23 CS Faculty Add Expertise in Linguistics, Visualization, Economics, and Data Science Education

Aug 11, 2022
In the News

UChicago Co-Leads $10 Million NSF Institute on Foundations of Data Science

Aug 09, 2022
UChicago CS News

UChicago CS Faculty Receive Industry Grants From J.P. Morgan, Google

Jul 19, 2022
UChicago CS News

UChicago London Colloquium Features Data Science, Quantum Research

Jul 01, 2022
Video

Is it Ethical to Use Facial Imaging in Decision-Making?

Jun 28, 2022
UChicago CS News

Faculty Bill Fefferman and Chenhao Tan Receive Google Research Scholar Awards

Jun 21, 2022
UChicago CS News

Two Incoming UChicago CS PhD Students Receive Department of Energy Fellowship

Jun 16, 2022
Video

Data Science Institute Summit

Jun 15, 2022
UChicago CS News

DSI Summer Lab Returns In-Person With 49 Students From Across the U.S.

Jun 14, 2022
In the News

WiGRAPH Features Asst. Prof. Rana Hanocka on Her Research and Career Path

May 11, 2022
UChicago CS News

UChicago CS Labs Join Museum of Science & Industry For Robot Block Party

Apr 20, 2022
UChicago CS News

Three UChicago CS Security Projects Receive Funding From c3.ai Institute

Mar 25, 2022
arrow-down-largearrow-left-largearrow-right-large-greyarrow-right-large-yellowarrow-right-largearrow-right-smallbutton-arrowclosedocumentfacebookfacet-arrow-down-whitefacet-arrow-downPage 1CheckedCheckedicon-apple-t5backgroundLayer 1icon-google-t5icon-office365-t5icon-outlook-t5backgroundLayer 1icon-outlookcom-t5backgroundLayer 1icon-yahoo-t5backgroundLayer 1internal-yellowinternalintranetlinkedinlinkoutpauseplaypresentationsearch-bluesearchshareslider-arrow-nextslider-arrow-prevtwittervideoyoutube