New Prototypes AeroRigUI and ThrowIO Take Spatial Interaction to New Heights – Literally

Envisioning a futuristic world usually requires creativity, problem-solving, and thinking outside of the box. However, a group of researchers and interaction designers from the Actuated Experience Lab (AxLab) at the University of Chicago believes we don’t necessarily have to look outside the box for the next great idea; but maybe just turn our attention towards the top of it.

Research for two new prototypes that will be presented at the upcoming CHI Conference, were developed with under-utilized ceiling space in mind. AeroRigUI, designed by UChicago students Lilith Yu, Chenfeng (Jesse) Gao, and David Wu, and ThrowIO, by UChicago students Ting-Han (Timmy) Lin, and Willa Yunqi Yang employ tiny robots to create spatial tangible interaction. Both projects were co-directed by Assistant Professor of Computer Science Ken Nakagaki.

“Ceilings are a necessary part of most buildings, but they are currently an underutilized area in our everyday space,” said Nakagaki. “Robots on floors or tables can easily bump into humans and other obstacles, where ceilings can be a solution. At AxLab, we look ahead at how technologies can enrich our physical environment to create a novel and immersive experience. To this end, we found ceilings to be one of the unexplored research spaces for deploying interactive mobile swarm robots.”

AeroRigUI uses strings controlled by self-propelled robots, called RigBots, to change a physical objects’ position and orientation in the air, much like how performers might be moved around at an aerial circus.The ceiling swarm robotic system allows the RigBots to magnetically walk on the ceiling and uses a reeling mechanism, similar to fishing reels, that dynamically controls the strings. Objects can then be attached to the robots via strings and controlled in mid air.

The small robots can be programmed to travel along the ceiling’s surface with an object, creating the possibility for help with everyday tasks. For instance, the RigBots could follow a person around at home with a ceiling light, tilting and angling it as needed to shed light on a specific area. An office space could be enhanced with moving post-it notes, storyboards, or projectors to facilitate the brainstorming process tangibly and physically. Objects could also be stationary but held at different lengths with the strings. This would allow for immersive artistic displays or even be used to create a more accurate 3D model of a constellation map.

Second-year undergrad student Lilith Yu drew inspiration from her desire to captivate people with the world they live in.

“I’ve always been fascinated with leveraging design and technology to enhance human interactions in everyday life. Rather than creating interfaces that consume people’s attention with digital screens, I hope the things we make can ground people into physicality and make people more connected in real life.”

The technology allows for a lot of creativity in the application.

“AeroRigUI has a high level of deployability because it isn’t limited to a single-use scenario; it’s a platform with diverse possibilities,” said predoc student Jesse Gao. “This makes it feasible to implement or merge with other input methods, like voice recognition or hand control, so people can use it for a wide range of applications.”

ThrowIO’s prototype uses the same type of ceiling platform but requires a bit more physical interaction from users. Instead of having objects hung from the interface by strings, the concept works by embedding items with magnets and then throwing them up to the ceiling for ThrowIO Robots to catch. The objects can then be moved around the ceiling and dropped down to users when they would like it again.

Throwing and catching is not a widely explored form of dynamic interaction, but it is a natural movement we use in sports or passing everyday objects. The idea to apply it to robots on the ceiling came from incoming PhD student, Timmy Lin.

“The motivation for this came from being in the juggling club at UChicago for many years. I’m really interested in combining my personal interests with what I study. During COVID, I was always by myself but wanted to juggle with a partner. Now I can juggle with robots. It’s a really cool and novel experience that no one has done before.”

Like AeroRigUI, the possible applications are plentiful. Since the system has a potential to be deployed to any overhanging surfaces, it could even be used to pass objects under a table. It could be used for storage on a ceiling or interactive storytelling at a museum. The two groups tried to focus on creating something with multiple applications instead of solving a single problem.

“We create novel tech to pursue creativity,” said first-year PhD student, Willa Yunqi Yang. “So although our approach might be unconventional from a problem-solving point of view, there is immense value in this type of research by bringing unique, new perspectives on how technologies can be fused into our everyday life.”

The team shared that these two projects are only the first steps in implementing the technology for future use.

“There are still technical challenges to resolve, including safety concerns and scalability of the system,” the research team noted. “However, we strongly hope our work inspires future researchers, roboticists, and user interface designers to look into the unexplored ceiling spaces and make our physical space rich and dynamic.”

The team also intends to explore and develop further use cases, both in practical applications such as rehabilitation, athletic exercises, or day-to-day assistance, and artistic applications, like interactive art and theatrical stage performance.

Nakagaki looks forward to seeing what the AxLab creates next. “I’m sure they will continue taking ideas to new heights- pun intended.”