Expressing emotions, creating art with Robots

Alum María Santos’ art is supported by robots. The goal of her research work is to experiment with creating a tool for artistic expression. // Photo courtesy of

Engineering and art are often seen as polar opposites, but this past Monday, Jan. 25, director of GT Arts Aaron Shackelford interviewed alum María Santos about her work combining the two topics, revealing more similarities than one would expect.

Santos is a double-graduate of Tech, with both an M.S. and a Ph.D. in ECE.

She is now a postdoctoral research associate in the department of Mechanical and Aerospace Engineering at Princeton University.

Her research focuses on the distributed coordination of multi-robot systems and is particularly concerned with the capabilities of large swarms of robots.

She explores the potential of these swarms in artistic expression, especially that of theater and music.

Explaining more about her work, Santos said that during her time at Tech, her team studied the creative ability of swarms to evoke emotions in an audience.

“Think about using a multi-robot system for a theatrical exposition,” Santos said. “How can you evoke happiness or sadness?”

The goal in Santos’s research is to convert these global sensations into individual commands that can instruct groups of these robots, even when the robots have different capabilities.

Her team also worked on a method for the robots to serve as an interactive tool between a human artist and a canvas for painting.

When asked what inspired her interest in this research, Santos said that while studying in Spain, she enjoyed the combination of her passion for music and love of engineering and more technical areas.

“I’ve always been fascinated by music,” Santos said. “I’ve been interested by these two very different thought processes, engineering, which is very structured, and music, which is structured as well but is thought about in a different way [than engineering]. I wanted to merge these but didn’t think it was possible. I was frustrated by the norms I saw in engineering.”

From this frustration, Santos decided that her goal was to merge the engineering into the arts by offering artists “intuitive” ways to create robotically-supported artwork.

Santos explained that for her, intuitiveness in the arts means being able to produce certain behaviors in the robots with very simple rules.

“In this emotive movement to create different emotions, we thought about how we could use standard behaviors to create sadness,” Santos said. “In painting, we focused on the robots as an active brush — red over here, blue over there.”

Santos began her work in earnest after taking a linear systems class taught by ECE professor Magnus Egerstedt.

“He got excited when I told him I studied music theory in Spain,” Santos said. “We started work on controlling robots through chords and harmony rules. This is quite challenging.”

“We developed a research agenda where we mixed the mathematical side of multi-robot system coordination with the artistic side, using tools for artistic expression.”

When Shackelford asked about challenges she faced during this project, she answered these mostly stemmed from the complexity of the artistic condition.

She also pointed out the high expectations that people have of robots and of what they can accomplish.

Santos explained her efforts to turn simple instructions for the robots into abstract methods of interaction.

“Sometimes, when you say you want to increase the efficiency in robots by a certain percentage, you have a more determined, narrow objective,” Santos said. “But when you do something artistic, and the possibilities are endless, there are difficulties you must face.”

Shackelford raised the issue of robot autonomy and whether robots would ever become their own artists.

Santos answered that she thinks of robots as tools to enhance the creative process.

“We have pre-programmed rules that are somewhat deterministic, but also somewhat random. The genius is probably where to break those rules,” Santos said. “… Art is boring if it’s predictable. Art is so related to the artist and I’m not sure how you’d do that with robots.”

As a final piece of advice for Tech students interested in similar areas of research, Santos emphasized the importance of working on something actually interesting to them.

“I was so happy to get to the lab every day and work with people I really enjoyed,” Santos said. “Don’t care about what everyone else is excited about.”

She then emphasized the value of communicating and learning from others in fields you are interested in exploring more of.

“This one took me a long time to internalize, but talk to people,” Santos said. “Don’t be scared of leaving your lab — you’ll gain new ideas. If you’re just in your lab 24/7, you won’t get that.”

Her work has the potential to make the work of artists not only more efficient but more integrative of the humanities and sciences, two fields that are frequently pitted against one another.

Moving forward Santos and her team would like feedback on what robotic abilities artists are interested in.

The event took place over Facebook Live. Follow @artsattech to learn about similar livestreams and events featuring artists, creators and alumni from the Tech community.