Students at Tech are well-known for their innovation in the areas of engineering and computer science; however, there are some who choose to do the impossible and bring liberal arts into the mix as well. Enter Mason Bretan, a Ph.D. student at the Center for Music Technology here on campus.
Bretan specializes in a brand of artificial intelligence that seeks to harness the mysteries of music through the medium of robotics. The robots he works with, nicknamed Shimon and Shimi, are currently coded to create music via improvisation and interaction with their surroundings; the Technique was recently able to pose a few questions to Bretan regarding his past experience, his current work and his hopes for the future of music technology.
Technique: What made you interested in combining music and technology in the first place? Is there someone or something that was a big influence on you and kind of led you to do research in the subject?
Bretan: Growing up I loved music and I loved science, but they were always separate and I had never really given much thought about combining the two fields until I started my undergrad at UC San Diego. There is a large music department at UCSD with a heavy emphasis on music technology; in fact it’s home to many notable names in the field.
I would attend performances and see these great musicians using computers to create systems that could automatically respond to them resulting in sounds and timbres I had never heard before. This completely changed my perspective on music and it really opened my eyes to the aesthetic possibilities that science and technology can provide to music.
So I started to explore this domain and as I began to become more familiar with higher level computer science, signal processing and acoustics concepts I decided that there was still so much to learn and explore within music technology, so I applied to grad school. And now I’m here at the Center for Music Technology at Tech, and I’ve enjoyed every moment of it!
Technique: What ideas led to the creation of Shimon and your other robots?
Bretan: Robotic musicianship in general combines two ideas, namely musical mechatronics which focuses on construction of sound producing mechanics, and machine musicianship which focuses on developing algorithms and cognitive models representative of various aspects of music perception, composition and theory.
By creating robots capable of understanding and generating music we hope to supplement and enrich the musical experience for humans in a way that would not be possible from only human-human musical interactions. Computers and robots are capable of doing things humans aren’t and I and other researchers in the field believe there is artistic potential in these non-human characteristics.
Technique: What kinds of things are you currently working on? Is there still more progress you hope to make with Shimon?
Bretan: There’s still plenty to do with Shimon, Shimi and in the field of robotic musicianship in general. Recently I’ve been working on a generative system for robot musicians that allow them to “think” about their embodiment and physical constraints when making musical decisions. I call it “musical path planning” and it works by jointly optimizing for the robot’s physicality and its musical knowledge and goals. Actually, part of the reason I made my video was to demonstrate my implementation of this idea in action.
Recently our group has also been focusing on wearable robotics and how we might be able to augment human ability with additional limbs and learn how the augmentations can influence the music.
There are plenty of challenges in this and our first prototype was built for an amputee drummer. We had some success and some nice musical results and now we want to go a step further and see if an additional “drumming arm” can be useful for an able-bodied drummer.
Technique: What aspirations do you have for the things you’re doing here at Tech — what do you hope will result from your work?
Bretan: I want to show the world that robots and machines can be used for applications other than merely computation or completing tasks. I think they can also be artistic, creative, and perhaps even emotional. Often my research is met with some resistance from “purists” who believe that music making and art is a uniquely human activity and could never be replicated by a cold, calculating, soul-less machine.
In my opinion, robots have the potential to expand artistic creation just as other technologies such as turn tables, the electric guitar or digital audio workstations have influenced musical culture. Technology is a good thing and creative people always find ways of using it to enhance artistic expression.
Technique: What kinds of real world applications do you see for your research?
Bretan: I think music is a real world application, but the research can also impact other fields. Extracting meaningful information from an audio signal to create an algorithm enabling improvisation encompasses everything from signal processing, to machine learning, to cognitive science.
Music is also an extremely time-demanding task, so a lot of effort is made to make things efficient and run in real-time. Beyond the science that goes into developing our robots, music is also a domain that is shared and enjoyed by billions of people, and such ubiquity can be leveraged to encourage people to engage with robots.
I think we will be seeing social robots become more integrated into everyday life and robots that demonstrate an understanding and even enjoyment of something that is so familiar to people can help to establish trust and confidence in the new technology.
Technique: After you receive your Ph.D, what do you plan on doing?
Bretan: I don’t know yet! Music technology and robots are both big and growing fields right now so hopefully there will be lots of opportunity when I graduate.