Four necks and the voice of a popstar

Hugi Garðarsson, 24 Jahre

Last Saturday I had a chance to interview Thor Magnusson, who is a research professor at the Iceland university of arts. He and his team of the Intelligent Instruments Lab at Iceland University of Arts (IUA) are working towards developing more advanced ways of instrument building that is mixed with computer sciences and programming. Thor applied for the prestiguous ERC grant two years ago which was difficult to get. Now, the 2 Mio Euro enable him to hire 8 people for 5 years! One of them is Halldór Úlfarsson, technician and instrument maker and the designer of the halldorophone.

You can see the list of instrument creators here: https://iil.is/people

“The purpose of the project is to better understand artificial intelligence (AI) and our ideas surrounding it, develop it further and nurture it”, Thor explains. “We try to do this through musical instruments with added AI.” These kinds of instruments adjust themselves to you and your playing. They can, for example, predict the next note that you are likely to play and continue with what you were doing.

One of the instruments we find in the lab is called a knurl, designed and invented by Rafaele Andrade. It’s a type of cello with four necks and thus 16 strings. It is an interactive and polyphonic instrument. It is connected to a tiny computer the size of a matchbox that is embedded within the instrument. This computer receives the soundwaves from the strings, adds to them a reverb or distortion, to name a few effects. The other strings that are not being played, can be used as buttons, because an electric current runs through them, which is how it is controlled.

Another instrument, by PhD student Victor Shepardson, is called Notochord. It is a machine learning model trained on 100.000 MIDI files. This model has learned from a wide range of Western music, from Bach to the Beatles. You can use a keyboard or plug in a guitar with a MIDI pick up and the computer model will recognize and answer back the music that you play. It can predict its next likely note. For example, you play a note on the instrument and the computer composes a chord around this note. If you play many notes in a row c-d-e-f, the computer will make a chord progression following the rules of western music theory.

Victor has olso developed a machine learning model that works with sounds, specifically voice. It works like this, you play a note on a plugged-in guitar and out comes the voice of somebody else. Similar to Deep-Fake, he agrees, but of course this is meant to be used for music. This could be developed so well, that it could mimic for example the voice of Freddie Mercury or any pop star. Thor explains that the computer could mimic the character of their singing, but it could of course never mimic their own personal character. The computer couldn´t mimic how these singers would react to specific circumstances in the music. “This is a development that we need to tread lightly with, because this is of course a bit scary thought, to mimic other people.”

Another project is the Organolib/Library of technical elements. “We put tiny sensors, computers and other tools that are used to add to an instrument. For example, the guitar. We hook up these tools to the instrument and it will change how the guitar works, and at the end of the day it has become a different guitar than it was.”

Thor is convinced that Artificial intelligence is an important element of the music technology of the future and we need to understand the dangers. “We want to understand AI through music. Because music is the best lab to deal with all kinds of problems and tasks.” And people are welcome to bring an instrument to the workshop in IUA on Fridays and convert it a little. The IIL team don´t recommend you bring your main instrument, though, just your old beginners’ instrument.

What is the final goal of this group?

“It is to create new musical instruments to understand AI, we then use these musical AI-instruments to understand how people understand AI. And finally, to use these instruments to research important data. We could then use them with their sonification to better understand complex data like the weather, DNA or earthquake activity and we use the musical instruments to review these data to understand them better. Musical instruments become scientific instruments. Microscopes, telescopes and such are scientific instruments and musical instruments are scientific instruments as well.”

ERC (European research counsel) wants to take on exceptional projects that will jump into the unknown in a way that is very likely to give back good results. It is important to know who is applying for the projects and where they come from, so money wouldn’t be wasted on a flop.

Are you worried that when the day comes you have to present that musical instruments can be used as scientific instruments, that you won´t be taken seriously?

“I am not afraid of that. First, many scientists are also musicians and understand music. Music often works similarly to mathematics, engineering and physics. Many of our best scientists and Nobel Prize winners have been musicians. And second, a common topic in science today is practice based research, which revolves around working with materials and tools, and that these tools have a thought progress. We are aware that formulas, pictures, graphs and other presentations to complex sciences need to be interpreted. If you draw up a picture of bacteria or DNA, that picture represents how we think about an obstacle. People understand that these pictures are certain approaches, but not reality. The pictures can remove certain things from the world and improve on other things. Sound can be another perspective of a scientific problem. Sound frame can be another factor to understand the world! I think scientists are open, curious and positive towards different approaches, and I am looking forward to working with as many of them as possible. We have always had positive receptions when introducing our work to scientists from other fields.”

Thor’s team in the workshop design and develop the instruments themselves and guests are invited to come and try out the instruments and give their thoughts and experiences with AI instruments. They enjoy working with people and are happy to tell you all about their different projects. The workshop is open every Friday at 15:00. The project is one year old now and many people have come for a try out.