From Data to Sound

For the Esch2022 project “The Sound of Data”, which will officially launch on 1 May at Rockhal, mathematician and electronic musician Valery Vermeulen is very much the interface between science and music.

Valery Vermeulen has two passions: music and mathematics, or more precisely theoretical physics. For a long time, it was a matter of choosing one or the other for him. Last December, the mathematician with a doctorate in mathematics released an album that combines both passions. The musical work with the somewhat unwieldy title “Mikromedas AdS/CFT 001” is, to a certain extent, the scoring of black holes to music. But because you can’t get close to them without being swallowed up, scientists must rely on simulation models. Valery Vermeulen used data from such simulations for his project. However, he sonified them instead of visualizing them, i.e., he converted the data into sounds.

In “The Sound of Data”, the mathematician and electronic musician therefore fulfills a very special role. Various datasets were collected as part of the Esch2022 project at the University of Luxembourg and the Luxembourg Institute of Science and Technology (LIST). These are traffic data, historical data, 3D body scan data and data collected via a crowdsourcing action. Valery Vermeulen’s task is to convert some of the data into musical building blocks that musicians selected for the project will then work with. In addition, the sonification expert will provide the musicians with 6 months of workshops and mentoring on data sonification, and on using data for musical purposes.

On 3 December, the result of this experiment will be presented at the Rockhal. Valery Vermeulen will also be on stage then.

Valery will explain exactly how he goes about his “data sonification” in a lecture performance at the “Science meets Music” event on 1 May at 6 pm. The title of his lecture performance is “Music, Deep Space and Black Holes”.

Science.lu interviewed Valery Vermeulen ahead of The Sound of Data’s launch event.

Valery, within the framework of “The Sound of Data”, all the data that is to be used for sonification ends up with you. What is the first step you take?

Well, the first step is to visualise the data – as far as that is possible. So, I first approach it like a data scientist to see what I can do with it. The data for this project come from four scientific sub-projects and are therefore completely different. Some data sets are therefore easier to handle than others.

Is it more difficult to work with the 3D data from the body scan than with the traffic data or the historical data, for example?

What is very important is whether the data has a temporal reference, as is the case with the collected traffic data, for example. This time component makes sonification easier. And the same basically applies to the crowdsourced data. The data from the 3D body scan, on the other hand, lacks this time component, which is why the sonification of this data is also more complex. And as far as the historical data is concerned, we move a little in between, because we are also dealing with the meaning of certain words and thus with semantic data. So, we are dealing with data that cannot be assigned so clearly.

Is the data sufficient for you or do you also need additional information about how the data sets were created?

The data alone are sufficient for me for the time being. But of course, I need to know what is behind these data and how they correlate with other data strands from the respective project. But since I have also worked as a mathematician in data science for more than 15 years, I also know how to proceed.

How do you prepare the data for the musicians? Are they converted into notes?

First, I look at which sonification strategies are best suited and then I experiment with them a bit. And in the workshops that I will lead, I want to familiarise the participants with these strategies. So that they know what they can do with the data. I need a few representative samples of the data. It’s not about me creating a big archive of samples from the data set. That is more an option for the workshop participants. Of course, it would also be possible to simply convert the data into notes. But I don’t find this approach contemporary. It is quite easy to create a programme that takes over this task. But it is much more interesting to dive as deep as possible into the data to learn something about the structure.

But that’s not your only role in The Sound of Data, is it?

No, not at all. I will hold a whole series of workshops over a period of six months during the project, where I will first familiarise the musicians with data sonification and certain sonification strategies. The musicians will receive instructions from me for their compositions, which they will present at the Rockhal on 3 December. In addition, I myself will compose a 45-minute work from the body scan data and then premiere it at the Rockhal on that day as well. All in all, I’m taking over most of the artistic side of the project.

Of course, you will also be at the big kick-off event “Science meets Music” on 1 May at the Rockhal. What can visitors expect at your lecture performance?

Among other things, it’s about the work behind my project Mikromedas, from which the album about the black hole emerged. But above all, my lecture performance will deal with how music and mathematics can be combined with the help of sonification. And everything I explain, I will then of course also demonstrate live with examples.

You use scientific data to produce music. To what extent do you think this creative process can also be useful for science?

Normally, data is visualised. If you also sonify it, you can sometimes discover new elements and structures. I am currently working with body data and therefore must look for new ways to sonify 3D data. It’s about the connection between geometry and sound. In my black hole sonification project, I was faced with the challenge of sonifying the gravitational waves that occur when black holes merge into one.

There was already sonified data of these black holes, but the resulting sound was not that inspiring. So, I looked for a new way to sonify the black holes. I had to deal more intensively with the physics of gravitational waves, simulated processes and dealt with processes that had not yet been researched in this way. I didn’t make any scientifically significant discoveries, but I did come across some interesting connections. In my case, sonification helps to deal with physics in a different way.

The Sound of Data is a collaborative project between

Media Partners

RTL Media Partner
Eldoradio Media Partner