Data Collection
When Science morphs into Music.
As part of the Esch2022 project “The Sound of Data”, various sets of data collected by scientists are being transformed into music.
With “The Sound of Data”, the FNR, the University of Luxembourg, the Rockhal and the Institute of Science and Technology (LIST) aim at building bridges between science and music. The idea is to create music from data, usually available in visual form. This process is called sonification. Valery Vermeulen, a Belgium-based mathematician, musician, and an expert in the field of sonification will introduce musicians, who will be part of the project’s artist residency, to the concept and process of sonification and help them create a music work based on sonified data sets. The outcome of this creative process will be performed live on 3 December at the Rockhal.
Let’s meet the researchers of the University of Luxembourg and of LIST, who collected the data for this project, and learn more about both their data sets and their motivation.
Traffic Data
“We wanted to do something related to our research and at the same time to the infrastructure in the Belval area,” says Emmanuel Defay, Head of Unit Nanotechnology at Luxembourg Institute of Science and Technology (LIST). “Since we work with piezoelectric materials here in our research team, we came up with the idea to use these piezoelectric materials to collect noise from vehicles,” the researcher explains.
The device used to collect the data is quite simple and effective at the same time. It consists of a large plastic box with a lid, like the kind you can buy at any hardware store. A round hole was cut into one of the sides and an empty and open biscuit tin was inserted into this hole with a perfect fit. The piezo buzzers are glued to the underside of the biscuit tin inside the plastic box, which then convert the sound generated by passing cars in the tin into electrical voltage. The open biscuit tin thus served as a microphone and the signals collected with it are recorded with the help of a handheld recorder.
The sounds recorded at various locations are not the only type of traffic data, however. “The original plan was for us to generate numerical data on traffic conditions from the audio data – but that would have been extremely time-consuming,” says Oliver Glassl from the University of Luxembourg, who is coordinating the project. “Fortunately, we were able to make things easier by tapping into the wealth of data that is constantly being collected here in Luxembourg,” he explains. For example, there are numerous induction loops on the roadway that send out a signal as soon as a vehicle drives over them. The data collection uses a Europe-wide standard format for the exchange of traffic data that anyone can access. Since this data has a time stamp, it can be synchronised with the data collected by LIST. “We don’t want to see traffic only as a burden, but also show that you can definitely make art out of it,” he says.
Historical Data
Stefan Krebs takes a completely different approach to collecting his data. The professor of contemporary history at the university has already dealt with sonification in other, interdisciplinary research projects. For “The Sound of Data”, Krebs draws on the data collection of another project, in which an interface application was developed together with colleagues from Switzerland. Thanks to this application, the digital editions of historical daily newspapers can be searched at will for specific terms and criteria. “So, we have a huge database of digitised daily newspapers that roughly covers the period between 1850 and 1950,” the historian explains. “And this data can then be exported into a format that can in turn be sonified,” he adds.
The ways in which this data is used are quite diverse. For example, one could form a keyword chain from various terms such as “Belval” and “steel” and then look at how often they appear in the editions of the daily newspapers and over what period, Krebs explains. The idea is that the participants in the artist residency workshops collect the data themselves according to their own criteria, export it and then import it into the corresponding sonification programmes. “Of course, it will have to be topics and terms that were popular and therefore frequently used,” says Krebs.
“What is interesting to me is, on the one hand, what is best suited for sonification from the data collection and, on the other hand, what I can hear from it as a historian with the corresponding knowledge and how certain peculiarities of the sound can be explained,” he says. Even though he does not believe that sonification can replace the visualisation of data in much of research, he still finds the project exciting. “The beauty of the project is the balancing act between science and art,” he says. “We are dealing creatively with data that in itself, however, definitely has scientific relevance.”
Body Scan Data
“Esch 2022 is just an adventure for now, but it could definitely add value to our further research,” says Djamila Aouada, professor at the university’s Interdisciplinary Centre for Security, Reliability and Trust (SnT). Together with her team, she is contributing data for sonification collected using a 3D body scanner. “It’s a project where we can show what we do,” the researcher explains. For example, she says, more than 4000 human data sets have been collected since 2017 – from different bodies, from clothing and from specific postures.
“Valerie Vermeulen already has data from us – but only synthetic data for data protection reasons – and is now trying to create a sound from it,” she says. While this is a more artistic approach, Aouada and her team are also looking at how the sound can be used for scientific purposes. For example, can you tell from the sound of a body whether it is a tall or short, whether it is a man or a woman?
The SnT researcher was inspired by an old research paper from the 1960s, which investigated the question of whether it is possible to draw conclusions about the shape of a drum with the help of the sound it makes. The answer, which according to Aouada was provided by mathematicians in the 1990s, is: No. You cannot hear the shape. “Different shapes can produce the same sound,” says Aouada. But “the sound can still tell us a lot about the shape”.
Data from Crowdsourcing Art
From the three-dimensionality of the body scanner to the two-dimensional surface of the tablet through which Hugo Parlier and Bruno Teheux generate their data for sonification. The two mathematicians at the university have a fascination for abstract art because, as Teheux explains, it often involves very simple geometric shapes. “Our consideration was therefore whether we ourselves could generate art with the help of the audience and mathematics,” he says. The two researchers were interested in what happens when you give the audience a graphic task: Is it possible to collect all the input and create something from it?
A few weeks ago, a first small stand was set up in the foyer of the Maison du Savoir on the Belval university campus with a tablet on which people can draw or paint with their fingers. Users can be creative in three different modes: They can draw lines, work with colour combinations or with geometric shapes. The programme then creates an image from this, which is saved and will eventually become part of an aggregated work of art as part of the crowdsourcing art project.
“Through the project, we create graphic images and thus also data that can then be sonified,” explains Parlier. “The special feature of our project is that the sonification is basically a supplement to our actual project, because we are primarily creating something visual.” Whether the whole thing will ultimately have added value for further research is something the two mathematicians cannot say. “It is first of all an experiment with many aspects that have a scientific connection,” says Teheux. “If you analyse the artworks mathematically, it will give rise to questions that may not have a direct influence on our further research work but could be interesting nonetheless.”
