(en)Automatic Listening and the Panacousticon

Automatic Listening and the Panacousticon

2022 · Print · Machine Learning

The emergence of smart devices equipped with microphones, whose use is becoming increasingly commonplace, calls for an updated examination. Auditory surveillance is being organized within our very acoustic spaces through digital infrastructures, the operation of which—obscure and impenetrable—is deliberately concealed from us.

The uses of these technologies may seem harmless at first glance, but I wanted to explore in more detail what this permanent listening of machines entails, which the French philosopher Peter Szendy calls "panacoustic."

The field that enables this machinic listening is described by the English term "Machine Listening," which could be translated as "automatic listening." Machine Listening refers to an interdisciplinary and rapidly developing field of science and engineering that uses audio signal processing and machine learning to extract meaning from sounds and speech. This is what allows us to be “understood” by Siri and Alexa or to recognize a song with Shazam. Personally, I focused not on voice recognition, but on technologies that listen to our sound environments and make sense of them. These technologies are commercially deployed in the form of software integrated into surveillance cameras, voice assistants, and smartphones.

I tried to make visible the mechanisms hidden in these new sound recognition tools, while also aiming to raise awareness among viewers about their usage, confronting myself with this constant algorithmic listening. These devices are arriving in our daily lives, on the smartphone market, in voice assistants, or in the connected home sector. They are entering our cities discreetly, occupying both our private and public spaces. Governments collect data for security and control purposes; private companies do so for marketing, personalization of offers, and improving their management. But private companies are beginning to focus on security and control, blurring the lines between state and corporate infrastructures. In my view, these devices must be revealed and demystified so that we can place them in the public debate.

This project was exhibited at the Centre de Création Contemporaine (CCOD) in Tours during the exhibition U.S.B #6 : nous vivons à la lisière, from March 24 to May 21, 2023,

At the ESAD Orléans for the exhibition U.S.B #3 ∙ Void draw () {Carte blanche aux diplômé∙es 2022}, from November 17 to December 1, 2022.

Photo : Paul de Lanzac

For my research on sound recognition software, I didn’t want to simply use off-the-shelf services, but rather try to understand the internal workings of these technologies, which are becoming increasingly complex and to which we have very limited access, by practicing a form of reverse engineering. So, I used an open-source sound recognition program, the code of which was published on GitHub. The program is called Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition.

Being somewhat of a beginner in the field of programming, I had the chance to be guided by Simon, a digital mediator at Labomedia, an association based in Orléans that specifically studies digital art and technology fields. Every Thursday for a little over a month, I went to their offices to get the code to work. This software analyzes an audio file by listing all the sounds it recognizes in order of appearance and recognition rate.

For my first experiment, I recorded myself for an entire day, from 9 am to 7 pm, using a microphone hidden in my jacket pocket. I then submitted this recording to the sound recognition program.

The program recognized an impressive variety of sounds. Some of them reflect the rhythm of my day: almost every hour and half-hour, the sound of the bell of the Orléans Cathedral was recognized under the label “Church bell,” “Bell,” and “Jingle bell.” The sound of my keys recorded the entry or exit from my home. Door noises signify that I’m entering or exiting a room. The category “Speech” suggests the frequency of my social interactions.

I decided to translate the result of these recordings into a monumental print featuring only 3 hours of sound analysis. Magnifying glasses are provided to read these tiny characters and, perhaps, imagine what sounds filled my day.

Photo : Paul de Lanzac

Exhibition U.S.B #3 ∙ Void draw () {Carte blanche aux diplômé∙es 2022}, Galerie de l’ÉSAD Orléans, November 17 to December 1, 2022

Photo : Communication ESAD Orléans