Interactive Music Installation

In my graduation project, I am developing an interactive music installation for music festivals that detects the user's movement and appearance and turns it into music. I am working together with the artist André  Kodde who came up with the initial idea. André  Kodde is a Media artist from Amsterdam [1] who does interactive art installations that often come from a socially critical perspective.

At music festivals, a common problem is the lack of diversity especially in the artist line-up and my installation addresses this topic intending to raise awareness. This is an important topic as exclusion has been present in the music industry for a long time and music festivals have always been discriminatory against non-male, non-white or LGBTQ+ artists [2]. Research has also shown that additionally, problems with ableism and ageism are present at music festivals. This discrimination stems from the inequalities present in the whole music industry. Female artists are treated worse for example by the media and problems with sexual assault are present [3]. Festival organizers are aware of some problems with discrimination but there is not enough pressure on them to change their practices [4]. Festival visitors on the other hand are often not aware of the lack of diversity and inequality at music festivals. Therefore, new ways need to be found to raise this awareness so that people can take into account these problems when choosing the next festival and therefore putting pressure on festival organizers.

Even though festivals are a place for leisure, they have increasingly become an opportunity to elicit social change [5]. They are a place for people to engage and discuss new topics and get to know new perspectives, which makes them a good starting point for this project.

The current plan for the installation's user experience is the following. The installation will be situated at a public spot on the music festival grounds to allow people to walk by and a crowd to form. The set-up will consist of a tunnel with a screen attached to the front of it and a control panel for the actors of the artist. The actors will then pick people from the crowd to walk through the tunnel to be turned into a music track. The movement and position of the head of the visitor while walking through the tunnel lets the visitor compose their specific music track. An aspect of their appearance determines the final sound of music. The installation mirrors the discrimination by letting only some groups of appearances create good-sounding music for the current instrument. Actors can change instruments for each visitor that walks through and the music tracks from different visitors are layered and played simultaneously to create a complete song. The screen displays visualizations that show the music track and the factor on which is discriminated, for example, the T-shirt colour. This way, the audience can try and figure out why the specific music track resulted from the current user. The actors can control the factor which is discriminated against to be able to adapt it to the crowd and change the music to fit the specific festival.

Motion and face tracking will be implemented using AI recognition technology. The recognition algorithm analyses the movement of the participants from which the data is determined that is responsible for composing the music track. Thus, attention needs to be paid to possible biases of AI technology. However, the installation picks up this problem by presenting the algorithms as the decider and the main reason for the discrimination as AI biases mirror the inequality in the music industry.

The current status of the project is that the first prototype has been tested at the Maker Festival in Enschede [6]. For this prototype, the user could place notes with their head to create a music track. The instrument could be changed and the different tracks of the different instruments were layered. The discrimination factor has not been implemented yet. However, the visitor's shirt colour is the colour of the music notes. This festival was used as a first opportunity to test engagement and reflect with the artist on the plans and visions.

References

[1] “Andre    Kodde,” Linkedin, [Online]. Available:    https://nl.linkedin.com/in/andre-kodde-7b3b5812/de. [Accessed 31 03 2023].

[2] “FACTS    2022,” female:pressure, [Online]. Available:    https://femalepressure.wordpress.com/. [Accessed 11 03 20232].

[3] H. V.    Amburgh, “A Study of Female Representation in American Popular Music,” The    Honors Undergraduate, vol. 14, pp. 33-49, 2014.

[4] B.    Swartjes and P. Berkers, “How Music Festival Organisers in Rotterdam Deal    with Diversity,” in Festivals and the City: The Contested Geographies of    Urban Events., University of Westminster Press, 2022, pp. 95-109.

[5] E. K.    Sharpe, “Festivals and Social Change: Intersections of,” Leisure    Sciences, vol. 30, pp. 217-234, 2008.

[6] Tetem,    “Maker Festival Twente,” Tetem, [Online]. Available:    https://tetem.nl/makerfestivaltwente/meerdaagsfestival/. [Accessed 14 05    2023].