EAR-Stretch: Enhancing aesthetic reception of contemporary music through active tempo control with physiological monitoring*
August 15, 2021 – Aug 14, 2023
Marie Skłodowska Curie Action Individual Fellowship
Emily Graber, MSCA Fellow
Elaine Chew, Supervisor
UMR9912 CNRS/STMS, Institut de Recherche et Coordination Acoustique/Musique
Paris, France
See all posts: https://emilygraber.com/category/ear-stretch/
Summary of the action
The EAR Stretch Action was about enhancing aesthetic reception of Contemporary Classical Music (CCM) using active tempo control. This action was proposed because CCM is often disliked by the general public, yet it is the lifeblood of current classical music development and contributes to the creation and evolution of culture by making unique artistic experience that express and confront modern problems faced by humanity.
EAR Stretch hypothesized that non-specialized audiences can quickly become uncomfortable or disengaged while listening to CCM due to a dearth of ongoing predictions. Compared to canonical Classical Music or popular music genres, CCM may alienate listeners before they can have emotional responses and develop an appreciation for the music.
EAR Stretch aimed to enhance the reception of CCM by adding embodied interactions to newly composed music to simultaneously boost and prolong engagement with the music and to guide the formation of temporal predictions while listening to the music. The embodied interaction explored in the action was direct tapping on a bespoke conductor interface. Tapping is an effective embodied interaction because rhythmic motion reinforces neural entrainment in the brain between the motor and auditory systems. Entrainment also guides temporal predictions and modulates attention in synchrony with the music.
Parsing the music through tapping was expected to reduce the perceived complexity of the music and improve the audience’s liking or appraisal of the music. In the implementation of the action, an interface for active engagement was created to give the user active control of the ongoing music such that the user was not only able to tap along to the music, but also select a tempo and vary it according to their preferences and needs.
Experiments were carried out to determine underlying physiological responses to CCM, baseline appreciation of CCM, how temporal prediction is disturbed by CCM-like pitch content, and how interactive tapping impacts reception of music. The findings of this work were disseminated in a number of international conferences, workshops, and public events. This action combined the domains of music composition, music technology and auditory cognition and took place in the unique laboratory for Science and Technology of Music and Sound (STMS) at the Institute for Research and Coordination of Acoustics/Music (IRCAM) in Paris, France. The grant (No. 101028532) was hosted by the CNRS.
Experiments
+ [1] Liking ratings for CCM and non-CCM online
This study was run online with 131 anonymous participants. The study was done both to understand the baseline preference for CCM and to determine if responses of like/dislike depended on the particular exemplar of CCM and the length of the exemplar. Subjective liking ratings were gathered for 35 different CCM excerpts for solo piano. Control stimuli for style (35 non CCM excerpts) and context (70 ultra-short excerpts) were rated as well. The ratings revealed that preferences could be formed for decontextualized musical clips, and CCM excerpts on a whole were significantly more disliked than non CCM excerpts despite the variance in the sample in terms of rhythmic regularity, dynamics, and harmonicity.
+ [2] Physiological differences while listening to CCM and non-CCM
An EEG/ECG study was conducted at IRCAM to characterize baseline physiological responses to CCM and non-CCM played by a Disklavier. A mixture of audio experts and CCM-specific experts volunteered for the experiment. They were tasked with listening to two 15-minute excerpts of human-performed MIDI on Disklavier and rating their liking of the music. The data were analysed using an adapted EEG decoding method. The method attempts to reconstruct the original audio signal from the physiological data; in this experiment it was used to determine if the audio could be reconstructed from the ECG alone, EEG and ECG, or EEG alone.
All reconstructions were statistically significant for the non-CCM piece, but only CCM experts had significant heart model reconstructions for CCM. This suggests that the CCM-experts had similar listening experiences for both types of music. On the other hand, the audio experts were not able to engage with CCM in the same way that they could with non-CCM.
+ [3] Tempo change perception while tapping to prefixed stimuli
This behavioral study was conducted at INSEAD to determine whether tapping would make temporal predictions more robust in several different pitch contexts, including a CCM-like context. 30 non-expert participants made judgements about the onset of tempo changes within pitch sequences while either tapping or not tapping to the sequence. Tapping made the detection of decelerations marginally more accurate, while significantly slowing overall reaction time (RT) to all tempo changes. Slowed RTs were interpreted to mean that participants were either a. experiencing more groove in the stimuli, making tempo changes less salient and detectable, or b. that the task of tapping interfered with the task of detecting tempo changes. The former may be supported because in the case of CCM-like pitch contexts alone, tapping did not change RT.
Tapping did not lead to more accurate temporal perception, yet these results still shed important light on the prevailing negative reputation of CCM. Specifically, if tapping fails to promote groove in CCM contexts, musical expression in CCM may be misunderstood. Consequently, listeners could have the impression that CCM is inexpressive or unemotional.
+ [4] Liking ratings for CCM and non-CCM while tapping to prefixed and interactive stimuli
This study was conducted at INSEAD to determine if interactive tapping to real CCM excerpts, including SALO (Graber, 2022), would improve the liking ratings given to the CCM excerpts. This experiment used the final version of the MIDI control interface, and the performed MIDI recorded by during the fellowship. The complete experiment was implemented in Max/MSP, including stimuli presentation and data acquisition.
Fifteen non-expert participants were tasked with rating their enjoyment of CCM and non-CCM clips that contained more or less information about the beat. As the participants listened to the clips, they tapped along as the music either continued unaffected, or interactively adjusted to their tap rate. The participants were not made aware of the adjustments. The results showed that participants tended to rate CCM more highly when the music adjusted interactively to their taps compared to when the music was non responsive. For non-CCM clips, the trend was reversed, i.e. interactive tapping slightly decreased the average enjoyment rating given to the music. Overall, CCM was rated below non-CCM.
Publications
Graber, Picasso, Chew (2022) Viva Contemporary! Mobile Music Laboratory. In Proceedings of the 30th ACM International Conference on Multimedia (MM ‘22), Association for Computing Machinery Multimedia, Lisbon, Portugal, 10-14 Oct. 2022.
Peer-reviewed abstracts
Graber (upcoming) Embodiment and Engagement in Contemporary Classical Music. Spoken presentation at the Virtual International Conference for Music, Body, and Embodiment: New Approaches in Musicology, Centro Studi Opera Omnia Luigi Boccherini, Lucca, Italy [09/12/2023]
Graber, Solinski, Chew (2023) [26/08/2023] [program] Decoding Music’s Amplitude Envelope from Neural and Cardiovascular Signals. Poster presentation (virtual) at the 17th International Conference on Music Perception and Cognition, Tokyo, Japan.
Graber (2023) [abstract, video] Effects of sensorimotor synchronization on temporal discrimination in different pitch contexts. Spoken presentation (virtual) in the Entrainment Workshop at RITMO Centre for Interdisciplinary Studies in Rhythm, Time and Motion, Oslo, Norway [17/08/2023]
Graber, Chew (2022) [05/08/2022] [abstract] Toward Enhancing Contemporary Classical Music Reception: A Comparison of Contemporary and Non-Contemporary Classical Music Reception. Spoken presentation at the biennial meeting of the Society for Music Perception and Cognition, Portland, OR, USA.
Graber & Chew (2022) Contemporary Music Embodiment and Perception. Spoken presentation at IRCAM Forum Workshop, Paris, France. [25/03/2022] [abstract, video]
Invited talks and demos
Graber (2023) Perceptual Inquiries on Contemporary Music: Reconstructing musical features from neural and cardiovascular signals & pitch distractions in temporal processing. Spoken presentation at Mercredis de STMS, IRCAM, Paris, France. [14/06/2023] [abstract, video]
Graber (2022) Embodying Contemporary Classical Music. Invited talk at RITMO Centre for Interdisciplinary Studies in Rhythm, Time and Motion, Oslo, Norway [23/11/2022] [abstract]
Graber (2022) EAR Stretch Conductor Interface. Invited demo at the workshop for Embodied Perspectives on AI, RITMO Centre for Interdisciplinary Studies in Rhythm, Time and Motion, Oslo, Norway [21/11/2022] [abstract]
Graber (2022) Women in Music Technology. Invited Talk at the Diversity Lunch for ACM International Conference on Multimedia (MM ‘22), Lisbon, Portugal [12/10/2022]
Graber (2021) Introduction to EAR Stretch – Shaping tempo to enhance aesthetic reception of contemporary music. Spoken presentation at Mercredis de STMS, IRCAM, Paris, France. [24/11/2021] [video]
Outreach
Cognivence Forum des Sciences Cognitives de Paris Open House EEG Demonstration [16/04/2023]
Disklavier Workshop co-organizer and instructor with Julie Zhu, CCRMA, Stanford, CA [27-29/03/2023]
Select Contemporary Performances
Sanchez-Gutierrez …voici le bateau por les calanques… (2000) – Contemporary music, Italy [July 9, 2022]
Cage Imaginary Landscape No. 4 (1951) – Broadcast concert for La Nuit des Idées, Paris [Jan 27, 2022]
- Follow Me
two percussionists. Italy [13/07/2022] *soundSCAPE, Italy premiere - Sanchez-Gutierrez Piano Quintet
Sanchez-Gutierrez …voici le bateau por les calanques… (2000) – Contemporary music, Italy [July 9, 2022] - SALO
See all posts: https://emilygraber.com/category/ear-stretch/
*This project has received funding from the European Union’s Horizon 2020 research and innovation programme No. 101028532.