Putting Your Foot Down: the Aesthetics of Latency

Paper presentation proposal, Dr. Gregory Sporton & Tychonas Michailidis, Visualisation Research Unit, Department of Art, Birmingham City University.

It will be presented at the Sixth Biennial International Conference on Music Since 1900
2-5 July 2009, Keele University

Traditionally, latency is considered as a problem in performance contexts, and nowhere has this been more apparent than in the use of the Internet for networked-based performance. The absence of latency in stand-alone computers contrasts with the experience of networked performance, creating ambivalence about the role the network can play in computer-based sonic and visual art.

This paper suggest that rather than being a problem for the digital artist, latency is an inherent property of the network, and as such is one of the determining features of the creative space. This demands that creative practitioners account in their work for latency as a creative and aesthetic aspect of digital arts practice.

The opportunities are more apparent when it comes to live performances or interactive installation using technology. Hardware communication, long cables and networking produce significant latency that does not respond instantly to the performance or installation situation. We discuss, through practical demonstration, not how to reduce latency to the minimum but rather how to use latency as a creative and aesthetic property within the presentation structure. Creativity through latency is exploited by suspending the expectations of what we experience visually, aurally and sensually.

We approach the aesthetics of latency in two ways. Firstly we examine the effect of different latencies by delaying the audio from the visual and vice versa. Secondly, we look at the effects of latency in audio-haptic domain. By examining the areas above we present some alternative approaches in digital creativity practice, experiencing latency from an audience perspective as well as the challenges for the performer.

Attractor Performance 06/01/2009

The performance took place at the recital hall of Birmingham Conservatoire on the 6th of January 2009






The program notes



The poster

Rehearsals comments

I think now is time to cover all that happened since the last time I wrote in this blogg.
Few things from the last update:

1. I had to decide about the artistic outcome of the project since my previous idea (about the audience interaction) did not work.
2. Problems with the performers or to be more precise, I didn’t know if they would show up since I had never worked with any of them in the past to know them and trust them.
3. The piece needed to be polished and ready for the performance

Since the last update 8/12/08 I had 3 rehearsals and the performance on the 6th of January. The three rehearsal was at the 16th and 20th of December and on the 5th of January.
I will start with describing what happened in the rehearsals:

16 December 08-Rehearsal 1 14:00-17:00

This rehearsal, as the other two and the performance, happened in the recital hall at Birmingham conservatoire. I had only three hours on my disposal so I thought that the best rehearsal would be to test if all the equipment works without the performers. We took all the equipment (me and Julia) laptops, soundcards, cables and more cables and tried to put everything together. Julia is the other MA student that she had a film installation performance on that day, 6th January 2009. After three hours and with a lot of help from the conservatoire technician (Matthew O'Malley) everything worked. The 8-channel setup was ready. I had to write down most of the setup configurations (cables, soundcards inputs outputs etc) because we had to unplug everything by the end of the rehearsal. This rehearsal was very important to found out if the technology works and any possible problems.

20 December 08- Rehearsal 2 10:00 -16:00

That day only the saxophone player came and that made me worried more about the other performers. After setting up again all the equipment I had to make sure that everything works. Even though I had done the same thing few days earlier I had still a few things that did not worked as planed. I was though confident because everything had worked the first time, and I had plenty of time to found out how. Again this rehearsal was more about testing the equipment rather a piece rehearsal. What I have gain from all this, is that technology and equipment need to be performed without the performers in advance. I believe this is something that electronic composers and performers start now to realise (at least I did).
Since the piece was not even close to any traditional performance I knew that writing down (on paper, notes) the whole performance would not worked. The 4 Wii players had to move in the three-dimensional axis, in space, and push different buttons on the Wii. For that reason I tried to approach this composition with a different technique. Instead of writing what they had to do, I saw what they could do and then wrote it down on paper.
At the rehearsal I had worked the way the saxophone and the Wii remote could work as one. What I did eventually was to allow the Wii remote to manipulate the sounds of the saxophone by controlling the amount of different effects (from Abelton Live) through the Wii remote. After knowing what the saxophonist could do I wrote a twelve bar melody in 3 different keys. The saxophone player could choose to play for any number of times any of the twelve bars. The Wii performer had all the twelve bars(sound files) pre-recorded and stored in advance so that he could recall randomly any of them. There are more details that I am not going discuss now.

5th January 09_Rehearsal 3 10:00- 17:00

The same process was done in this rehearsal as well. Set up everything, make sure that everything works and finish the composition. All the performers finally showed up ☺. I had to explain them what I have done in Max and how it worked with the Wii remote. I tried to do this as simple as possible for the reason I explained above, if it was too complicated then the performers could not play with it. Since the Wii remote is a new instrument that you can control and manipulate sound, I had to approach it in a way that could feel right and logical from the first time. Same thing happened with the actual Wii remote for gamming; it felt right from the very beginning. The piece was actually composed on that rehearsal, the setup, the sections, and everything else was decided with the performers at that day. For example the idea of the 3 performers sitting in the audience and then coming on stage was decided because the performers were very interested in doing this. Bass and saxophone player decided what they wanted to do as a group, change of keys, tempo etc. Finally this was the idea from the very begging, how group see themselves in other groups and other individuals in a group.

I had 7 groups.

1. The bass player
2. The Sax and the Wii that controls the sax sound
3. 1 and 2 together
4. The 3 Wii players
5. The Sax Wii and the 3 other Wii players
6. 1, 2, 4 together
7. 1, 2, 4 and me the Conductor

I can go further by saying that also the audience form a group since the 3 Wii performers were sitting in the audience.
I will post soon some picture and videos form the rehearsals. I will also comment of the actual performance soon.

Crucial point (part 2)

This crucial point in my project is a nightmare. The problems of the project become more obvious at this point since all my ideas I had before are now taking shape and form.
The most important thing I have to do is to realise that my compositional outcome is not about technology but an artistic approach through technology. After the meetings with Greg and Lamberto, I have realised that what I've done so far had to do with technology not with art. Because of this situation I have to come up with something new, using the already configured technology. Actually I have to reproach my whole concept for this composition since I had dumped my earlier ideas with the audience participation and colour tracking. I have some troubles with the people that I am supposed to participate with. Is a bit hard for me because I have no idea if they will actually come to the rehearsals or even to the performance. My first degree was not here in Birmingham Conservatoire so I had to ask people to perform that I don’t know if I can trust them. I guess this is part of the collaboration project so I have to prepare for the worst and have a few back up plans just in case.
The performance is on the 6th of January 2009, and the conservatoire and the art department are closed. So I have to plan this really good in terms of equipment since I have to take out everything before the 19th of December (Friday). Also to make a poster in advance so people can came to the performance.

New compositional concept

My new concept approach has to do with how people see themselves as an individual, in relation with other, in relation with a group of people, and in relation with other groups. I am exploring this approach through sounds and physical gestures. The sound files are words that can express gestures. I am using words like, Me, You, They, Them, Is, Are in addition to that I am using two different languages, Greek and English. The performers will trigger the sound files, randomly, and then he or she will do the appropriate gesture. For example the Me sound, he/she will be showing themselves, for the You sound the performer will make a gesture showing the other performers or the audience and so on. Another level of this concept lays with in the actual setup of the performers. There is a base player who is free to do whatever he want, play or not play, fast slow, rhythmic and so on. This is an attempt to show individuality within a group. There is a saxophone player that is linked with one laptop player that holds the Wii. The sax-Wii performer is able to manipulate the sounds coming from the sax in real time. So 'They' may form another group. The other three-laptop performers will perform holding the Wii and making gestures, they are allready a group. From the audience point of view we are a big group with the conductor the leader of the group.

Crucial point (part 1)

I am now at the point, in my project, that I have to decide what sound I am going to use. How I am going to use them it was all figure out more or less at the programming sequencer procedure (and still is). This point is crucial because my decisions now will bring the composition alive or not. Bad quality and bad combination of sounds will of course produce a bad composition. In addition to that I have to consider how sound effects in Ableton Live will change the final outcome. It is very similar to cooking a Christmas turkey dinner with the staffing. If the turkey is not good quality whatever you do with the rest of the dish the results will be less than medium. From the other hand if the turkey is free range and good quality, fresh from a butcher, then is more likely to have an unforgettable Christmas dinner. There are two ways to do these tests. The first option, which I don’t have, is to test each sound at the rehearsals to see (hear) whether are appropriate for this composition. In other words buy ten turkeys from different places and put them in the oven to find out which turkey taste better so that you will buy it for the Christmas dinner. Rehearsals in my case are tricky because the performers are doing this for free so they are not so happy to do rehearsal just to figure out what type of sounds I will use. The other part that is also very related to the final outcome of the composition is the Christmas stuffing (sound effects). Ableton live have a wide range of sound effects that I can use but which of those will taste the better?
In addition to all this I have to use 8 different sound files (two for each performer) that the qualities and character of the sounds will be a pleasant mixture. Again like a four-course meal with two ingredients for each course. The reason for this decision to have 8 different sounds files, two for each performer, is because I the composition is in an 8-speaker surround set up. Each performer will have a pair of speakers with the ability to pan and choose the sound file on the fly. The only sounds that I know so far are the alto saxophone and the electric base.

My approach here would be a combination of experiments with sounds. I am aiming for faster and most efficient way to choose between the sounds. I know that I will have base and saxophone so having those two sounds as a given I will start choosing my sound around that area. I am approaching the sounds as a saxophone quartet with frequencies low middle and high. I will update this as soon as I decide what sounds I am going to use.

After the meeting at VRU 25/11/2008

Today’s meeting was very helpful. I was able to formulate my aims for this composition-project in a clearer and more directional way. As I’ve mention previously in my Blog the main purpose for this project is to involve the audience in a way that they will transform and interact with the music in a live performance. Furthermore is the idea and fact that audiences can influence the performance consciously and subconsciously. In the first section of my programming with Max/MSP I am using cameras to track the colour from the audience and therefore a consciously interaction. I’ve decided to drop out this part completely since the other two parts are quite complicated. Also another reasons for dropping out is the fact that since I was going to use cameras I had to do some tests to see how the cameras will act in a performance situation. How far away can the camera track colour. How the lights in the performance affect the colour. Am I going to use LEDs or mobile phones from the audience? All these are questions that will produce even more questions if they are answered. Even if the tests are done the final outcome, from the section one, was going to be unstable and would crucially influence negatively the other two sections. It occur to me, after the meeting, that what I was trying to do in section one with the cameras, was to track the position of the particular colour within the frame of the camera. This all could work in a much simpler manner. I will track the amount of colour (Red, Green, Blue) within the frame of the camera. I will need to do some test as well, if I am going to include this, but with this it might work in a more secure way.

The next video shows how the, abandon for now, idea of colour tracking works. It works fine with only one colour received form the camera. When are two colours, in this case two red sources, it automatically connect the lines from the outer X to the outer Y. Watch the cursors at the right video. Imagine this in a group of audience of 20 people. The incoming data would be very unstable.





Knowing that the first section is not included, I will have to do a few modification to to the second part to be completed as a compositional piece. I will add another Wii as the master Wii or in musical terms a conductor. I will conduct the other four Wiis with the master Wii. The button in the masters Wii will act as a gate for the effects on Live Ableton. I am working on this modification and I don’t know the actual outcome of this process. Hopefully I will be able to control the final musical outcome from the Master Wii . I am still working on the third section. I am waiting from the base player and the saxophone player to let me know if they can do the performance.

Three sections for the project

I have decided that I will need six people to participate in this project apart from the audience and me. Four of them will perform with laptops and Wiis and the other two will be performing traditional instrument probably on base and sax. I have found two of them so far that are willing to collaborate with me.

Project construction

I have mentioned earlier that I had to broke down my programming to three sections. Thanks to Jonathan Green that already made a few patches in Max/MSP for colour tracking (similar my section one programming) I have decided that it was better to work on the second section to see how I was doing. In this second section I have done more or less half of the work that I need to do. I have connected the Wii with Max and route different buttons to the sequencer. Using the Rewire in Max I have connected it with Ableton live which I will have all the audio effects for real-time processing. Have a look at the video.