Reflective Analysis

September 22, 2007

Click here to view the PDF


Screen Test

September 16, 2007

The video below shows the testing of the piece in situ. Using one projector to stretch across two back projection screens.

The sound is picked up from the cameras mic so not great, also features some added sound from the builders in the next room.

Final setup test

September 11, 2007


Video no.3

September 7, 2007

I wanted to integrate a more ‘ ambient’ type of interaction, one that isn’t just determined by the users specific movements. I thought about using another camera to input aesthetic data from the user ie colour of clothes or to incorporate their image in to the visual composition. This would mean even more technicalities, probably the user of another computer.

The other idea I had was to use sound, input the ambient sounds from the room and use it in the sound composition. Once I worked out how to do this in max, I used a delay line, to give the sound ‘some atmosphere’, but I also discovered I could take a reading from the input level, which means I could send this data to Isadora. A short example is below:

Screen test photo

August 23, 2007


Practical Project Experimentation

August 13, 2007

Below is the next video documenting my progress.

I can now use two symbols to control the visuals and create something that vaguely resembles music.

I am still using Reactivision, Isadora and Max/Msp, but now (after much brain ache) I have managed to make the audio playback engine in Max. So now I don’t need to split the OSC signals between Isadora and Reaktor, which makes the system a little faster. It also means that I now have more control over the audio.

I have made an engine in max that takes the input signals and using some equations, divides the y axis in to octaves and the x axis in to notes of a scale. I thought that this would be the most intuitive way of ‘playing’ the instrument.

The x and y positions of the fiducials are sent to Isadora which then puts an oval shape at that position. I like the kind of ‘Orb’ shape, i think it works well as the hand controllers I have prototyped are big and round too. I have used an alpha mask and motion blur, to make them look a bit more interesting.

I made engine in max that divides the screen in to four hotspots, when a symbol lands in the spot it triggers a sample and changes the picture within the orb.

The problem at the moment as you will hear, is that there is quite a delay and latency issue. The other is that the messages come through so fast from Reactivision, that the midi is triggered several times a second, which sounds as if some one iss constantnly hitting the piano key. At the moment not sure how to solve this.

The next stage will be to either solve this or find an alternative, refine the interactions and visual and sound content. Then test with a projector at uni.

Any comments or feedback would be much appreciated.

Below: Max Patch so far

Max Patch so far

Still Searching

August 9, 2007

After much help and guidance from my project tutors I have decided to put the practical project to one side and concentrate on defining a question for the written piece.

I learned that two of the questions I had in mind that interested me most, relate to great debates that have been on going through history within art.

The first. Do our minds fuse the elements of music and imagery in the same way or do we each create our own experiences?

Apparently this relates to a 20th Century debate within art.
Cultural universals vs Cultural Relativism.
As far as I understand it is related to the nature vs nuture debate, and the question is this: Are there certain images or sounds that universally appeal to us as humans or is it our cultural upbringings and surroundings that determine what affects us.
This relates to work I have already looked at by artists like kandinsky and klee at the bauhaus and kupka and mattis who were among the first to explore ‘visual music’. They tried to represent classical music using paint on a canvas, in turn exploring whether their own images could represent music to others.
This is something that really interests me and I think could also tie in with the second question I had, which is:
Does the combination of music and imagery allow more or less room for our minds to include our own imaginations.

Perhaps each form on their own allows us to create our own experience, each having an individual engagement with the piece.
Does the combination prescribe more of the experience and even the setting. If so does this immerse us more in the experience and enable us to imagine we are there.
Bringing back the question – Which allows more room for the imagination?

This relates to another debate about the ambiguity in art, and is often said that the more ambiguous pieces are the ones that effect us on a more emotional level. Is this because there is less prescribed to us, so we can bring more of our own imaginations and experiences to them. Often it only takes basic colours or shapes to strike meaning.
As my tutor pointed out, music is the most ambiguous of the art forms.

I am trying to look at all the different areas that are related to this subject and what interests me. Then as it was suggested, lay them all out in front of me and go with my gut instinct when choosing a specific route.


August 9, 2007

I have been reading Music, The Brain and Ecstasy, by Robert Jourdain.
‘using discoveries in science, psychology, music theory, paleontology and philosophy… the author explains why music speaks to us in ways words cannot, and why we form such powerful connections to it.’

I have also visited two exhibitions.
Howard Hodgkin’s prints at the Victoria art gallery, Bath.
This was interesting as the exhibition featured two different versions of the same print using different colours. The different use of colours completely changed the experience of viewing the image.
Mourning - Howard Hogkin Bleeding - Howard Hogkins

The second was Eileen Quinlan & Cheyney Thompson – TBA at the Arnolfini Bristol.
This exhibition featured some nice abstract photography.

The images were almost collages, bold in colour, and were fragmented almost like cracked glass. Each very abstract, containing un- recognisable segments. I felt that although they were un-recognisable they suggested different settings or landscapes. For me they worked on two levels, firstly they grabbed my attention and suggested something instantly just by use of colour. Secondly by exploring the fragments you could see images within images, simple lines or colour variations suggested different things to me.

I liked these pictures and each made me feel something different, I think that this may have been based on the use of colour and the fact that I could almost see images that reminded me of things I have experienced and seen elsewhere.
I thought that these were a good example of the ambiguity allowing more room for viewer engagement, and it would be interesting to know what other people ‘saw in the images’.

Dissertation thoughts

August 1, 2007


At this stage I am still unable to formulate a specific question.
I know the general area I want to explore.
This being the combination of music and imagery and it’s effect on a person.

Each form can trigger direct responses from our senses and minds and the combination of both delivers something that neither can on their own.

Does the combination trigger different responses in our brain.

‘Visual Music is an art form with a primary objective of intentionally activating the mind’s ability to fuse visuals and music into a “synaesthetic experience”.’

Is this an interaction in itself. Do our minds fuse the elements in the same way or do we each create our own experiences.

Does the combination of music and imagery allow more or less room for our minds to include our own imaginations.

I would like to consider two of what I believe to be examples of music and imagery having the strongest impact. These being film and television scores and VJ culture.
Within these examples, one form is supportive of the other. Although I would like to explore how each change an audience’s perception of the other.

Something else I have come across in my research is the combination of music and imagery as a form of psychotherapy, used to bring about a therapeutic change.
One organisations main approach is to play patients certain types of music in a relaxed state and ask them to describe any images that they experience in their mind.

I found this particularly interesting as this suggests that everyone has the ability to create images when listening to music. I would like to look in to this further.

Also I would like to look into possibilities for the future. Will there be a time when we will be able to see images created by a persons mind, are the images we see in our heads actual translatable images.


July 24, 2007

This short video is to show me testing my system so far.

I have networked two computers so they can pass OSC signals between each other.
One inputs from a webcam using ‘Reactivision’ (this is what is on the left hand side of the video). I have then used Max/MSP to send this data out as OSC signals, which then get sent to ‘Isadora’ which is what is creating the visuals and to the other computer running ‘Reaktor 5’ which is what is creating the sound you hear.

The first two clips are a test of a simple type of ‘colour organ’. The Third clip is using a desk lamp to control the amount of light to the webcam. I have set Isadora to analyse the amount of light inputted then to output a value to control the sound and the imagery.