|
James Ellis post of
Fri, 29 Nov 2002 16:46:34 -0500 to the
iota-centre email-list.
Color/Sound one to one, 10 fingers, ten toes, and a bee
I
hope I'm not being too redundant with what
others have posted here about "one to
one" correlation of sound to image,
but I (and many I know) have given this
topic much thought and discussed it
endlessly. There are a few conclusions that
we have come up with, both in practice and
theory on the creation of real-time music
related visuals. Some I don't agree with,
some the panel is still open, and on others
I take them for as much temporary truth as
I can assign to them. I can only truly
discuss my own temporary beliefs. Perhaps
someone might find these insightful, or add
new dimension to these... both I hope. Or
who knows, perhaps even kindly discredit
them.
Odorama vs. Synesthesia :
There
was a movie created in the late 1950's
called "Scent of a Mystery". This
film (I've never seen or smelt it
personally) had scent pumped through pipes
into the theater at certain key moments of
the movie. This process was known as
"Smell-O-Vision". I'm not quite
sure why it didn't work out. I imagine it
was quite expensive, and that there was no
real means to negate previous smells enough
to cleanse the olfactory pallet.
John
Waters released a film later called
"Polyester" (in Odorama) that
came with scratch and sniff cards that had
the smell of shit, sperm, bleach, etc.. I
believe that the audience would take
scratch/sniff cues from the screen at
specific moments in the film. A cute trick
to add gross scent to heighten the overall
shock value of the film.
I
certainly think this area of sensory
immersion has tremendous future
possibilities (especially with new scent
synthesis technologies), however I'm going
to primarily use them as an example of why
"one to one" correspondence of
visual and sound is a limited approach to a
larger overall experience. I believe that
the "See and hear at the exact same
time" falls into the same category as
"see and smell at exactly the same
time", and is about as effective (in
the case of the color piano) as John Waters
scratch and sniff movie. It is a great
starting point, but it must be more fluid I
say.
The
idea of hitting a keyboard frequency and
creating a specific frequency of light (or
Chakra frequency) simultaneously is
certainly a worthwhile endeavor to say the
least. However I believe that aside from
the goal of purely seeking to induce trance
by the rigid synchronization of these two
elements through minimizing their scope and
interrelation, there might not be as much
fertile ground as some might think here. I
admit that I could be wrong on this, and if
anyone can truly show me a working example
of this (without me spending my life
savings flying to shows all over the world)
I'll gladly "eat my hat". I
believe it's an interesting experiment in
scientific/mystical cross referencing, but
I fear it might be nothing more than an
"Om chant"... not to dismiss the
power of an "Om chant", I just
believe that there are many different ways
of experiencing/communicating with
existence, and I don't listen to "Om
chants" on my stereo or wish to do
them all day long. I am certainly for the
inducing of trance, and such a formula may
very well do that, as well as helping to
establish an initial "starting
point/vocabulary" for the
artist/audience, but where does it go from
there. Some might say it's infinite, I
would disagree, I believe it's too
limiting.
C/Yellow E/Blue G/Pink
Limits of
course can be a great frame work in which
extraordinary creative pieces can grow, but
"bong color" simultaneously I
believe has very little to do with how we
take in sensory information in general...
not to mention that I believe almost any
similar system not based on hertz would
have a similar effect.
I'd
like to share a seemingly silly fictitious
example perception, and why I believe
"one to one" is not the be all...
this leads into tangents which I can't
ignore, but will hopefully yield some good
fruit.
_______________________________________________________
When I say... walk outside, perhaps
I see what's around me as a whole, but my
mind has not really found anything to truly
focus on yet, so I'm kind of lost in my
thoughts about how I should be cleaning or
house. My eyes reflex mechanism has me
squinting because the sun is bright, so my
thoughts almost immediately flash that
"gee it's bright out here"... and
then I forget about it. Suddenly there is a
sound behind me, I turn to where the sound
is emanating from... but it's not there. I
know this sound... "airplane", I
then visually scan the sky and see an
airplane. I am then able to connect the
sound to the plane even though they are
apparently emanating from different
locations, due to the delay inherent in
traveling sound. I'm now staring up at this
plane, bored because I've figured out what
it was, when a bee passes in at some
distance from of my face. Through life's
training, I immediately get defensive, but
then realize that the bee is no threat due
to it's distance, so I relax. At first I
can't hear the bee, but then it approaches
closer, zips out of my scope of vision and
past my ear (I can hear it and I get a bit
defensive momentarily) and I then turn my
body to visually attempt to find it's
flight path. I see a bee that looks like
the same one over near a beautiful flower.
I determine that this is indeed the same
bee, but now my primary focus is the flower
with the bee now secondary. I decide to
buck my minor defense mechanism, and
approach the flower, the bee leaves and I
touch the flower... so nice and soft. I
then sniff the sucker, and it smells
absolutely horrid to me, but I know my
girlfriend, and I bet she would like it's
smell... but she also may be allergic to
it's pollen.
______________________________________________________________
So, what we have here is a very simple scenario which encompass:
predetermined mind state:
"Should be cleaning my house".
Lost in thoughts, almost oblivious to what
I have seen so many times before in my
yard. In fact, all sensory stimuli is
basically unprocessed, and only considered
on a immediate physical need... such as
walking, personal safety. In some of my
performances/screenings I feel that the
audience may be dwelling too extensively on
preconceptions of "what has been
experienced before" or "how they
should be cleaning their house". Part
of my job is too help them out of this
trapped perspective and into my world... or
attempt to fathom their mindstate, and give
them an airplane or bee to focus on...
before I push the envelope.
controlled physical activity that has now become almost intuitive:
"Me walking". Trained
coordination and rhythm are not thought of
per se, but they are under my control, and
allow me to explore my surroundings. These
inherent rhythms and physical activities
can not be understated in aiding the
artist/audience in their involvement and
mindstate when encountering art. It is part
of the physical automated song that leads
us to new experience. Travel, Dance,
Driving, Reading, Breathing, and many
others... all have physical rhythms in
which we can use to "ground" (or
otherwise) ourselves or the audience in our
works.
inborn reflex eliciting a passive thought:
"eyes squinting because the
sun is out.... gee it's bright out
here". Proof that sensory stimuli is
capable of evoking a response on a physical
level, which in turn has a direct parallel
in immediate thought. Manipulating thought
attached to reflex is of personal interest
to me, but thought doesn't necessarily have
to be so direct. Could a strobe light then
be called "Reflex art" ?
shifts in focus (and nonfocus) of visual stimuli:
"The airplane". A pervasive familiar sound that grabs attention and beckons to be attached to, (and mentally confirmed by) visual attention. Here is a case where "one to one" correspondence does not apply in real life (although art doesn't really have to have real life as it's reference point). The sound was heard first, and then seen.
apparent discrepancies in sound/visual
placement and forced integration of
sound/visual stimuli through past
experiences and judgment of distance and
speed:
Even though the airplane was
heard first, and then this lead to the
bodies movement to visual find it, once it
has been found, the sound is delayed from
the apparent location of the visual. The
mind is hard wired to accommodate for minor
delays in sound, this aids in the immediate
judgment of distance. However when such
great distance is accompanied by such
speed, it then takes a bit of mental effort
to attach the two. Foley artists know very
well the principles of this interplay and
use it to the overall benefit of simulating
visual-speed/sound-delay, and even sweeten
it with such things as Dopler. This to me
as a visualist, seems like a nice starting
point to be expanded upon, and then twisted
around to defy all it's inherent rules.
Again, no "one to one"
correspondence.
boredom incurred by the solving the of the visual affirmation of the creator of the sound:
"Look it was the airplane that
made that sound, I'm bored, move on."
A certain amount of mystery, will always
help keep one intrigued. "What's
making that sound?" In the strictest
sense, "one to one"
correspondence does not have this in its
approach to audio visual interplay. You
always know that the sound and the visual
happen at the same time.
The almost complete mental focus on an
attractive pervasive visual stimulus that
is devoid of sound:
"Again the
sonic mystery has been ended by the
airplane, so the mind turns off its visual
thought focus, until something jumps at it
and demands its attention... the bee".
The relatively fast movement (compared to
its surroundings) and its immediate
proximity are probably the initial key
factors in the minds engagement on this
visual stimulus. This is then reinforced by
the issue of personal safety.
Once
again, depth is a key factor in creating
mental focus. The relationship of the angle
of the two eyes creates numerous
convergence points (to choose from) in
which they tend to focus best at the point
of interest. Once a point of interest has
been established, most everything else
outside of that area, tends to blur a bit
as it falls away. I have seen this effect
so nicely used in films. Can the same be
said for ears? Waves enter the ears, but
how do we mentally interpret them, and what
happens to our focus of the larger
soundscape when we focus on a particular
area of space? An experiment is in the
making here for me, perhaps there is a one
to one of my own... but back on track. As a
primarily non-representational artist, I
certainly wish to make more use of this
idea of convergence point... and of course,
turn it on it's ear, and ignore it whenever
I choose. Also this is a case of "no
sound, but visual".
Trained defensive mechanism overridden
by almost instantaneous evaluation of
personal safety:
"A bee... will
it sting me... nahhhh, it's far away...
your safe, keep and eye out." There is
such an overwhelming amount of sensory
information that is inborn or that we have
been trained to interpret and evaluate in a
specific way, and yet it often conflicts.
Certain vague conclusions can be drawn on
how to manipulate this for artistic means.
You can say that a "bee" when it
is of threat, could be Orange, but when
there is no immediate threat, it's Blue. It
depends on the circumstance. The same can
be said of note value, timbre, etc..
Circumstance dictates the emotional color.
Perhaps there is an overlap, but when does
the western scale of pure visual color have
the same dynamic range as that of music? I
haven't seen it. The depth of variation
through combination of notes of music, in
my humble opinion, it far far far greater
than blank sheets of color performed in
succession. Perhaps it's just me, but I
don't see it. This thought process of the
bee (again in my opinion) could be
presented much more affective through sound
by itself, where as a pure color sheets by
themselves could be affective... but not
nearly so. Together, would the two really
add anything to each other? I doubt it.
Perhaps some of you can change my mind by
giving me examples (performed). I really
must say, that the notion of the
color-piano/organ is an interesting one,
but it seems to cut its visual options too
small. Too much adherence to a formula...
and not nearly exciting enough.
the visual stimulus being replaced by auditory stimulus from the same source:
"The bee was seen, then
disappeared from site, but was heard".
Another reason why "one to one"
is not enough. Shifting of sensory input
for the same stimuli.
conclusions drawn based on recent sensory information:
"Can't hear the bee, can's see
the bee, it was headed in that direction...
ahhh that must be the bee". Here the
observer gets to come to some conclusion
(based on various sensory information
obtained recently) to help fill in sensory
gaps with thought and reference. In the
hollywood movies, the audience sometimes is
allowed to come to some conclusions before
the actor is. Neat trick. Again a bit of a
mental game, that once made aware of our
constant use of it, can be manipulated
creatively. Magicians love to use this for
misdirection, why should we not use it as
illusionists and sonic manipulators? In
"one to one" there is not nearly
as much room for misdirection and
conclusion.
refocusing of point of interest:
"The bee is cool, but the flower is cooler dude". Enough said.
visual gratification in conflict with scent gratification:
"Da flower sure is purdy, but
it smells terra-bull!" A pretty sight
is not always a pretty smell. A pretty
visual, is not always a pretty sound. Check
out the Oscilloscope. You could say a
"color combinations of the light piano
adequately convey such attributes as
dissonance"... perhaps. Again seems
too limiting in it's emotional scope, even
if it adequately can contribute on some
form of "frequency relation" to
the sound.
evaluation of experience of another's
sensory gratification as well as inborn
physical aversion:
"This flower
smells terra-bull ta me... but I bet my gal
Bessie Sue woulda love it's smell... if'n
it don't make her sneeze." We all have
different taste in music, art, food... and
even different physical reactions. It's
safe to say that certain things are
poisonous to just about all of us, and that
candy is sweet. How about aversion to
hearing screams of agony... is that trained
or inborn? What about that twisted fuck who
enjoys screams of agony because he likes to
hurt other life? Or what if a someone not
mentally disturbed just found screams of
agony to beautiful?
_________________________________________________________
Now that I've written all that... there is more...
"no no NOOOOOOOOOOOOOOOOOOOO!!!!!!"
yes it's true.
You see, I think
to sum this part up:
"One to One Correspondence" (in general) does not allow for enough
variation between triggered sources. It can
work (and does) just
fine. I think I personally prefer taking it
to the next step in composition... counter
rhythms. I also personally need more than
just colored lights... or I'll just go see
a great light show... but even that is not
strictly "one to one" and is all
the better for it. Primarily performed
separately. [ AuzGnosis's emphasis ]
On the subject of Synesthesia:
it
seems to occur when the mind is attempting
to interpret something that is beyond it's
normal scope of separate sensory
interpretation... so the senses blend... to
help give a deeper perspective, much like
dream or hallucination. To see color and
hear sound, pretty cool, but I don't think
it strictly adheres to any given formula.
My personal research says "No".
To create such a formula based on Hertz and
apply it to a piano as sheets of color, is
a cool creative exercise, but that's all it
comes across to me as... no golden answer
there... perhaps one small clue. To do so
is in my opinion to ultimately create the
scratch and sniff card from the film
"Polyester"... just more
abstracted.
This
idea of going beyond "one to one"
also leads into the notion of "Ten
fingers". The argument is that there
are not enough fingers to manipulate
real-time animation parameters. This can be
true in some cases. Obvious solution number
one... "foot triggers". Ok ok,
that will only get you so far. The solution
we found at Derivative for Touch use in the
Rush show, was various short sequence parts
that can be triggered, and then different
elements can be overridden live when you
choose to. When you are done playing that
element live, you return to normal
sequence. You can also have entire
visualist "Band". Numerous
persons contribute various elements of the
visual composition which can then be
composited together by the mixer/conductor.
Say one person can play "visual
atmospherics" for the first part of
the song, and then progressively change
into something more rhythmic, and then
something that is almost dancing and
demanding attention. In this manner you
could compose songs with musicians that
left plenty of sonic and visual space for
each other to fill, devoid of cluttering
each other. The visuals might play the 2
and 4 beats, and the musicians play the 1
and 3. Then you trade. Then the visualist
go off into chaotic rhythms, while the
musicians are orderly. The small light
flashes, and then a delayed beat. The
slightly larger light flashes again, less
delayed beat... it's getting closer.
wellll I'll stop now
As long winded as ever
Blah Blah Blah
I could be wrong
best
James Ellis vimeo.com/album/1877924
|
