music and visual arts communities have been working for the past
two decades on creating hybrid physical digital
music compositions, visual art works and installations. However,
human experience of the world is embodied and multi-modal. Arts
can best investigate and depict our current hybrid reality through
works that include movement. The
main drawback for such works has been the difficulty of digitizing
and analyzing movement.
Sampling, digital analysis and digital
synthesis of sound and image are already fairly
advanced. Complex processing of sound and image can be realized
with good accuracy in real time.
User-friendly software for interactive analysis and
synthesis of sound and image are readily available.
Some of those
Max/MSP/Jitter) allow for the simultaneous processing of both
image and sound. The tools for coherent, digital, sound-image interactive
compositions are available and increasingly used in all types
settings--from installations in galleries, to jam sessions in
music bars, symphonic concerts, educational
settings and biodesign experiments.
and analysis of motion has until recently relied on 2D video
capture that provides limited information on the spatial
and kinesthetic aspects of movement and has limited capabilities
in real time use.
of motion capture technology in the past two decades has opened
up many possibilities for
advanced digitization and analysis of movement.
The motione project is developing techniques for advanced real
time capture and analysis of movement. These motion analysis
systems are being integrated with real time systems for image
analysis and synthesis to create unified platforms for the
creation of multimodal physical-digital works.