Combining Performers with
Soundtracks
some personal experiences
(pdf version)
Why combine live and pre-recorded?
- skilled live performers provide a level of
musicality and often virtuosity that is entirely different
from what is possible with electroacoustic means, and
therefore can be regarded as complementary
- the live performer provides not merely a visual
focus for the audience, but can act as a "persona" in their
interaction with the soundtrack (some relationships are
suggested below); this can lead to a sense of theatricality
and drama; however, since the performer draws visual focus,
I tend to keep the soundtrack very strong and present, not
ambient (I prefer a soundtrack that is interesting to listen
to on its own, yet leaves enough "space" for the live
performer)
- an interesting timbral dialogue can be
established between the performer and soundtrack,
particularly if the sound sources used for the soundtrack
are derived from the performer, or other related sounds (in
the early days with electronic synthesis, it was very
difficult to match the richness of the live performer's
sound with the available synthesis techniques, though
digital FM synthesis was an
improvement ; this has become much easier with sample-based
work)
Some possible relationships
between the live performer(s) and the soundtrack:
- "figure in a landscape": the soundtrack provides
an enhanced sonic environment, particularly when
instrumental or vocal materials used in producing the
soundtrack are those provided by the performer, and
electroacoustically enhanced (examples are East
Wind, Nightwatch, Inside, Tongues
of Angels, Bamboo, Silk and Stone,
and The
Way of the Spirit where the acoustic
virtuosity of the performer is pitted against the enhanced
instrumental sounds on the soundtrack)
- timbral extension: the vocal or instrument's
pitches and timbre extend and complement the often stretched
sounds on the soundtrack; sometimes the source sounds are
entirely different from the live component and the performer
blends with the soundtrack or at times emerges as a
foreground voice (e.g. Dominion, Steam
which are based on environmental sounds) whereas in From
The Unseen World, the solo piano attempts to
blend with the convolved and stretched sounds of the
instrument in a form of spectral composition (example page
from Steam)
- text enhancement: when there is significant text
material on the soundtrack (e.g. Song of Songs, Wings
of Fire, Androgyne, Mon Amour,
Beauty
and the Beast) the live instrumentalist can
mirror and elaborate on the text; this can involve deriving
the melodic line from the inflection patterns of the voice
(particularly when the voice is time-stretched) and
elaborating on the pitches musically (example
page from Song of Songs)
- when the performer is a vocalist, then a
spectrally based approach is useful, i.e. the vocal pitches
can be derived from, for instance, the resonated and
stretched sounds on the soundtrack, giving them a pitch
reference (e.g. Powers of Two, Enigma,
Thou
and I, Orpheus Ascending,
Twin
Souls); in most cases, I have found that a
unique "mode" of pitches has emerged from the soundtrack,
presumably related to the harmonics that have become
prominent via processing (example page from Beyond,
a section of Powers of
Two)
- algorithmic extensions and counterpoint to the
soundtrack: in my early works, I used the POD compositional system with its
stochastic use of Poisson distributed events within
designed tendency masks to generate the live part (suitably
adapted to Western rhythmic notation); with Nautilus,
Trigon, Aerial
and Sonic
Landscape
No. 4, both the digital soundtrack and the
live part resulted from the same algorithms; with East
Wind and Nightwatch, the
soundtrack was created with analog processing of the
instrumental sounds, and most of the live part created
algorithmically (during a period when digital synthesis was
unavailable); example page from Nightwatch
starting at 5:20
Practical considerations:
- I always insist on the live performer being miked
which allows the person at the mixing console to balance the
levels between live and the soundtrack; also this feeds the
live performer's sound to the same speakers as a spatial
enhancement (usually only front and side speakers, not rear
ones)
- synchronization: the score can include ideal
timing marks for key moments of synchronization, with a fair
degree of freedom in between (performers always worry about
"getting behind" the fixed soundtrack at first and use a
stopwatch for security, but with greater familiarity, they
end up just treating the soundtrack as a dependable
accompanist, knowing aurally what's coming next); still,
it's best to leave a bit of "flex" in the live score and the
timing of the soundtrack (in some cases I even leave the
exact rhythm to the performer, example from Wings
of Fire)
- some works designed for a dedicated soloist (e.g.
Song
of Songs, Androgyne, Mon Amour)
have very tight synchronization throughout, and key bits of
text or other sound events are noted in the score; this
works best for someone who will perform the piece many times
and really make it their own (example page from Song of
Songs, Androgyne, Mon Amour)
- singers worry about their pitch references, but
when the vocal part is derived from pitches/harmonics found
on the soundtrack (as described above), singing in tune
becomes easier
- live performers may require foldback speakers
either to hear themselves or to hear the soundtrack better
(a separate mix can be fed to a foldback pair of speakers)
- when recording a mixed piece for demonstration or
publication, I never use a live pickup; instead, I arrange a
studio recording where the performer can hear the soundtrack
over headphones, and then the mix happens later when all the
edited sections of the live part have been selected and
arranged in the proper synch; I tend to favour a strong
level for the pre-recorded material in the final mix
home
