The Artifacts of Motion
One thing that television and the movies have in
common is that both portray motion by capturing and displaying
a series of still pictures, albeit in very different ways. The
capture and display of moving images in these ways can generate
artifacts, which are visual effects created by the capture/display
process that were not in the original scene.
Some of these artifacts are reversible and may
be removed by artful processing, while some cannot be removed
at all. Some artifacts are less obtrusive than the effects of
their removal would be. It is possible, for example, to remove
the visible scan lines from a television screen by placing a ground
glass filter over it, but this will also remove enough detail
from the picture that we would rather live with the scan lines.
FREQUENCY RECONSTRUCTION
Temporal sampling artifacts may be generated in
both film and video capture. We know that sample theory dictates
that we need at least two samples per second of any frequency
that we capture, in order to perfectly reconstruct that frequency.
In digital audio, we sample at a rate faster than twice the highest
frequency of interest, and filter the input signal so that no
frequency above the highest frequency of interest enters the sampler.
For example, we sample at 48 kHz, while limiting
the highest input frequency to 20 kHz. We know that filters have
finite slopes, and we have therefore chosen a sampling frequency
sufficiently high that we may ensure flat response to 20 kHz;
the frequencies within the filter's attenuation slope are well-below
half the sampling frequency. Sampling at exactly half the sampling
rate, in this case 24 kHz, is called critical sampling and is
avoided in digital audio.
Sampling frequencies greater than half the sampling
rate result in the replacement of these frequencies with aliases,
which are frequencies that were not actually present in the original
sampled material. Once aliases have been created, they may not
be removed from the reconstructed material, as they are within
the band of frequencies of interest.
THE SAMPLE THEORY
Any kind of sampling is subject to the sample theory,
and this includes moving pictures. Motion picture photography
may be used to illustrate the temporal aliasing phenomenon. A
film camera that shoots 24 frames per second has a temporal sampling
rate of 24 Hz. This means that any temporal frequency this camera
photographs that is above 12 Hz will generate a temporal alias.
Because Westerns, America's own epic dramas, have
been so prevalent in our cinematic and television-viewing experience,
backward-turning wagon wheels are one of the most-frequently seen
temporal aliases. The spokes in a wagon wheel form a repetitive
pattern when sampled - that is, each snapshot of the wheel turning
"freezes" the spokes at some given position. The frequency
at which spokes pass a given point depends on how many spokes
the wheel has and how fast it is turning.
WAGONS, HO!
If a wheel has 24 spokes, for example, the spokes
are spaced at intervals of 360/24 = 15 degrees. This wheel may
turn at a speed such that at the instant each spoke in turn reaches
"zero degrees" (or straight up), a photograph is taken.
In this case, the wheel is turning 15 degrees each 1/24 second,
making one complete revolution each second.
For the moment, we will ignore the aperture effect
and pretend that each snapshot is taken instantaneously, knowing
that in reality the camera's shutter must remain open for a finite
period of time, causing higher-speed motion to blur. Because all
the spokes look alike, we cannot tell which particular spoke is
passing zero degrees, or any of the other 23 points, in any given
snapshot.
In fact, we cannot tell that anything is happening
at all, because the wheel appears to be stationary. In this case,
only one sample is being taken each second, and sample theory
tells us that we cannot accurately reconstruct this frequency.
We find this to be true, because the 24 Hz spoke travel frequency
has been replaced by the alias frequency of zero Hz.
If the wheel is sped up to two revolutions per
second (with spokes still miraculously in-sync with the camera
shutter), a snapshot will be made as every other spoke passes
zero degrees, and the wheel will again appear to be stationary.
Another way to say this is that every integer multiple of 24 Hz
will generate a zero Hz alias.
TALKIN' ÎBOUT A REVOLUTION
If the wagon wheel travels at one-half revolution
per second, the frequency of spokes passing a given point is 12
Hz, or exactly half the sample frequency. As mentioned earlier,
this is called critical sampling. The visual result of this will
be that a given spoke will be at zero degrees for a given snapshot,
while the spoke immediately ahead of it in the direction of travel
will be at 15 degrees and the spoke immediately following it will
be at 345 degrees.
At the next snapshot, the first (or "zero")
spoke will be at 7.5 degrees; the spoke immediately ahead of it
will be at 22.5 degrees; and the spoke immediately following it
will be at 352.5 degrees. At one-quarter revolution per second,
the spoke repetition rate will be 6 Hz, and the spokes in a given
snapshot will be advanced by 3.75 degrees in the next snapshot.
The visual effect for spoke frequencies between
zero and 12 Hz (the wheel is turning between zero and 1/2 revolution
per second) is that the wagon wheel will appear to turn normally.
If the wheel turns at three-quarter revolution per second, the
spoke frequency becomes 18 Hz, and the "zero spoke"
in our reference snapshot will appear advanced by 11.25 degrees
in the next snapshot, while the spoke immediately behind it will
be at 360 ö [15-11.25] = 356.25 degrees, or 3.75 degrees before
the zero degree point.
The net result of this is that when the 18 Hz spoke
frequency is sampled, it is replaced by an alias of 6 Hz. Further,
the zero spoke will have advanced by more than half the spacing
between two spokes, while the spoke behind it will be 3.75 degrees
behind zero degrees.
To the eye, it will appear that the zero spoke
has moved not forward but backward by an amount equal to a spoke
travel frequency of 6 Hz. Visually, the wagon wheel appears to
turn normally as its speed is increased between 0 Hz and 12 Hz
- and as its speed is increased above 12 Hz, the real spoke frequencies
are replaced by aliases, and the wheel appears to turn backward
rapidly at just above 12 Hz. And as the wheel speeds up (progressively
more slowly until, as a spoke frequency of 24 Hz is reached),
the wheel appears to have stopped.
SPINNIN' WHEEL
Thus, the pattern between 12 Hz and 24 Hz is visually
a mirror image of the pattern between 0 and 12 Hz. As the spoke
frequency is increased above 24 Hz, the pattern repeats itself,
but the wagon wheel will appear to be traveling at half its real
speed, until at 48 Hz, it will appear stationary again. This pattern
will repeat ad infinitum (or until the bearings burn out),
with the wheel appearing stationary at every integer multiple
of 24 Hz.
All the visual representations above a spoke frequency
of 12 Hz are invalid, as the real frequencies have all been replaced
by aliases, just as the sample theory tells us they will be. Further,
none of these aliases may be removed, because they all fall within
the 0-12 Hz frequency band.
The only way to prevent this aliasing is to filter
out all the frequencies above 12 Hz before sampling, analogously
to the way digital audio is sampled. However, this cannot be accomplished
in film photography, as the requisite sharp-cutoff optical filters
cannot be realized. We must, therefore, live with the aliases
generated by optical sampling, and we do live with them on a daily
basis - both in cinema and television.
When 24-frame film is transferred to 60 Hz video,
another form of artifact is generated that we know as 3/2 judder.
This column has discussed 3/2 pulldown in the past, but the essence
of the process is that in order to transfer 24-frame material
to 60-field interlaced video, 60 fields must be made from 48 discrete
fields; that is, for every four film frames, not eight but 10
fields must be generated.
This is done by repeating a field after each four
fields. For 60-field interlaced video (30 frames per second),
one pair of fields is made from each film frame, and one of the
fields generated by every second film frame is repeated, so that
four film frames comprise five interlaced video frames. An analogous
process is used to generate 60 Hz progressive video, except that
full video frames replace fields.
FRAME REPITION
When 3/2 material is displayed, the repetition
of film frames (as fields or progressive video frames) produces
a distortion of what is sometimes called the optical flow axis,
which interferes with eye's tracking of motion. As an object moves
through the field of vision, the eye attempts to track it. Repetition
of frames means that as the object moves, instead of each picture
appearing at a unique place on the eye's retina, identical versions
of each picture appear at different places on the retina - three
identical pictures followed by two identical pictures, followed
by three identical pictures, and so on.
This interferes with the eye's ability track the
object, causing it to appear to "stutter" as it moves
through the field of vision instead of moving smoothly as the
eye knows it should. As one in five fields of progressive frames
is repeated, the stutter frequency is 60 x 0.2 = 12 Hz.
When the camera pans to track a moving object,
the background is not tracked, and the background judders. Judder
is readily visible during pans, making cinematographers keenly
aware of panning speeds. Unlike aliasing, this artifact of 3/2
pulldown is reversible.
Next month, we will explore these temporal artifacts
in greater depth and look at some other artifacts that may be
seen when film and video are displayed.
