Question 2: What Do Lines Tell?

Yuri Tsivian

A diagrammatic representation of the plot should offer a succession of peaks and valleys, each peak a little higher that the last and each valley above the level of the one before it. The highest peak represents the climax, and from there the diagram slants sharply toward the bottom. (Epes Winthrop Sargent, Technique of the Photoplay (New York: The Moving Picture World, 1916), p. 45)

Running lines, hand-drawn or conjured up, have been part of thinking about cinema for more than one hundred years. As described by a screen writer and screen-writing instructor in 1916, the diagrammatic landscape sketched in the epigraph above is similar to ones a number of other film analysts and filmmakers have come up with in order to visualize the internal dynamics of a movie. There, as here, the movie is represented as a time series and its dynamics, as a curve.

Since diagrams like this were not based on statistical data and showed no numerical values attached to this or that point on the curve they are not time-series analyses in the strict sense, statistics warns us. This does not put old time series diagrams out of court, if only as pieces of historical evidence. To distinguish them from time series analyses proper, I will call non-numeric time graphs of the kind time-series models.

We too followed this kind of model when we launched the Cinemetrics site, peaks and valleys is what interested us in the first place; we even reversed the usual Y-axis of shot lengths upside down for our graphs to dovetail with the time-honored tradition to represent the climax as an apex, not as an abyss:

Figure 1a

Figure 1b

Figure 1: Cinemetrics-generated time-series diagrams for The Skin Game (above) and Easy Virtue measured in simple and advanced modes by Charles O’Brien, the trendline flexibility set at degree 6. Note the inverted shot length scale (Y-axis) for the bouts of faster cutting to be represented as peaks, not valleys.

We hardly suspected at the time that an alternative way of visual display could (and would) prove useful for analyzing editing: the non-time-series statistics (I looked for, but could not find, a non-negative way of describing it; would “static statistics” sound like stuttering?). What Barry Salt, Nick Redfern (and James Cutting and Mike Baxter more recently) have been doing with familiar film titles looked as fascinating and mysterious to us as when a magician comes up to you and produces a “boxplot” or a “violin plot” from your own pocket, as it were:

Figure 2: Comparative boxplots and violin plots, on a log scale, for Easy Virtue and The Skin Game as they appear on Mike Baxter’s (2012) Figure 6.4, see http://www.mikemetrics.com/#/cinemetrics-data-analysis/4569975605

I would not find it hard to explain a time-series graph to the man in the street, were it in Chicago where I teach or in Latvia where Cinemetrics was born, for the metaphors I’d need to bring the message home derive from our common terrestrial environment, familiar to all: peaks and valleys; and even if some Latvians or Chicagoans (two flatlands, each beautiful in its own way) were unable to grasp the alpine metaphor, I could easily resort to the metaphor of waves. But when confronted with objects as shown on figure 2 the man in the street might call an ufologist for help.

The two graphs on Figure 1 are time-series analyses of Easy Virtue (1928) and The Skin Game (1931) by Alfred Hitchcock; the paired graphs on Figure 2 are the same two films presented through comparative graphs: boxplots to the left, violin plots to the right. Which of these pictures, one might be tempted to ask, are closer portrayals of Hitchcock’s editing style in those (still British) years?

Wrong question: the main thing I learned from the first lap of our conversation on film statistics is that no statistical method alone gives us a privileged access to the truth. The median or the mean? Both are needed to make the picture bifocal. Descriptive statistics do not compete, they team up, and so do methods of visualizing them. That its data proved to be open to unpredicted uses is the best thing that could have happened to Cinemetrics.

What we owe one another are some comments. It goes without saying that unfamiliar shapes like the ones shown on Figure 2 deserve explaining; less obviously, so does the fact that wavy lines on Figure 1 look so familiar to all. Let me start with a quick glance at time-series modeling from here to antiquity; I will then pause for us to hear what you might have to say.

Every film is a time series and so is every plot, as we happen to know from Aristotle. According to his Poetics (350 B.C.: Chapter 7), the plot is a whole that has a beginning, a middle, and an end. We do not know whether or not Aristotle was in the habit of drawing in the sand as Archimedes is said to have been, but if he was the plot diagram would likely look like this: /\

Why like /\ and not like --- or like \_/? Such is the curve of all dramatic tension, explained nineteenth-century drama theorist Gustav Freytag from whose 1876 book Die Technik des Dramas the following diagram is borrowed:

Figure 3: The dramatic structure diagram known as the Freytag pyramid. The vector runs from a to e, and the whole consists of 5 elements, not 3.

Importantly, Freytag’s graph is a dynamic affair. Introduction (a) and catastrophe (e) found at the base of Freytag’s pyramid are the only two terms with no explicit kinetic connotations; the other three relate to the movement of the curve: b) rising; c) high point; d) fall or U-turn (Umkehr).

In 1890 another theorist, Alfred Hennequin, used his mind’s eye to zoom in on the rising side of the dramatic pyramid to find out that it is a multiple composed of reduced-size copies of the whole:

Figure 4: The dramatic structure diagram from The Art of Playwriting by Alfred Hennequin (1890)

In this diagram, the rising slope is not a single line, but a ladder that consist of nine small rises, apices and falls. It is Hennequin’s idea of dramatic structure that Barry Salt’s Film Style and Technology names a precursor to narratives in film. Sargent’s verbal diagram which I used as my epigraph conforms to this scheme, as well as a metaphor-in-motion which Victor Oscar Freeburg used in 1918 (Freeburg must have been on a train when he thought it up):

Let us symbolize the progression of dramatic attention by a loosely hung cable which ascends a hillside rhythmically over a row of posts. The angles, or apexes, of the cable would each represent a crisis, except the highest, which would represent the climactic point of the plot (Victor Oscar Freeburg, The Art of Photoplay Making (New York: The Macmillan company, 1918): 258).

What happens after the highest point is reached? Sometimes the structure of a drama has this form, and sometimes its form is like this, George Rockhill Craw’s 1911 essay The Technique of the Picture Play – Structure states using two micro-diagrams inserted between words – the “sparkline” type of display reinvented a century later by Edward Tufte and imported into Cinemetrics by Gunars Civjans – to illustrate the this and this:

Figure 5: The dramatic structure diagrams by George Rockhill Craw in The Moving Picture World (January 28, 1911, Vol. 8 no 4, p. 178) to the left; Cinemetrics sparklines to the right

Both time-series diagrams on Figure 5 are intuitive and informative. The Skin Game (figure 5 to the right) has three strong bursts of faster cutting; Easy Virtue, two weaker ones. Craw’s first sparkline (to the left) is closer to how the stage drama is structured; the second one is closer to film: a steep fall after a longer rise was a firm rule of ending a picture play in 1911 (as now).

A time series diagram can also tell us things about cultural idiosyncrasies of film style, for instance, whether or not this or that picture conforms to Aristotle’s triad or to instructions issued by his interpreters from Freytag to Craw. Take the following diagram found in The Art of Cinema and the Film Montage written by Soviet filmmaker Semen Timoshenko and published in Leningrad in 1926. Timoshenko’s system is too complex to address here in detail; it suffices to say for our purposes that his model of editing hinges on perceptible changes in editing rhythm at the moments of dramatic tension.

Timoshenko calls those moments “percussive spots” (udarnye mesta) which roughly corresponds to the Hollywood term punches (in place in the 1910s). The time series diagram of the prototypical six-reel movie according to Timoshenko looks like this:

Figure 6: The montage diagram of the prototypical (normalnogo) film as it appears in Semen Timoshenko’s The Art of Cinema and the Film Montage (Iskusstvo kino i montazh filma) (Leningrad: Akademia, 1926), p. 69. Explanations in the text.

Timoshenko’s whole consists of 2 lines and 8 posts. The bottom line is straight, and is notched and labeled by reels: Reel 1, 2 … 6. The curvy line above it is punctuated by 5 single and 3 double circles. The double circles are film-scale punches; the single ones, reel-scale punches, the legend below explains. Vertical posts of different height project the punch spots upon the bottom line: in 5 cases out of 6 the reel-scale punch happens either half way into the reel (reel 2) or briefly before the end of each reel (reels 1, 3-5).

If Freytag (1876, Figure 3) or Hennequin (1890, Figure 4) could cast a glance at Figure 6 (1926) they would have hard time recognizing in Timoshenko’s abstraction an evolutionary descendant of their own. The three peaks are there all right, but their placement is strange. The good old drama must open and close on a calmer note; this calm before the storm comes back (Umkehr) at the closure either as a happy ending or in the form of the eternal sleep. Not so here: two double punches out of three mark both the opening and the closure of the film; and the intensity of the middle-punch (delivered only 8 minutes before the end of the 1-hour long movie) is lower, not higher than the third one! No plot can move like \_/, Gustav Freytag would exclaim.

Is this dramatic illiteracy at work? There may have been some of it too, but the main thing is that Timoshenko’s model squared well with views on these matters on the part of left-wing Soviet filmmakers in whose circles he moved. Forget Aristotle. A revolutionary movie must grip you from the start and electrify at the end. Closures, happy or tragic, are the thing of the past. The key task for the proper ending of the quintessential Soviet movie was not to provide a closure but to furnish an exit: from the past to the future, from fiction to reality, from the screen into the viewing hall. The examples are too well-known to burden you with titles. The main thing is: that the highest peak on Timoshenko’s time series diagram is not followed by any slope, steep or gentle, is not an oddity or an error, but an outcome of an artistic doctrine.

Time-series modeling is not pure speculation. What makes time models relevant to cinemetrical studies is their embeddedness with all three echelons of film production: preproduction, postproduction and production proper. Preproduction is mainly about scenarios and scripts. Some (not all) Soviet shooting scripts of the 1920s included projected footage (in meters) for each shot. Here is one from 1929:

Figure 7: Details from a shooting script by the Vasiliev brothers for The Sleeping Beauty (1930) before shooting (to the left) and after some of the shots (crossed out) have been filmed (to the right). The shooting script of this type is a table in 5 columns the 4th of which lists shot lengths in meters: 2, 2, 1 ½, 1, etc. Explanations in the text

As we can see from the two scraps from Vasilievs’ shooting script shown on Figure 7, what needs to be filmed and how, is laid out as a table with thousand-plus rows, one per shot, and five columns specifying the position, place of action, closeness, length and contents of each shot. “I thoroughly approve of the dictum that if anything is worth putting into a table it is worth analysing statistically,” says Mike Baxter in his recently published "Picturing the pictures: Statistics and film" (Significance 2012, Volume 9 Issue 5, p.6). If the dictum applies here as well, one can select all 1661 time values from the 4th column of the script (available in full, while only around a third of the resultant film survives) and convert them into a time-series graph. No one has done this job, so if someone volunteers he or she will be able to discover the time-series model which the Vasiliev brothers envisaged for their yet unmade film.

To what extent pre-timing defines what would take place on the set is a separate question. “Photoplays are put on,” one 1913 manual of screenwriting aphorized, “with a stop-watch in one hand and a yardstick in the other” (J. Berg Esenwein, Arthur Leeds, Writing the Photoplay (Springfield, Mass.: The Home Correspondence School, 1913, p. 147). Did many directors use a stop-watch to know when to say “cut”? Yasujiro Ozu did; we are unaware of many other examples. More typically, film directors think about the action rather than cutting on the set. Lev Kuleshov was a notable exception: his rehearsing and staging method foresaw that however actors moved their movements had to inscribe into a temporal grid. Here is a time-series model (1935) which Kuleshov built to explain the method:

Figure 8: Diagram from Lev Kuleshov’s book The Practice of Film Direction (1935), English translation in: Kuleshov on Film (Berkeley, Los Angeles, London: University of California Press 1974, p.193). The curve represents action, the direct line, cuts between shots. Explanations in the text

The X-axis on Figure 8 is marked by 6 notches; these are cuts between 5 shots of unequal length. The sinusoidal curve depicts the ups and downs in the tempo of action along the imaginary Y-axis. The montage rhythm of a film, says Kuleshov, emerges from the interplay between the beat of cutting and the flow of action. That each time a cut occurs on the AF-axis the curve, too, crosses it is a reminder: always think about cutting when directing actors on the set.

Post-production is when time-series modeling takes center stage, especially in documentary filmmaking where the temporal shape of a film is harder to script before or to control during the shooting. That I refuse to use scenarios and actors, the Soviet die-hard non-fiction director Dziga Vertov declared in 1922 does not mean that I rob my films of structure. Appended to this manifesto is the following visual construct:

Figure 9: Diagram from Dziga Vertov’s manifesto “We” (Kino-Fot 1922, No 1, p. 12). The smaller drawing below is a skewed and angular version of the arch-like chart above. Explanations in the text

The lines we see on the upper part of the drawing form seven arches of different sizes, two pairs of smaller arches nested beneath two larger ones, the latter ones beneath the king-size arch. The overarching one (ak) represents the entire film which Vertov calls “work”. The small ones (ab, b…k), called “phrases,” represent sequences of shots. Each sequence has its peak (marked by dotted verticals), as does the work as a whole (the solid vertical).

This, of course, is an abstract compass-drawn time-series model (its smaller-scale version below tries to amend this: here peaks are peaks, and are poised forward); an interesting question to ask, however, is to what extent if at all Vertov’s practice of editing lived up to his own theoretical standards. Of the films Vertov made in 1922, Kino-Pravda 9: (8) Description: https://www.cinemetrics.lv/sparkline.php?&movie_ID=8001&degree=8ASL 5 turned out to be the closest approximation to the model:

Figure 10: Detail of the diagram from Dziga Vertov’s manifesto “We” (Kino-Fot 1922, No 1, p. 12) top left; Cinemetrics diagrams for Kino-Pravda 9: (8) Description: http://www.cinemetrics.lv/sparkline.php?&movie_ID=8001&degree=8ASL 5 at degrees 2 (top right), 4 (bottom left) and 8. Interpretations in the text.

I am not sure if the comparisons made on Figure 10 make much sense mathematically and statistically (or, for that matter, any sense at all), but what I did above was to see what happens with the trendline at different degrees of smoothing. I chose degree = 2, degree = 4 and degree = 8, for the mere reason that the 2, 4, 8 series has two common multiples with numbers 1, 2, 4 which define the self-similar pattern of Vertov’s time-series model (Figure 10 upper left). At degree = 2 (upper right) the curve on the Cinemetrics diagram looks quite similar to the umbrella curve on Vertov’s ideal graph; degree = 4 gives us two humps, somewhat like Vertov’s two second-tier arches; degree = 8 gives us 4 hunches which is equivalent to the number of AB B…K “phrases” on Vertov’s graph, though, of course, they are not nearly as neat. All this may be smoothing tricks, of course; but then, isn’t using a pair of compasses to draw the ideal time-series lines also a form of smoothing?

To round up: time-series analysis inherits from time-series modeling which is a medium-specific (Figure 5), culture-sensitive (Figure 6) and movie-generative (Figure 7) procedure. This is the statement part of my question. The question proper will be about that newer, more sophisticated but also more esoteric part of our common toolkit, the thing called statistical distributions.

Cinemetrics is editing-in-little. What we do as we click on cuts to measure a film and what happens when Cinemetrics converts our series of clicks into a graphic semblance of a movie replays, in a nutshell, what the editor had been doing when our film was still on his or her editing table.

Figure 11: Shots related to the process of editing copied from Dziga Vertov’s Kino-Pravda 18 (1924) (B) and Man with a Movie Camera (1929) (A & C). Interpretation in the text.

The order of shots shown on Figure 11 is in sequence with three operations which film editing before video involved. First, shots of different lengths are stored tails up on the backlit shelves for the editor to see what’s on which. Operation B is to choose one, trim its length and join to another one against the backlit flatbed of the editing table (C); and so on till the film ribbon has grown into a reel.

The ABC sequence of editing operations is remotely like what they call the positional distribution in linguistics, which is about which elements of language fit (or do not fit) in this or that position within the current of speech. As I understand from reading Salt, Redfern and Baxter, statistical distribution entails the inverse: the analyst removes shots from the serial time, from the “montage syntagm” as it were. It is as if the ABC order on Figure 11 were read right to left to became CBA. I may be wrong about this, but isn’t lognormal distribution an intellectual equivalent of disassembling, unediting a film and putting shots of different lengths back on the shelves (Figure 11, shot A) or into a number of “bins” to see which bins are loaded denser? Both ABC and CBA are heuristically significant moves. We have seen what we can learn looking at how shots behave when spliced together; what do we find out when we put them back into bins?

“The preceding paragraph is indeed a rough way of describing how a distribution is created,” Barry Salt wrote to me a few days ago in response to a rough-draft version of this. It may be rough, I agree, and this is exactly why I leave this paragraph intact. If it has caused Barry’s response it works as a question. What makes statistics look warm and human in my eyes is that there is as little agreement on any issue as there is in humanities, and as many methods to use to approximate the truth. The more we care for films the more we worry and quarrel over them. What is better for them, parametric or non-parametric treatment? Does Foreign Correspondent look lumpy or not? And even the question of each film’s normality seems as opinion-based among statisticians as it would be among psychiatrists.

I wish I could come up with more pointed questions, Barry, but these would require more statistical competence than I have. I know I should have done my homework, Nick. On the other hand, if I had this conversation might risk turning into a shop talk, which is not its goal. Do not feel tied to my questions if they look too general to make much sense – feel free to ask your own.