Brogan Bunt, Chair of Multimedia, Murdoch University
Pubtalk presentation for Thursday August 30 2001
What potential does interactive multimedia have as a documentary medium? We know it can serve as a convenient means of linking together a diverse set of documentary materials, but are there interactive paradigms available to enable a more sophisticated work of interpretation? This paper considers this question both at a general theoretical level and in terms of the issues raised by a recent Turkish/Australian interactive documentary project.
There is really no such thing as interactive documentary, certainly not as a clearly delineated genre. There are only experiments - diverse attempts to chart a relation between traditional notions of documentary and the possibilities associated with digital media. This article begins by trying to clarify some of the theoretical issues involved in envisaging interactive documentary. It then moves on to consider aesthetic issues arising from the production of a recent experimental work - Halfeti, a CD-Rom that enables the user to explore a small south-eastern Turkish town on the eve of its partial submersion beneath the waters of a large hydro-electric project.
The following constitutes a kind of urban myth of interactive multimedia: an author discovers a box that belonged to her late grandmother in the garage. It is full of photographs, letters, official documents, knick-knacks - all the small stuff that sums up a life and that nobody quite has the heart to discard. The author sifts through it and recognizes the potential for a book, a film, or perhaps a CD-Rom. It is the latter that seems to offer the greatest potential, providing a means of viewing and tying together all the contents of the box, as well as the utopic hope of creating a work that embodies the associative logic of subjective experience and memory.
It's not so much the narrative of the myth that interests me (the conceit of discovery), nor the specifics of what the box may contain, as the notion that multimedia offers a particularly effective vehicle for conveying aspects of the grandmother's life. What are the differences between imagining this intimate documentary as a book, or as a film, or as a piece of multimedia? What are the different aesthetic horizons that these media open up and why the preference for interactive multimedia?
I should stress that this preference is not universal, and indeed is probably a minority view these days. There has been a critical backlash against the utopic view of interactive multimedia for at least the past five years or so (Kahn, 1996). There is a widespread perception that interactive multimedia has failed to live up to its over-hyped aesthetic promise, yet digital convergence constantly throws up new interactive technologies and creative experimentation continues. What I have posited above then is a kind of naive myth, intended to open up and clarify the issue of how an interactive documentary might differ from other forms of documentary.
At first glance, one of the obvious advantages of multimedia is that it provides a means of assembling all or most of the contents of our imaginary box into a single accessible form. Multimedia enables the easy combination of still image, text, audio, and video. A book could not easily incorporate all of these different media. A film could manage it but would need to be very selective due to time constraints. This points to the archival value of multimedia - a great deal of information in all sorts of forms can be assembled in the one work. Here the documentary potential of multimedia is linked to the root sense of the term "documentary" - the capacity to document and record.
Yet without wishing to denigrate the importance of primary historical evidence, a documentary is much more than merely a collection of records. We expect documentaries to make arguments and tell stories, to select, combine, and interpret whatever may be found in the box. There are all sorts of traditions within film documentary for accomplishing this task, from explicitly interpretive voice-over narration to the subtleties of cinema-verite mise en scene and editing. Most of these traditions depend upon the temporal language of narrative or sequential argument. Interactive multimedia can draw on these traditions but there is a sense that they are not strictly native. The emerging language of digital interactivity has its basis is in the trajectory of the user rather than the linear artistry of the traditional author. The interactive author builds structures of interaction rather than finished compositions.
This opens up a key question. What kinds of structures of interaction can be developed to lend interactive documentary a properly documentary character rather than simply a glorified archival one? Is there some interactive alternative (or slight twist) to the existing traditions of documentary storytelling and argumentation? I realise I am posing these questions in an oversimplified, either or fashion. My aim is not to deny multimedia access to processes of linear composition, but to pinpoint the distinctive potential of interactivity.
Let's consider some interactive paradigms, some ways in which the contents of the box may be approached, or articulated, in interactive terms.
* The hierarchical tree paradigm. A main page introduces the grandmother and provides a menu of links to subsidiary sections that represent particular periods in the grandmother's life. These in turn link to further subsections that provide relevant materials from the box. Here the emphasis is upon facilitating access to information through the construction of a logical hierarchy. It is the structure typically adopted by the corporate website. This structure can often prove useful, but can hardly be said to engage with, or develop, the discursive aesthetics of documentary. If there is a creative documentary element here it is likely to be in the composition of particular pages. Especially promising in this respect is the emerging dialogue between graphic design and documentary traditions. The interplay and juxtaposition of elements on a page - text and image, foreground and background, interface and content - offer all sorts of possibilities for documentary experimentation.
* The associative network paradigm. A main page sends us randomly to one of many pages that represent particular moments in the grandmother's life. One page shows an immigration document and lists her parent's names. Click on the names to jump to pages on her childhood, or perhaps pictures of her parents, or a diary entry concerning her earliest memories. There is no menu to guide the user, only a selection of links embedded in images or text. This paradigm sets up an associative (often playful) relation between a set of pages (or media). Rather than an hierarchical tree there is the sense of a network of interconnecting nodes. This is the model that literary hypertext adopts. It can also be regarded as a web paradigm, if stress is placed on the macro structure (or experience) of the web rather than the typically hierarchical structure of particular sites. Multiple logics may guide the pattern of associative links - time, cause and effect, resemblance, difference, etc. The notion of a non-hierarchical associative network offers considerable potential as a novel means of approaching documentary topics, especially when the user can engage closely with the associative logic of the piece. This is especially so when there is little need to tell a specific story or make a particular argument. An observational style documentary in which the user develops an overall picture of a topic by selecting from a range of 'views' would seem to lend itself well to this approach. The major challenge for this paradigm lies in finding means to motivate the user's choices and to establish a balance between the content of any particular page and the aesthetic imperative to interact. A kind of idle (fetishistic) clicking can easily supplant more focused interaction.
* The simulatory matrix paradigm. An exterior view of the grandmother's old house. Click on the front door to enter. A view of a hallway with an open door to the left. Click to enter a living room. The sound of an old record starting up. Click on an album of photographs on the coffee table. Turn through the pages of photographs of the grandmother's life. Here interaction is structured via a spatial-experiential metaphor. The emphasis is less on logical association (jumps, juxtapositions, correspondences) than on experiential contiguity between one node and the next. There is the sense of moving through a defined matrix rather than a loose associative network. While convincing simulation may not be a high priority, the interactive structure is simulatory in that it draws upon experiential metaphors rather than more abstract logical schemas. This paradigm gains its clearest realisation in the structure of first-person, real-time, 3D computer games. The artificial, agonistic, and viscerally immersive character of these games may seem very distant from the realism and critically-reflective stance of mainstream documentary traditions, but there are undoubtedly creative possibilities here. A documentary on ancient Rome could, for instance, enable the user to wander the streets of ancient Rome - interacting with the people, performing various set tasks, solving puzzles, etc. This is to move away from the traditional conception of documentary as a mode of representation intimately linked to the authentic texture of reality - opening up the possibility that it might also serve as a mode of simulatory exploration and insight.
* The games paradigm. A jumble of jigsaw pieces, each depicting a portion of an image of the grandmother. Beside them an empty frame. The user must drag the pieces into their appropriate place within the frame. When they are correctly placed then a section of the grandmother's story plays. Here the documentary takes shape as a jigsaw puzzle. It is structured as a game. Games have a linear character in that they point to a particular goal, but they differ from narratives and arguments in that they place crucial stress on user interaction. Games can take many different forms. They need not employ experiential metaphors, although typically strategic puzzle-solving and spatial exploration are linked. The possibility of linking games-play to documentary representation deserves experimentation. There is a fairly obvious satirical potential but perhaps it could also serve more serious illustrative or argumentative purposes. The problem is once again in making the shift from a representational to an interactive mode, or, more precisely, in finding a satisfactory means of enabling both modes. A naive view imagines documentary 'shows us reality'. The viewer is positioned at a respectful distance - seeing and listening, but certainly not literally interacting. How can the viewer ever literally interact with an 'authentic record'? And then to interact in terms of a game? There are crucial theoretical and ethical issues to wrestle with here.
* The participatory paradigm. A web page poses the question: what do you know of your grandmother? Is there any single memory or object that best defines your relationship to her? If so then create a web page that deals with this topic, send us the link and we'll establish an on-line database-documentary of grandmother memories. Here the user is not simply interacting with a pre-constructed piece of documentary multimedia, but is instead called upon to collaborate in the work of documentary creation. This paradigm draws upon the potential of networked multimedia systems to enable literal social interaction/participation. This need not take the form of producing collaborative works. It could also open up documentary as a field of dialogic exchange - an open forum in which the traditional documentary subject (the people represented) and the wider public (the viewing audience) address social issues via interactive multimedia. Discursive novelty is established here less in terms of the actual content of any particular contribution than in terms of the broadly participatory framework. This need not imply a naive and utopic vision of public communication. The web is full of examples of focused public interaction, from the websites associated with commercial reality television shows (with their on-line voting, discussion forums, and chat rooms) to the many low-tech lists devoted to specialised (potentially documentary relevant) topics.
It should be stressed that these paradigms are not mutually exclusive. It would be possible to imagine all kinds of combinations of the above. My aim in describing them separately has been to clarify the possibilities, not to envisage five discrete interactive genres.
Halfeti is (or was) a small town on the banks of the Euphrates in southern Turkey. In the middle of last year (2000) it was partially submerged beneath the waters of the Birecik dam - a large new hydro-electric project some 100km north of the Syrian border. I travelled there in early 2000 to document aspects of the town prior to its flooding. My primary focus was on preserving a navigable photographic record of the physical space. I'd been working on programming constructs to enable virtual exploration of imaginary spaces - haunted houses and the like, and the opportunity to apply these constructs to a complex real space was tempting. Another aim, probably less clearly conceived, was to document the responses of the Halfeti people to the prospect of having to abandon their stone homes, cobbled lanes, and alluvial orchards for a new housing development on the rocky escarpment.
I have no wish to position this project as some kind of realisation of the potential of interactive documentary. Indeed for much of the production phase, I scarcely even considered these larger aesthetic issues. I was mainly concerned with the various technical challenges involved in facilitating a sophisticated level of spatial navigation. I had a very humble sense of the scope of the project at a documentary level. It was the record of a town on the eve of its disappearance.
I had taken over 3000 digital photographs of the town and my associate, Bora Kanra, had recorded some 6 hours of audio and video. This material had to be edited and optimised to fit on a single CD-Rom. The main challenge was in developing an efficient way to piece all this material together into a responsive and spatially cogent interactive experience. After a number of aborted efforts, I was forced to develop specialised authoring software for the creation of multimedia 'virtual tours'. This took considerable time, but made the final work of putting everything together much simpler.
The aim was to employ an experiential paradigm. The user 'wanders' the streets of Halfeti - moving forward and back, turning left and right, looking up and down, etc. Each move represents a jump from one photographic image to another. Choosing a direction to proceed involves clicking on relevant parts of the screen - to turn left, the user clicks at the left of the screen, and so on. Atmosphere sound supports the sense of first-person exploration. Screen hotspots enable access to additional media, so that clicking on the image of a person may, for instance, display interview-based video material.
The lack of grand documentary ambition proved advantageous in ways. There was an emphasis on directly and straightforwardly showing the town. The photographic style is very frontal. Each photograph represents a view - north, south, east, or west. The images were building blocks, compositionally bound by the requirements of the larger tour. In a similar way, the exploratory interface is deliberately low key, placing the stress on images and sounds of the town rather than ostensible interpretation. Of course none of this is stylistically or interpretively innocent. The avoidance of obvious interpretation is the classic trope of the cinema verite documentary tradition. My point here is that this approach arose from and meshed nicely with the focus on technical implementation. That it also worked aesthetically (and perhaps even interpretively) was a bonus.
As I say, it was only late in the production phase (once most of the technical problems had been solved) that aesthetic/interpretive issues returned - more or less unexpected - to the fore. These issues of selection and combination, and of discursive positioning, suggested that there was something more at stake here than just producing an historical record. These issues harked back to the concerns of traditional documentary, but were also suggestive of another space of aesthetic possibility. They arose from very specific creative issues, so I'll deal with them in those terms.
* Snow to the east. Each positional node tended to involve at least four images, one for north, east, south, and western views, so that when you click to the left or the right the view switches ninety degrees in the relevant direction. From a technical perspective, I should have taken more images at each position, but this would have complicated the programming task (I'd been thinking in terms of a vast chessboard-like matrix), push up the file size of the overall work, and take that much longer to photograph. The problem with my approach was that there were no areas of overlap between one compass view and the next. There was overlap if the user moved forward or back, but not if they turned on the spot. Older style, still image based, spatial-exploratory computer games typically employ an hexagonal style matrix that enables six views from any particular position. This creates a minimal level of visual overlap, ensuring continuity between one turning view and the next. These days, of course, the typical user expects much more than static views. They are accustomed to the seamless panoramas of Quicktime VR and the immersive experience of real-time rendered 3D. In these terms, the approach I adopted in Halfeti is likely to seem anachronistic and experientially weak. Yet apart from technical exigencies, there was another rationale for my approach. I was never going to produce an experience that was equivalent or adequate to the experience of actually exploring Halfeti. I was producing an imperfect record - a very ostensibly stitched together one. The distance between clicking through the streets and walking through them was something vital to the project. There was a need for a mode of simulation that would provide access - but not glib access - to the alterity of another space (and another time). The ambivalent, never quite adequate, sense of spatial continuity also had a creative dimension. I realised this very clearly while piecing together the images of the central town area. It turned out that I had taken images of the area on a number of different days in very different weather conditions. At first I thought that I'd better strive for visual consistency, but it ended up working better to allow sudden shifts from sunshine to snow, daylight to darkness. The same view might even have two potential states - differing depending upon the direction the user approaches it from. It became possible to open up secret corridors of darkness or snow through what was ordinarily a sunny space. This worked to emphasise the constructed (discontinuous) nature of my simulation, but also to enable (and plainly manifest) a creative work of spatial collage.
* The sound of card playing. I'd spent most of the time labouring with images - adjusting, organising, and choreographing them. I'd always intended to add atmosphere sound, but the sound was a problem. I really only had what was on the videotape, and most of that was interview-based stuff. It was a matter of searching for all the in-between bits on the videotape, all the moments when nothing was being said and perhaps some slight bit of atmosphere sound could be extracted. Note to self; in future always bring along a dedicated sound recordist. The sound from videotape is not ideal - all sorts of strange clicks and camera operator grunts make it very difficult to cull out anything very useful. Despite the poor quality of the sound it ended up being crucial to the final project. It made the whole exploratory experience much more compelling. All it took to make navigation through a room of card players come alive was to add a single, twenty-five second sound loop. I'd discovered the obvious truth that sound, even more than vision, is vital to our sense of space. But sound did more than just lift and support the images (lending them greater continuity). It also opened up a kind of dialogue with the static space of the photographs. It introduced a temporal element, but in complex ways - at once reaching out and summoning the other (past) time of Halfeti and yet also constituting a very present-focused musical play. The atmosphere loops are all obviously loops and their loose association with particular images means that moving about becomes more than just spatio-temporal simulation. Navigation becomes a playing with sound and with image sound relations. The pace at which the user moves has a crucial effect on which sounds play. In this way, the experience of sound becomes different for each user. Sound then has an ambivalent status - working both to heighten realism and to open up another terrain of (interactively grounded) distanciation.
* How to cross the Euphrates. Abdullah ran the ferry service across the Euphrates. He rowed back and forth throughout the day - huffing and puffing at the oars, negotiating the difficult currents, and abusing those who annoyed him. It was generally agreed that once the town was flooded the need for his ferry service would disappear and he'd be forced to find work elsewhere. Abdullah emerged as a crucial emblematic figure. Adopting the perspective of an imaginary ferry customer, I photographed him rowing back and forth across the river. Ultimately this provides the basis for the user to approach the river, climb into the ferry, experience the sights and sounds of the crossing, and then disembark on the opposite side. The interesting point is that crossing the river becomes more than just a spatial transition. It becomes a temporal sequence - a process. This made me aware of the potential to create more than just a static record of a place. It was possible to create navigable sequences that both made spatial sense and also told stories. So, for instance, in one of the crossings the user must click back and forth on the screen to observe donkeys being led down the bank and then placed on the barge. The photographs follow a shot-reverse shot logic. Without disrupting the sense of spatial navigation, they open up a narrative dimension. It gradually dawned on me that there were all sorts of opportunities for subtle interplays of space and time. If I'd realised this earlier, I might have devoted more time to following the trajectories of particular people (characters) rather than concentrating so exclusively on spatial paths. I can now imagine an interactive documentary mode that discovers in space stories and in stories space.
* The hidden phrasebook. I had played with the idea of introducing game-style interactivity - setting tasks for the user as they roamed about. Perhaps they would need to find a hidden Turkish phrasebook in order to get an English translation of what people were saying. In the end, however, I decided to avoid game elements. They were tempting because they provided an obvious means of motivating the user's exploration, but they also threatened to render Halfeti a mere picturesque tableau for game-based activity. The exploration that I was trying to encourage was different. It wasn't about solving logical puzzles. It was about engaging - in a limited but audio-visually compelling manner - with the alterity of another place and another time. Perhaps this would provide the basis for increased understanding, but even more importantly I wanted to encourage an emotional response. This response would emerge from the gap between simulation and reality. It was about making loss tangible, or almost so.
This paper has been an attempt to clarify what is at stake in imagining interactive documentary. At a general conceptual level, it specified five different paradigms of interactivity: the hierarchical tree; the associative network; experiential simulation; game play; and networked participation. It then moved on to consider a specific experiment in documentary multimedia production. The creative and theoretical issues that the Halfeti CD-Rom raise indicate that traditional questions of documentary interpretation remain relevant to the interactive terrain. Even an ostensibly non-interpretive, archival piece has all sorts of scope to creatively articulate aspects of space, time and interaction.
In terms of the prospects for developing an interactive documentary genre, theoretical projection is clearly vital, but so too are the realisations that emerge more or less unexpected in the midst of creative practice. We can only imagine so far, then there is the need to get started and see what happens. This should not imply the need to abandon documentary's 'linear' past. On the contrary, it is most likely in the friction between tradition and an as yet unclear future that something new will begin to take shape.
Khan, D (1996). "What Now the Promise?", in Burning the Interface, Museum of Contemporary Art, Sydney, pp. 21-30.
New: 29 August 2001 | Now: 5 May, 2015