Source: Annotating multimedia: possibilities for teaching and research

For all intents and purposes, annotation is an utterly ancient technique. Users of texts made in all manner of media have found ways to comment upon, add to, or even radically alter the original source materials. Analog texts often visibly bear the traces of annotation, with the Talmud being one of the most prominent examples. In the Jewish Rabbinic tradition, annotation of scriptures (and then annotation of that annotation, etc.) was one of the primary modes of engagement with and exchange of religious and cultural information, across vast periods of time. Of course, there are countless other examples of annotation occurring in more or less codified ways for all kinds of media: someone writing on the back of a family photograph, a play list and liner notes for a mixtape. We might also posit more ephemeral instances of annotation. Taking up a Derridean position that all speech is always and already a form of writing, we might imagine forms of annotation for oral poetry, such as comments made aloud by the audience at a recitation, or changes made to a work as a new poet takes up an old poem.

Some of the digital tools that we’ve been considering are specifically geared towards facilitating annotation of digital objects, such as Thing Link, for the annotation of images, or the Oral History Metadata Synchronizer, which can be used to index and annotate both audio and video files among other functionality. The most popular video sharing platform on the web, YouTube, makes annotation one of its core editing functions, even though annotations for YouTube videos might include pop-up messages and advertisements more frequently than any kind of substantive content. As researchers and teachers using digital tools, how should we think about one of the earliest and most essential scholarly techniques—annotation—in the digital realm?

In many ways, digital media simultaneously resists and opens itself up to annotation. As hailed by artist Douglas Davis in his manifesto “The Work of Art in the Age of Digital Reproduction (An Evolving Thesis: 1991-1995),” “what begins to emerge in the first digital decade is a fine-grained sensitivity to the unique qualities of every copy,” an observation that Davis expands upon to assert that every interaction of a user with a digital object has the potential to produce not just an endless string of copies, but a new object entirely, a “post-original original.”1 Seen in this sense, the annotation can be construed as a generative and creative act: one user’s tweaking of a digital photograph, say by labeling one of the figures in the image, not only adds to the original image, but creates an entirely new ‘original’ image. With the ready ability of users to manipulate digital objects, creating a unique version of a digital text through the course of use is the rule and not the exception. Like writing in the margins of a favorite book, annotation in the digital realm can be seen as a means for personalizing of a text, contributing your unique insight to an existing, ‘published’ articulation.

However, the very ease and rapidity with which users are able to manipulate digital objects also troubles another aspect of annotation—its additive and cumulative nature. I think that we need to be careful not to overextend our definition of ‘annotation’ in the digital realm to include any and all manipulations of a digital object, but instead should think of annotation as a purposeful addition to some source, which still in many cases may be difficult to parse out with digital objects. In analog media, annotations are able to build up over time. To return to the Talmudic model, a clear trajectory can be drawn from original source text to first commentor, from this to second commentor, and so on. For this model, annotation is a way to accumulate knowledge, as well as to track this accrual, something that is of great importance to the scholarly endeavor more generally.

This kind of archaeology—to see exactly who said what where and when—is much more difficult to perform in the digital realm. This is one of the reasons why archives expend so much time and energy on maintaining preservation metadata for their digital objects, keeping track of any and all ‘events’ in the lifecycle of the object so that a later version cannot be confused with a previous version of a digital file.2 While establishing a precisely clear line from original text to later addition is important for Talmudic study and for digital preservation, this kind of precision may not be wholly necessary for all kinds of digital projects. Indeed, for many digital projects, especially those that thrive on user participation, the inability to parse out an ‘authoritative’ original can be central to the meaning of the text itself. Even still, it can be helpful to have the capacity to manage the processes of annotation in some real way.

To this end, tools like Thing Link are potentially quite useful for scholarly annotation in the digital realm. Speaking to the concern expressed above, Thing Link clearly demarcates annotations as an additional layer of information on top of the ‘original’ source text. Even if Thing Link provides a platform for users to manipulate digital objects, and thus to create a new ‘original,’ the interface with this new creation continues to demonstrate the relationships between the addition and the source—much like the physical layout of a page of the Talmud separates scripture from commentary. While the Thing Link interface facilitates this more scholarly and additive mode of commenting upon a source text, the researcher or teacher is still ultimately responsible for making valuable use of the tool, by clearly labeling the source of their original image and properly citing any information that they draw in as part of their commentary.

I’ve included here an example of my own experimentation with Thing Link as an annotation platform, my commentary on a work by Markus Raetz, “Drei Farben Mappe.” Even with the relatively brief time that I’ve spent using the platform, I can see that there are a wide variety of potential uses. One particularly intriguing aspect is the capacity for simultaneous social annotation; by embedding my object here, and enabling a ‘public’ setting, I’m opening up the possibility that anyone else could also comment upon this work. For instance, I’ve added a note on when and where I first saw this work; someone else might add their own story of when they’ve encountered the work in another setting. While annotation has always been social, digital platforms like Thing Link dramatically open the possibilities for social exchange through annotation. As scholars, we will have to see how this changes or shapes our endeavors moving forward.


[1] Douglas Davis, “The Work of Art in the Age of Digital Reproduction (An Evolving Thesis: 1991-1995),” Leonardo 28, no. 5 (1995): 385, doi:10.2307/1576221.

[2] For more on preservation metadata: