When I think of what I’d like this blog to be or how I’d like it to be used, John Resig’s post “Using Computer Vision to Increase the Research Potential of Photo Archives” comes to mind as a model. In a marathon post Resig lays out an entire experiment on the efficacy of image recognition tools. His post provides not only the results of this inquiry, but a thorough yet concise summary of what tools are out there, why they are important, and how to most effectively use them. He opens the post with a brief summary and an explanation of the blog format:
I’m taking the approach of publishing my results openly on my site so that they can reach a wider audience. Please feel free to share this with whomever you think will find it useful. If you have any questions, comments, or concerns I welcome an email with your feedback
John Resig
Not only is his project of digital image recognition and matching a digital humanities one, but the way in which Resig is conceiving of it and sharing it further the accessibility aims of digital art historical projects.
Now that I’ve made my side comment on the accessibility of digital humanities projects and how fabulous it is that digital humanists are so willing to openly put their work out there, I want to turn to the specifics of the project. Much of the nitty gritty technological details of the tools Resig references still allude me, but he does do a great job of writing in plain enough language that I was able to grasp the essential details. Essentially, by tweaking existing open source image matching systems Resig was able to quickly go through incredibly vast (think tens to hundreds of thousands) numbers of images with much higher accuracy than human archivists could ever accomplish due to input and cataloging errors. He claims that utilizing these types of image matching systems would help in analysis and error correction, expediting the digitization process, and facilitate the merging of vast archives.
I’d like to first turn to the assertion that these services could help in analysis and error correction. While I assume that the majority of metadata input by archivists is accurate (and that Resig perhaps overemphasizes human error), I will concede that older archives especially could benefit from the implementation of these systems. For one, old duplicate images could be eliminated or at least grouped. Alternate views of the same works (one perhaps including the frame in the image for example) can also be grouped using this tool. Aside from this perhaps more obvious benefit, I think the point Resig brings up regarding the matching of images of works before and after restoration is interesting. This could help us to track changes to works over time and is something that I can agree may be harder for the human eye to detect. Similarly, the detection of detail shots from larger works is important as the more closely cropped images may be so disarticulated from the original that an archivist may misinterpret them as their own works. Resig’s discussion of the the various images of portions of the work he labels as “Florentine, 13th century, Uffizi Museum in Florence” demonstrate this capability really well (see screen shot below). These portions have no overlapping segments, but the image matcher was able to accurately group them so that users can better understand the whole work and its context.
Second, I’d like to comment on the use of these tools in the merging of archives. I hadn’t thought of this issue prior to the reading, as I’d always assume archivists would somehow know where the overlap was in their collections and avoid double-digitizing. This, I realize now, is naive. I didn’t fully comprehend the scale of some of these archives. That the Frick Photoarchive alone has 1.2 million photographs of works of art is mind boggling to me. With this quantity no archivist could truly have a handle on the contents of the entire archive, making a seamless merge between two huge collections nearly impossible. I also hadn’t taken into account language barriers that would impede merging metadata in an efficient way between archives from various countries.