DCI Lecture: “Unleashing the Infinite Archive”

Born-digital historical sources have the potential to reshape the humanities and social sciences. The sheer volume of cultural information generated and, crucially, preserved every day presents exciting opportunities for historians, political scientists, sociologists, and other scholars. Much of this information is captured within web archives containing billions of URLs, including individual homepages, social media sites and feeds, institutional pages, and corporate sites. This material introduces important new avenues of research for historians working in diverse fields. For example, historians broaching topics dating back to the mid-1990s will find their projects enriched by this web data: social historians can explore aspects of everyday life through blogs, homepages, comments, and guestbooks; and economic historians can explore commercial activity online. Other fields, from political science to sociology, can also benefit from working with this data at scale. In short, these web archives make possible reconstruction of large-scale traces of the recent past.

But the opportunities presented by all this data are counterbalanced by the challenges of working with digital heritage at scale. The Web Archives for Historical Research group accordingly joins historians, librarians, and computer scientists at Waterloo, York, and Western. We explore how historians can use web archives, the assemblage of cultural information that various organizations have been collecting since 1996, now providing an invaluable amount of information about the previous twenty years. In blogs, personal websites, corporate homepages, and beyond – now over 16 TB in size – we have the building blocks of historical narratives. Accessing collections on this scale requires specialized computational knowledge and infrastructure.

In this talk, I reflect on the processes that make our research group cohere, as well as to achieve our overall goal of making this data accessible. Our group brings together various perspectives to develop infrastructure, documentation, user interfaces, and sustainable practices to work with cultural heritage at scale. How does scholarly credit work when disciplines come together? What pedagogy do we need when involving undergraduate and graduate students? How can we get “analog” historians involved? The general public? What implications might this have for how we approach professional training? In general, what can we do to ensure that we are ready to write histories of the past twenty years?