Archive-It

July 11th, 2017

by Karl-Rainer Blumenthal

This is the third in Archive-It’s ongoing series of discussions with partners who share resources, policies, and personnel in order to archive the web collaboratively. For Part 1, see Better together: Collaborative web archiving at IIPC, and Part 2, Collaborative collecting: It’s a piece of KAIC.

The New York Art Resources Consortium (NYARC) has blazed trails into digital art historical collection development and access for more than ten years. Since forming among forward-looking librarians at the Brooklyn Museum Library & Archives, Frick Art Reference Library, and Museum of Modern Art Library, NYARC has produced online exhibition resources, led digitization projects, and created a consortial catalog, among other initiatives. Since 2012, this portfolio has also included a rapidly maturing web archiving program. Once grant-supported and now flying on its own, NYARC’s program collects web materials that complement its member institutions’ already world-renowned art historical resources in other media. To better understand (and share!) why and how NYARC made this investment in web archiving together, I interviewed Sumitra Duncan. Sumitra is Head, Web Archiving Program, at the Frick Art Reference Library, where she oversees the web archiving process from nomination to long-term preservation on behalf of NYARC’s network of art librarians and their patrons.

Karl-Rainer Blumenthal: Was there anything in particular that compelled the NYARC institutions to collaborate on web archiving? In this case, why are three heads better than one?

Sumitra Duncan: As Dr. Stephen Bury, Andrew W. Mellon Chief Librarian of The Frick Art Reference Library, mentions in a post on the Library of Congress’ Signal blog, “collaboration has become the watchword for NYARC. It allows NYARC partners to punch above their individual weight and access technology and innovation which they would otherwise be unable to afford.” Since its inception, NYARC has demonstrated a commitment to creating collaborative solutions that enhance access to information resources, as well as their sustainability, and collecting and preserving art resources through web archiving was a logical (and intimidating!) next step. The NYARC Directors noticed early-on the trend of art and art historical publications drifting to a born-digital format and they have been quite proactive in investigating viable solutions for collecting and preserving these valuable resources. Well before we fully dipped our toes into web archiving, NYARC learned from our previous collaborative projects that, while our institutions are small in size, we are able to be quite nimble, and this has allowed us to accomplish a great deal with our collective expertise and resources.

Capture of The Medieval Bestiary, one among 60+ seeds in NYARC’s Art Resources collection

We are very much invested in the notion of sharing our work with others–whether it be through the documentation we have produced about our program or via training emerging art information professionals new to the field. The challenges posed by the scale of web archiving demand a collaborative and information-sharing approach, but I also view it as integral to our collective task as information professionals.

KB: Speaking of scale then, what does NYARC now collect? Has the scope expanded beyond auction catalogs as the program as a whole has been built-out?

SD: Yes, we now have ten growing collections in Archive-It and we are actively incorporating new sites into these collections as part of our formalized consortial collecting practices. NYARC’s Archive-It collections are devoted to our institutional websites, NYARC’s own website and project sites, art-rich websites of significance to the study of art and art history, artists’ websites, auction house sites and their embedded catalogs, catalogue raisonnés, sites of NYC galleries and art dealers, and sites dedicated to restitution efforts and provenance of artworks that may be lost, stolen, or looted.

I think our NYC galleries collection will prove to be particularly valuable to researchers in the future, as this collection is crawled every month and it documents the local art scene as galleries are either rapidly expanding or closing up shop based on trends in the art market (and rising rents in NYC). We’re interested in expanding the geographic scope for collecting gallery websites in the U.S. through partnerships with additional art libraries.

February 2016 capture of the now closed Laurel Gitlen art gallery on Manhattan’s Lower East Side.

KB: Can you describe how the librarians and archivists among your institutions contribute directly to the joint effort today?

SD: Our staff developed a consortial collection development policy drafted with the specific objective of acquiring websites and their embedded content. Selection, nomination, and description of websites are shared responsibilities of the NYARC libraries and this is reflected in the diverse resources we are now web archiving. We also formed a Discovery Working Group as a collaborative initiative between our institutions. Through this group we implemented the Primo discovery layer and integrated our Archive-It collections into a single-search environment. The Discovery Working Group does testing, provides feedback, and contributes to ongoing development of the NYARC Discovery search tool interface.

The majority of the quality assurance (QA) of our web captures is conducted by our part-time web archiving staff and web archiving fellows. Our fellows, graduate students in the Pratt School of Information, are placed with NYARC’s web archiving program each academic year and they receive a stipend to cover tuition costs. The partnership with Pratt has been highly successful and we’re very fortunate to have developed a mutually beneficial relationship where our fellows learn about web archiving, the use of Archive-It (primarily through conducting quality assurance reviews and patch crawls to our Archive-It captures), cataloging of archived web resources, and gain exposure to the rich international community of web archiving professionals/enthusiasts.

KB: So the scope of your collaborations has extended beyond the NYARC institutions; are there other kinds of cooperation with the local or art library communities that have impacted the program?  

SD: Yes, NYARC has been quite active in extending our collaborations beyond our institutions (in the NYC-area and beyond). As I mentioned, in the New York area we work with the Pratt School of Information via the fellowship program. I’m a member of the Digital Art History Lab (DAHL) at the Frick Art Reference Library and through this group I co-hosted an introductory workshop on web archiving for public researchers last fall (we will repeat the workshop in November and you can register here). NYARC also meets regularly with our web archiving colleagues at Columbia University Libraries. Our relationship with Columbia is a long-standing one, and given their impressive collection of archived websites, it was extremely helpful to us to meet with them for information-sharing in the early stages of our web archiving program implementation. With Columbia’s Alex Thurman, I co-chair the METRO Web Archiving MeetUp group and the Archive-It New York Users Group to further foster the exchange of web archiving information in the NY area.

On a larger geographic scale, I am co-chairing the Art Libraries Society of North America (ARLIS/NA) Web Archiving Special Interest Group (SIG), which has now met at two annual conferences. We hosted a workshop at the most recent annual meeting in New Orleans for those interested in getting started with web archiving. ARLIS/NA is in the planning stages of an Annotated Digital Art History Registry and the Web Archiving SIG will be contributing expertise through the archiving of digital art history project websites. My team at NYARC has also contributed to beta testing initiatives and studies with colleagues in the computer science department at Old Dominion University, LS3, and Rhizome, in addition to Archive-It.

KB: Now that you have managed web archives through their lifecycle on behalf of a library consortium, is there anything in particular that you would want a new collaborator in your or another network to know before they get started? Any prospective pitfalls or opportunities that you’ve learned from the process?

SD: It’s important to define the parameters of your project or program as early as possible, as this will allow you to scale your effort in a deliberate and organized way. In regards to web archiving via a collaborative effort, you might find that things can take longer than they would if you were operating in a solo capacity, but in the end it’s worth the time that it takes to determine your shared workflow or solution.

I would say definitely leverage the documentation that is already freely available from web archiving colleagues — they are a truly generous group and will share their web archiving horror stories with you if you ask! Perhaps do somewhat of an environmental scan of existing projects and documentation that mesh well with your collecting vision and see what you can learn from those successful (or unsuccessful) efforts. There is a lot of inspiring work being done, both nationally and internationally, so don’t reinvent the wheel. Web archiving is daunting work (admittedly the challenges have kept me awake more than one night), thus I would recommend to those getting started in web archiving that they tap into existing professional groups devoted to web archiving, attend meetings/conferences on the topic, keep up-to-date on developments for tools and technical solutions, and grant-funded initiatives. Reaching out to your colleagues is truly invaluable.