The Slow Formation of Durable Software

Screenshot of the first alpha of what we initially called SmartFox and then Firefox Scholar, before we consulted 1) a lawyer, and 2) an Albanian dictionary for other options, June 2006 * * * Today, software can be created more or less instantly with AI, for a large user base or for yourself, for any purpose or for no serious purpose at all. As Robin Sloan enticingly puts it, just “ask for what you want.” The genesis of Zotero, the “free, easy-to-use tool to help you collect, organize, annotate, cite, and share research,” used by over 20 million people in dozens of languages and countless disciplines, is the opposite of this instant gratification. It took five years, not five minutes, to produce a rough initial prototype, and years after that to refine and expand the application. If AI had existed in the early aughts, we could not have accelerated Zotero’s conception, because we did not know exactly what we wanted, and so could not have written coherent prompts for an LLM. Instead, it took a great deal of time and collaboration to develop a clear vision for what Zotero should be. But that slow formation led to software that was durable rather than ephemeral, with a strong foundation that could be built upon. As programming increasingly becomes a caffeinated bender with magical bots, this measured pace and emphasis on communal thinking likely holds an important lesson. On this 20th anniversary of Zotero’s launch on October 5, 2006, I will do my best to recount Zotero’s pre-history, what it took to conceive, design, and build the 1.0 release. It’s a story about a bunch of historians who were knowledgeable about emerging technologies and knew how to code — as a secondary rather than primary skill — and who spent a lot of time, some of it wasted time, at a big table chatting. That conversation eventually coalesced into ideas about the future of scholarly research. As the well-worn saying correctly asserts, failures are orphans while successes have many parents, and Zotero is no different. A proper accounting would require far more than this relatively modest post, and what follows is, unsurprisingly and necessarily, from my own perspective. My contributions to the success of Zotero date largely to the years before and immediately after the launch of the 1.0 beta, when I was an Assistant Professor of History and Director of Research at the Center for History and New Media at George Mason University. This institute is now the Roy Rosenzweig Center for History and New Media (RRCHNM), named after the visionary historian who founded it and really the entire field of digital history. Roy tragically passed away in 2007 when he was only 57, and his illness and death hang like a dark cloud over this narrative. I succeeded Roy as the Director of RRCHNM, and from that point on, my day-to-day work on Zotero ebbed, as I handed off responsibilities to other, more capable people: Zotero’s talented lead developer Dan Stillman; Sean Takats, who has deftly overseen the massive growth of the project for much of the last two decades; many others in the orbit of RRCHNM and the non-profit Corporation for Digital Scholarship who handled outreach, support, and countless details; an international team of developers and volunteer contributors; and still more colleagues, only some of whom, regrettably, I’ve been able to highlight below. On this anniversary, you should visit Zotero’s Credits and Acknowledgments page to honor the full slate of productive collaborators. Sean Takats also has a great post on the continual improvement and influence of Zotero since its launch. I’m proud to have been there at Zotero’s origin and early years, and to have helped bring it into existence. At the college graduation of one of my kids this past spring, an attendee who had heard that I had been involved with Zotero hugged me, unexpectedly and at length, in appreciation. No one has ever done that for my scholarly writing, or ever will. Members of the Zotero team in late 2006. Roy Rosenzweig, months into chemotherapy, is kneeling at center; he had just gotten the Zotero license plate for his Prius to the great delight of all of us. Standing, L-R: Sean Takats, Trevor Owens, Josh Greenberg, me; Kneeling: Kari Kraus, Roy. Photo credit: Sharon Leon * * * Software development was not originally on the docket for the Center for History and New Media. With its now delightfully retro, turn-of-the-century name — new media rather than digital — it was largely in the business of developing sites for the web, which was just a few years old in 1994 at the center’s founding. I joined RRCHNM in January 2001, as a newly minted Ph.D., to work on a project on the history of science. Alongside another new Ph.D. who had studied the history of technology, Jim Sparrow (now a history professor at the University of Chicago), we built a website called ECHO: Exploring and Collecting History Online, funded by the Alfred P. Sloan Foundation. “Collecting” was a new gerund for the center, and hinted at an expansion of our methods. Since science was growing exponentially but the number of historians of science was not growing at all, we thought we could use the web to help scientists self-document their work. For this, our site needed to be interactive, so we started to tinker with “tools” — small web apps — that could not only display history online, but also allow it to be uploaded, sorted, and archived. Toward this end, I wrote several applications in PHP, then swiftly becoming the standard programming language for web applications, because it commingled well with HTML, the web’s lingua franca. One of these apps, Web Scrapbook, got a bit of traction, especially in classes, since it allowed students to capture images, links, and other resources from the web browser into a collection that could be shared with their classmates and instructor. Users clicked on a browser bookmark, which had a small bit of JavaScript embedded in it, to select the desired items and pass the data to the application. Web Scrapbook was not, shall we say, a rigorous piece of software — my initial version sent passwords, unencrypted, across the internet — and it paid only basic attention to metadata and other scholarly information that would be needed to assist in writing an article or book, and to form a proper footnote. Be forewarned, ye who dare log into my web application c. 2002 (screenshot of the Web Scrapbook landing page) Fortunately, at the same time, another colleague at the center, Elena Razlogova, who was the webmaster at RRCHNM while pursuing her Ph.D. (and is now a history professor at Concordia University), was working on a more scholarly application called Scribe, for taking notes and storing citations. Elena used a database application called FileMaker to build Scribe, which users downloaded and ran on their personal computers. Elena Razlogova’s Scribe application, c. 2002 Scribe became popular among fellow historians as a good, free replacement for commercial software like EndNote. It ran locally on a Mac or PC, and had the organizational, search, metadata, and annotation capabilities that Zotero would later expand upon. Elena made several improvements to Scribe in the first few years of the century, as I worked in parallel on Web Scrapbook. * * * So by 2003, we knew how to write web apps and standalone apps, both helpful but both also naggingly insufficient. We were now starting most of our research on the web, and we thought that any robust research application of the future had to be connected to this environment where primary and secondary sources increasingly resided, either as metadata in library catalogs, or as full objects if they had been digitized. What we really needed was an application that had some aspects of Scribe and some aspects of Web Scrapbook: a full-fledged research tool that could exist independently of the web, but be fully cognizant of what was going on in the web browser, aware as the researcher surfed across digital collections and library catalogs. On November 29, 2003, Roy asked Elena via email what she would like to do for a next version of Scribe. Roy, Tom Scheinfeldt, another recent history Ph.D. who had joined us to work on the September 11 Digital Archive (now a professor of digital humanities at the University of Connecticut), and I were working on a grant proposal for a new stage of ECHO, and we thought we would include some improved software tools as part of that. Elena wrote back on December 1, 2003: Roy--Here’s what I'd like to do:setup to connect to online bibliographic databasesmore citation styles (at least APA and legal)setup to add custom citation stylessetup to add more fields and types of referencesuser/password setup to share and co-edit online bibliographies/notessetup to work locally then publish all or parts of bibliographies/notes online (like ical)foreign language supportbetter manualonline discussion list (using the software marty had installed)best,elena This was a good list. Tom wrote back that “it would be nice to update and ‘webify’ Scribe.” A good word: webify. Elena, Tom, Roy, and I agreed on this essential next step for Scribe: we needed to find a way to put it into the web browser, like Web Scrapbook, while retaining Scribe’s provenance as a detail-oriented research tool. As I later summarized our holy grail in a journal article: We wanted the best of both worlds: the best parts of standalone applications and Web applications. We envisioned a tool that lived in the browser and that was very smart about what was going on in the browser, including the recognition of scholarly metadata and objects, and that could interact with elements both on the desktop, such as word processors or other programs, and via standards, services, and communication protocols to other tools and resources across the Web. But how to merge these disparate worlds? That was not at all clear in 2003. * * *