Digital History Year in Review

In August, AOL released some of their search data for more than half a million users. Among other things, it gives us some idea of what people are looking for when they search for history. Dan Cohen continued to underline the importance of APIs for the humanities.

Google digitized millions of Books, leading Gregory Crane and others to wonder what you can do with a million books. The Open Content Alliance had same plan, different vision.

Cyberspace, home of Collective Intelligence and Convergence Culture? Or just a stupid and boring hive mind? Either way, I think it has interesting implications for pedagogy.

Digital history: graduate seminars (1, 2, 3) and a bunch of new bloggers.

Google Earth added historical maps.

Josh Greenberg decided to leave his blog Epistemographer fallow, with good results. Meanwhile, Tom Schienfeldt highlighted the "unintentional, unconventional and amateur" in Found History. Emma Tonkin argued that Folksonomies are just plain-text tagging under a new name.

Niall Ferguson wrote about historical gaming and simulation in New York magazine; Esther MacCallum-Stewart (Break of Day in the Trenches) and Gavin Robinson (Investigations of a Dog) extended the discussion in interesting new directions. At the end of the year, I was musing about alternate reality games.

Sheila Brennan began an ongoing survey of online History museums at Relaxing on the Trail.

I is for Information Aesthetics.

(Just realized why this kind of thing can be tiresome.)

Keyword in context (KWIC) is easy to implement, but Jeffrey Garrett argued that it doesn't really fit with the way humanists think.

Brian Hayes discussed the search for the one, true programming Language in American Scientist. It's LISP.

Dan Cohen noted that Machines are often the audience these days. He also wrote about data mining, provided an example in his blog, and warned that we shouldn't allow available tools to guide our inquiry. I spent much of the year developing tools to mine the Dictionary of Canadian Biography and other online repositories.

In September, Google released N-gram data to researchers. I argued that it could be put to use in digital history, and might help to change our notions of plagiarism.

O is for old. Just when you thought it was the new new, it turned out to be weird weird.

Manan Ahmed penned a Polyglot Manifesto in two parts. I switched from Perl to Python.

Q is for the <q> element, of course. Paula Petrik guided historians through the ins and outs of (X)HTML and CSS at HistoryTalk.

Choudhury and colleagues demonstrated a new tool for document recognition.

Exactly one year ago, in my first substantive post to this blog I suggested that historians be taught to search, spider and scrape. I still think it's necessary. Re: searching, Dan Cohen discussed the appeal of the single box, and Phil Bradley wrote about the past and future of search engines.

Tags: Mills Kelly thought they could be used to subvert the archive. TEI: Keith Alexander talked about authoring directly in it. Timeline: the SIMILE group released a "DHTML-based AJAXy widget."

Alan MacEachern started a series called "the Academic Alphabet" in University Affairs: "A is for admissions," "B is for books," and so on. Although he occasionally talks about digital history, he won't get to U until 2008. Good luck, buddy.

Tim Burke wrote about the history of virtual worlds in Easily Distracted.

W is for Wikipedia. Roy Rosenzweig wrote about whether history can be open source; Joseph Reagle blogged about his dissertation project on Wikipedia; Mills Kelly thought up classroom assignments; John Jordan discussed national variants. The New Yorker and Atlantic ran stories, too.

I discovered the joys of XAMPP when I set up the new digital history server at Western. Jeff Barry wrote a similarly positive review at Endless Hybrids.

Yahoo! Term Extraction, a handy way to find keywords in content.

And finally, in September I got my first look at the cool new Zotero. Expect it to play a big role in digital history next year.

Tags: digital history

Pedagogy for Collective Intelligence

In my last post I suggested that digital history might be able to harness the power of what Pierre Lévy calls "collective intelligence," a group of self-selected, networked individuals who work together to solve problems that are far beyond the capacity of any one person. In such a setting, knowledge becomes valuable when it is shared, and the strength of the collective depends on the diversity of skills and information that individuals can bring to it. Theorists like Lévy, Henry Jenkins and others suggest that such collectives may already be flourishing in online fandom and massively multiplayer games. To take a single example that Jenkins discusses, when children play with elements from the multibillion-dollar transmedia franchise Pokémon, they enter a world that is too complicated for any child to understand by him or herself. "There are several hundred different Pokémon, each with multiple evolutionary forms and a complex set of rivalries and attachments," Jenkins writes. "There is no one text where one can go to get the information about these various species; rather, the child assembles what they know about the Pokémon from various media with the result that each child knows something his or her friends do not and thus has a chance to share this expertise with others" (Convergence Culture, 128). He continues

Children are being prepared to contribute to a more sophisticated knowledge culture. So far, our schools are still focused on generating autonomous learners; to seek information from others is still classified as cheating. Yet, in our adult lives, we are depending more and more on others to provide information we cannot possess ourselves. Our workplaces have become more collaborative; our political process has become more decentered; we are living more and more within knowledge cultures based on collective intelligence. Our schools are not teaching what it means to live and work in such knowledge communities, but popular culture may be doing so.

How do we go beyond the autonomous learner? I've only begun teaching, so I don't have any ready answers. For the past few years I've been experimenting with assignments that require all of the students to work on related topics and make use of online archives of sources. These assignments have been relatively successful: the students can share information with one another, talk over interpretations, and answer each others' questions. At the end of the day, however, they've had to go off on their own and write a paper to hand in ... mostly because I haven't figured out how to assign grades to a collective intelligence in a way that isn't going to get me fired.

Our public history MA program is different from my undergraduate classes because it stresses teamwork in a community context. The students do some individual work but they also contribute to museum exhibits, websites, walking tours and other kinds of joint project. This year, Alan and I have both noticed that there is a new level of cooperation and cohesiveness among the students. I'm sure he wouldn't put it this way, but I think they've been acting more as a collective intelligence. There are two things that we're doing differently. For one thing, the students had a digital history grad class this year where they read a lot about Web 2.0 stuff (tagging, social search, mashups, open source, the economy of reputation, and so on). So they've been exposed to an ideology of collective intelligence and some of the tools that can facilitate it. We're also making much more extensive use of online, collaborative software. The students contributed to a shared archive by digitizing sources, and designed their part of a museum exhibit using a wiki. They've also been blogging and responding to each other online.

I think there are three areas that need more thought. First, the tools for collaboration could be greatly improved. (Personally, I hope that an extended Zotero could serve as the basis for future interaction.) Second, we have to get beyond the idea of autonomous learners. I'm not sure how long it will take the academy to make this transition, but individual academics can surely get involved with projects that teach the skills needed for these new knowledge communities. Third, we have to try to open up academic collectives so that they can mesh with ones outside the academy.

Tags: collective intelligence | pedagogy | public history | rhizome

Collective Intelligence and ARGs

With my digital history grad seminar done for the year, I find myself mulling over two things. One is the idea of histories of the future, the subject of my last post. The other is what the French theorist Pierre Lévy calls "collective intelligence." In Henry Jenkins's formulation, "None of us can know everything; each of us knows something; and we can put the pieces together if we pool our resources and combine our skills" (Convergence Culture). I use Jenkins's version of the idea because he develops it in a fascinating discussion of alternate reality gaming.

In 2001, a secret Microsoft team known as the Puppetmasters put together a new kind of game called the "Beast." The puzzle posed by the game narrative, which would be delivered in fragments via every medium the designers could think of, would require the cooperation of many players to solve. It eventually was solved by the Cloudmakers, a self-selected team of hundreds of players. Jenkins says, "From the start, the puzzles were too complex, the knowledge too esoteric, the universe too vast to be solved by any single player." (For more, see Sean Stewart's introduction to the game, an academic paper by Jane McGonigal, and Cloudmakers.org).

Setting aside the fact that the Beast sounds like it was a heck of a lot of fun, it seems to me that digital history could harness this kind of collective problem solving if framed in the right way. This is already the stuff of fiction. In Vernor Vinge's Rainbows End, for example, computing is ubiquitous, high school students take courses in "search and analysis" and collectives of intelligence analysts work together in swarm-like fashion. One of Jenkins's provocative claims is that games and other media are already teaching children to work together in this fashion to solve problems. "In a hunting culture," he says, "kids play with bows and arrows. In an information society, they play with information."

Tags: alternate reality games | collective intelligence

Histories of the Future

My grad seminar in digital history wrapped up this week with a discussion of "histories of the future." As I explained to the students, I was trying to capture three things with the (syntactically ambiguous) title. In a course about history and computing, I thought it might be nice if we discussed some readings on the history of computing. More generally, I was also responding to an intriguing essay collection by the same title, which looks at a few of the different ways that people in the past imagined what was to come. And finally, I also wanted to provide a space to talk about how history may come to be done in the digital age.

As I've mentioned before, digital history is new enough that there's no real gap yet between the frontiers of research and classroom discussions. So for me, the idea of "histories of the future" represents a problem that I'm struggling with. Many of the histories that are being written right now don't really reflect the present, at least not my present. (In his wonderful Blessed Among Nations, Eric Rauchway uses the metaphor of an eyeglass prescription that no longer makes things clear.) I have the sense that if I could only figure out how to relate the two senses of the phrase "histories of the future," I'd know what history should look like in the present.

Tags: digital history | macdougall, rob: influence of

The Difference That Makes a Difference

I recently had a conversation with some colleagues about a PhD student in our program who is close to finishing his dissertation on a 20th-century topic. One of them expressed concern that he might be embarassed if he didn't consult a particular set of sources, that it would suggest that his research wasn't exhaustive enough. I was reminded of a conversation that I had with my supervisor Harriet Ritvo when I was first starting my own doctoral research.

"How do you know when to stop doing research," I asked her, "Don't you keep finding new sources?" "Of course," she said. "You always find new material. Your research is done when it stops making a difference to your interpretation." Since then, I've found that a number of historians that I admire have a similarly pragmatic criterion. In The Landscape of History, for example, John Gaddis writes, "Some years ago I asked the great global historian William H. McNeill to explain his method of writing history to a group of social, physical and biological scientists attending a conference I'd organized. He at first resisted doing this, claiming that he had no particular method. When pressed, though, he described it as follows:

I get curious about a problem and start reading up on it. What I read causes me to redefine the problem. Redefining the problem causes me to shift the direction of what I'm reading. That in turn further reshapes the problem, which further redirects the reading. I go back and forth like this until it feels right, then I write it up and ship it off to the publisher. (p.48)

If the idea of exhaustive or complete research ever made sense, it doesn't any longer. Around the time that I asked Harriet about how I would know when I had finished my research, I also did a Google search for "Chilcotin," the place I was writing about. I got about 2,000 hits and looked at each one of them. If I were to start the same project today I would find far too much material to wade through. (Google now indexes 543,000 pages that mention the Chilcotin.) More to the point, new material is spidered by Google at an exponential rate, whereas my ability to read through it is increasing in a linear fashion at best. By the time I finished the study there would be far more unread material than when I started it.

One of the consequences of the infinite archive (or of what Roy Rosenzweig calls the "culture of abundance") is that we can't wait to run out of sources before ending a line of inquiry. We can't even pretend to do so. Instead, we have to focus on how much new information we are getting, on average, from what we're learning. Following the work of CS Peirce and William James, Gregory Bateson famously defined information as "the difference that makes a difference." When it stops making a difference, it's no longer information.

Tags: bits | convergence | information | pragmatism

In the Trading Zone

I'm afraid I haven't posted much this month because I've been working on a big grant application with a number of colleagues. One component of the grant is dedicated to digital infrastructure for environmental history, and as part of the grant-writing process I've been required to negotiate a number of partnerships with institutions that are creating digital repositories of various sorts. In an odd reversal, I've been finding it much easier to talk to librarians, archivists, curators and new media specialists than to my own colleagues in environmental history. I now realize that this is because I share a language with the former that the latter have not yet adopted: the language of open source, open access, digital libraries, markup, and web services.

Two other recent experiences have helped me put this in perspective. Yesterday in my digital history grad class we were talking about machine learning and data mining. Some of the students wondered whether we would let available tools and techniques guide research questions, basically agreeing with a concern raised by Dan Cohen earlier in the week. I was reminded of Maslow's remark that "If you only have a hammer, you tend to see every problem as a nail." (The Psychology of Science: A Reconaissance, New York 1966). One of the ways that historians often differ from more theory-driven social scientists is by commitments to holism, nuanced messiness and complexity. Exceptions don't prove the rule; they show that rules are always impoverished.

I've also been reading Fred Turner's new book From Counterculture to Cyberculture (Chicago 2006). Talking about the emergence of an interdisciplinary group of scientists, engineers, and entrepreneurs he draws on ideas from Peter Gallison ('trading zone') and Geoffrey Bowker ('legitimacy exchange'):

Legitimacy exchange helped transform cybernetics from a relatively local contact language suited to the particular needs of scientists in wartime Cambridge into a discourse commonly used for coordinating work across multiple research projects and multiple professional communities. As Bowker suggests, cybernetics facilitated not only the interlinking of research, development, and production activities, but also the development of new interpersonal and interinstitutional networks and, with them, the exchange and generation of a networked form of power. To the extent that members of two or several disciplines could succeed in creating a relatively closed system of interlegitimation, they could make it extraordinarily difficult for nonexperts (i.e., noncyberneticians) to challenge their individual agendas. They could and did stake claims for research funding, material resources, and popular attention. Working together, in pairs and networks, each acquired a legitimacy that none could have had alone without the exchange of legitimacy afforded by cybernetic rhetoric (pp. 25-26).

Gallison's ethnographic metaphor of a trading zone leaves room for the very different understandings that humanities scholars and intelligence analysts will bring to their encounter. In Image and Logic he writes "Anthropologists are familiar with different cultures encountering one another through trade, even when the significance of the objects traded--and of the trade itself--may be utterly different for the two sides. And with the anthropologists, it is crucial to note that nothing in the notion of trade presupposes some universal notion of a neutral currency. Quite the opposite, much of the interest of the category of trade is that things can be coordinated (what goes with what) without reference to some external gauge" (p. 803). So much better to enter the trading zone than to wait and form a cargo cult.

Tags: digital history

Wikis and Philological Criticism

In "Digital Maoism: The Hazards of the New Online Collectivism," Jaron Lanier recently wrote that "Reading a Wikipedia entry is like reading the bible closely. There are faint traces of the voices of various anonymous authors and editors, though it is impossible to be sure." Among other things, Lanier was arguing against the idea of the collective as "all-wise," something that Wikipedia's many other detractors have also tended to concentrate on. What most of these discussions miss is a feature of Wikipedia and other wikis that is, to my mind at least, what makes them so interesting: wikis automatically generate and maintain an extensive philological apparatus that is always available to the user.

It's fitting that Lanier should refer to close reading of the bible. After all, generations of textual scholars honed their critical skills on religious and classical texts, using exactly such 'faint traces' to determine the origin and authorship of sources, relate variants to one another and repair corruption. Such criticism, sometimes known as 'external' or 'lower' criticism was prelude to 'internal' or 'higher' criticism which "render[ed] a verdict ... on the source's significance as historical evidence" [Ritter, Dictionary of Concepts in History, s.v. "Criticism"; see also Greetham's excellent Textual Scholarship.]

Wiki software automatically tracks every single change made to a given page and lists the edits on a corresponding history page. For contested encyclopedia articles where there may be thousands of edits, it is possible to use sophisticated visualization tools like History Flow to study insertions, deletions, rearrangements, authorship, edit wars, and so on. Furthermore, every wiki page is also accompanied by a discussion page where authors and readers can annotate the texts. Both of these features are very valuable for close reading; neither is available in any of the more traditional encyclopedias that I'm familiar with.

Far too much emphasis has been placed on the content of Wikipedia, and not nearly enough on the practices of reading that it supports. Our students should be using wikis to learn the philological underpinnings of their craft, not told to avoid them because they are a 'bad' source.

Tags: history flow | philology | reading | Wikipedia