Introduce data mining to your History course in 2 minutes

I’ve been playing with https://www.mashape.com/, a freemium (credit card required but some free services) no-coding-required data mining analysis tool. I’m pressed for time, and have literally spent less than 5 minutes with it, but like the possibilities. Once you’ve got an account set up, including credit card details, you can produce an introductory ‘Data Mining’ tutorial very easily.

Background

https://www.mashape.com/ offers Application Programming Interfaces (APIs) that perform common text analysis tasks, including the one we’ll use for this tutorial: Named Entity Recognition (NER). Text analysis is a good way to introduce students to data mining. You can use text documents, or a URL.

Purpose

To introduce non-technical historians and students to data mining. The idea is to present students with a purposefully naive example of data mining, working on the assumption it’ll prompt them to critically engage with data mining as a method. The questions they explore are in many ways more important than the results you present them with.

Step 1: Get the NER results

  1. Go to the Fordham Modern History Sourcebook and find a document that works for your class. After blindly clicking I’ve chosen ‘Instructions for the Virginia Colony 1606’. Copy the URL.
  2. Sign up to Mashup.
  3. Go to the Mashape text analysis API. Scroll down until you find the Named Entity Recognition tool [Figure 1], copy your document URL into the appropriate field and click Test Endpoint [Figure 1].
  4. At this point you’ll need to enter credit card details. Be sure to subscribe to their ‘Freemium’ service! After your credit card has been accepted click Subscribe. You’ll be returned to the Text Analysis API page.
  5. Scroll back down to the Named Entity Recognition tool, copy your document URL into the appropriate field again and click Test Endpoint [Figure 1].
  6. The results of the query will appear. Copy this (I’ve included my results below). You could get your students to sign up for the service and generate the results themselves, but since a credit card is required it’s probably better to just provide it for them. At any rate, this is the content of your tutorial.

Step 2: Produce some questions

So, entity extraction for historians – useful? Yes, but. If you’ve got a body of thousands of documents (or millions of emails, perhaps), it can give you an idea of the type of content and tone of the corpus etc. It might be useful to test commonly held opinions about a given document or corpus that were formed when people could only read a subset because of time / labour constraints. In some cases the document set might simply be too large to analyse manually. It can also help to develop secondary websites that present the content using maps and timelines, or enrich online publications with contextual links.

Different data mining algorithms offer different perspectives, of course. Latent Dirichlet allocation (LDA), for example, returns common topics. The algorithms tend to have been developed for the sciences, so they work well with factual prose, but increasingly poorly the more subjective the text gets (although those cases result in interesting unexpected results anyway). Data mining algorithms benefit from ‘training’ , so the results below are fairly raw.

Questions for students are obvious enough:

  1. ‘Are you confident that the source document used in this tutorial is adequate for our analysis? How could we ensure it’s a perfect copy of the original?’
  2. ‘How useful is data mining to historians?’
  3. ‘Is data mining using an algorithm superior or inferior to manual approaches?’
  4. ‘Should you draw any conclusions from the results without understanding what the data mining algorithm is designed to do?’
  5. ‘Do the results suggest a fully successful analysis, or do some things need tweaking?’
  6. ‘What are the implications of using a machine and algorithms to conduct historical research?’
  7. ‘Will these techniques ever render historians redundant?’
  8. ‘Does it mean students should be being taught how algorithms work, so humanists aren’t reliant on ones designed for scientists?’.
  9. etc

Step 3: Further Reading

Named Entity Recognition and History

Crane, Gregory, and Alison Jones. “The Challenge of Virginia Banks: An Evaluation of Named Entity Analysis in a 19th-Century Newspaper Collection.” In Proceedings of the 6th ACM/IEEE-CS Joint Conference on Digital Libraries, 31–40. JCDL ’06. New York, NY, USA: ACM, 2006.

Further afield

The readings below will get the students thinking about the use of data mining in historical research. Most of the work I’ve come across has used topic modeling and particularly the Mallet tool, which would be a good next step for anyone interested in taking things further. For projects specific to your historical sub-discipline, take a look at http://www.diggingintodata.org/.

Blevins, Cameron. “Topic Modeling Historical Sources: Analyzing the Diary of Martha Ballard.” DH2011 Conference, Stanford University, 2011.

___________. Mining the Dispatch.

Cohen, D., F. Gibbs, T. Hitchcock, G. Rockwell, J. Sander, R. Shoemaker, S. Sinclair, S. Takats, W.J. Turkel, and C. Briquet. Data Mining with Criminal Intent, 2011.

Graham, Shawn, Scott Weingart, and Ian Milligan, ‘Getting Started with Topic Modelling and Mallett‘. The Programming Historian.

Step 4: Next Steps

Experiment with the other text analysis tools offered by Mashape and try different data mining tools, like Mallet.

Results (scroll down to see the entities returned)

HEADERS

Connection: keep-alive

Content-Length: 12883

Content-Type: application/json;charset=UTF-8

Date: Fri, 21 Feb 2014 19:53:04 GMT

Server: nginx/1.4.1 (Ubuntu)

X-Mashape-Proxy-Response: false

X-Mashape-Version: 3.1.11

RESPONSE

{

“text”: “Instructions for the Virginia Colony 1606 < 1600-1650 < Documents < American History From Revolution To Reconstruction and beyond. In the first decade of the seventeenth century England began a second round of colonizing attempts. This time jointstock companies were used as the vehicle to plant settlements rather than giving extensive grants to a landed proprietor such as Gilbert or Raleigh, whose attempts at colonization in the 1570s and 1580s had failed. The founding of Virginia marked the beginning of a twenty-five year period in which every colony in the New World was established by means of a joint-stock company. A variety of motives intensified the colonizing impulse – international rivalry, propagation of religion, enlarged opportunity for individual men – but none exceeded that of trade and profit. The companies were created to make a profit; their in vestments in the colonies were based on this assumption. Early in the 1630’s merchants and investors discovered that they could employ their money in other more rewarding enterprises. After 1631, therefore, no colony was founded by mercantile enterprise, but by that date the enterprisers had left a legacy of colonization that was to endure. In these instructions for the Virginia Company, the power of Spain and the fear derived from past failures invade every line. The detail and precision of the instructions reflect the work of experienced men; Richard Hakluyt, the younger, for example, probably had a hand in writing them.  As we doubt not but you will have especial care to observe the ordinances set down by the King’s Majesty and delivered unto you under the Privy Seal; so for your better directions upon your first landing we have thought good to recommend unto your care these instructions and articles following.  When it shall please God to send you on the coast of Virginia, you shall do your best endeavour to find out a safe port in the entrance of some navigable river, making choice of such a one as runneth farthest into the land, and if you happen to discover divers portable rivers, and amongst them any one that hath two main branches, if the difference be not great, make choice of that which bendeth most toward the North-West for that way you shall soonest find the other sea.  When you have made choice of the river on which you mean to settle, be not hasty in landing your victuals and munitions; but first let Captain Newport discover how far that river may be found navigable, that you make election of the strongest, most wholesome and fertile place; for if you make many removes, besides the loss of time, you shall greatly spoil your victuals and your caske, and with great pain transport it in small boats.  But if you choose your place so far up as a bark of fifty tuns will float, then you may lay all your provisions ashore with ease, and the better receive the trade of all the countries about you in the land; and such a place you may perchance find a hundred miles from the river’s mouth, and the further up the better. For if you sit down near the entrance, except it be in some island that is strong by nature, an enemy that may approach you on even ground, may easily pull you out; and if he be driven to seek you a hundred miles [in] the land in boats, you shall from both sides of the river where it is narrowest, so beat them with your muskets as they shall never be able to prevail against you.  And to the end that you be not surprised as the French were in Florida by Melindus, and the Spaniard in the same place by the French, you shall do well to make this double provision. First, erect a little stoure at the mouth of the river that may lodge some ten men; with whom you shall leave a light boat, that when any fleet shall be in sight, they may come with speed to give you warning. Secondly, you must in no case suffer any of the native people of the country to inhabit between you and the sea coast; for you cannot carry yourselves so towards them, but they will grow discontented with your habitation, and be ready to guide and assist any nation that shall come to invade you; and if you neglect this, you neglect your safety.  When you have discovered as far up the river as you mean to plant yourselves, and landed your victuals and munitions; to the end that every man may know his charge, you shall do well to divide your six score men into three parts; whereof one party of them you may appoint to fortifie and build, of which your first work must be your storehouse for victuals; the other you may imploy in preparing your ground and sowing your corn and roots; the other ten of these forty you must leave as centinel at the haven’s mouth. The other forty you may imploy for two months in discovery of the river above you, and on the country about you; which charge Captain Newport and Captain Gosnold may undertake of these forty discoverers. When they do espie any high lands or hills, Captain Gosnold may take twenty of the company to cross over the lands, and carrying a half dozen pickaxes to try if they can find any minerals. The other twenty may go on by river, and pitch up boughs upon the bank’s side, by which the other boats shall follow them by the same turnings. You may also take with them a wherry, such as is used here in the Thames; by which you may send back to the President for supply of munition or any other want, that you may not be driven to return for every small defect.  You must observe if you can, whether the river on which you plant doth spring out of mountains or out of lakes. If it be out of any lake, the passage to the other sea will be more easy, and [it] is like enough, that out of the same lake you shall find some spring which run[s] the contrary way towards the East India Sea; for the great and famous rivers of Volga, Tan[a]is and Dwina have three heads near joynd; and yet the one falleth into the Caspian Sea, the other into the Euxine Sea, and the third into the Paelonian Sea.  In all your passages you must have great care not to offend the naturals [natives], if you can eschew it; and imploy some few of your company to trade with them for corn and all other . . . victuals if you have any; and this you must do before that they perceive you mean to plant among them; for not being sure how your own seed corn will prosper the first year, to avoid the danger of famine, use and endeavour to store yourselves of the country corn.  Your discoverers that pass over land with hired guides, must look well to them that they slip not from them: and for more assurance, let them take a compass with them, and write down how far they go upon every point of the compass; for that country having no way nor path, if that your guides run from you in the great woods or desert, you shall hardly ever find a passage back.  And how weary soever your soldiers be, let them never trust the country people with the carriage of their weapons; for if they run from you with your shott, which they only fear, they will easily kill them all with their arrows. And whensoever any of yours shoots before them, be sure they may be chosen out of your best marksmen; for if they see your learners miss what they aim at, they will think the weapon not so terrible, and thereby will be bould to assault you.  Above all things, do not advertize the killing of any of your men, that the country people may know it; if they perceive that they are but common men, and that with the loss of many of theirs they diminish any part of yours, they will make many adventures upon you. If the country be populous, you shall do well also, not to let them see or know of your sick men, if you have any; which may also encourage them to many enterprizes.  You must take especial care that you choose a seat for habitation that shall not be over burthened with woods near your town; for all the men you have, shall not be able to cleanse twenty acres a year; besides that it may serve for a covert for your enemies round about.  Neither must you plant in a low or moist place, because it will prove unhealthfull. You shall judge of the good air by the people; for some part of that coast where the lands are low, have their people blear eyed, and with swollen bellies and legs; but if the naturals be strong and clean made, it is a true sign of a wholesome soil.  You must take order to draw up the pinnace that is left with you, under the fort: and take her sails and anchors ashore, all but a small kedge to ride by; least some ill-dispositioned persons slip away with her.  You must take care that your marriners that go for wages, do not mar your trade; for those that mind not to inhabite, for a little gain will debase the estimation of exchange, and hinder the trade for ever after; and therefore you shall not admit or suffer any person whatsoever, other than such as shall be appointed by the President and Counsel there, to buy any merchandizes or other things whatsoever.  It were necessary that all your carpenters and other such like workmen about building do first build your storehouse and those other rooms of publick and necessary use before any house be set up for any private person: and though the workman may belong to any private persons yet let them all work together first for the company and then for private men.  And seeing order is at the same price with confusion, it shall be adviseably done to set your houses even and by a line, that your street may have a good breadth, and be carried square about your market place and every street’s end opening into it; that from thence, with a few field pieces, you may command every street throughout; which market place you may also fortify if you think it needfull.  You shall do well to send a perfect relation by Captaine Newport of all that is done, what height you are seated, how far into the land, what commodities you find, what soil, woods and their several kinds, and so of all other things else to advertise particularly; and to suffer no man to return but by pasport from the President and Counsel, nor to write any letter of anything that may discourage others.  Lastly and chiefly the way to prosper and achieve good success is to make yourselves all of one mind for the good of your country and your own, and to serve and fear God the Giver of all Goodness, for every plantation which our Heavenly Father hath not planted shall be rooted out.”,

“entities”: {

“location”: [

“Virginia”,

“England”,

“Raleigh”,

“Spain”,

“Florida”,

“India Sea”,

“Caspian Sea”,

“Euxine Sea”

],

“keyword”: [

“Instructions”,

“the Virginia Colony 1606 < 1600-1650 < Documents”,

“American History”,

“Revolution”,

“Reconstruction”,

“Virginia Colony”,

“American History From Revolution To Reconstruction”,

“the seventeenth century England”,

“colonizing attempts”,

“jointstock companies”,

“plant settlements”,

“extensive grants”,

“a landed proprietor”,

“Gilbert or Raleigh”,

“colonization”,

“a twenty-five year period”,

“a joint-stock company”,

“New World”,

“motives”,

“the colonizing impulse”,

“international rivalry”,

“propagation”,

“religion”,

“opportunity”,

“individual men”,

“trade and profit”,

“vestments”,

“merchants and investors”,

“mercantile enterprise”,

“the Virginia Company”,

“Virginia Company”,

“experienced men”,

“the younger”,

“especial care”,

“Majesty”,

“the Privy Seal”,

“Privy Seal”,

“God”,

“a safe port”,

“some navigable river”,

“choice”,

“runneth farthest”,

“divers”,

“portable rivers”,

“hath two main branches”,

“North-West”,

“Captain Newport”,

“election”,

“the strongest , most wholesome and fertile place”,

“time”,

“great pain transport”,

“small boats”,

“fifty tuns”,

“ease”,

“mouth”,

“nature”,

“boats”,

“Melindus”,

“this double provision”,

“some ten men”,

“a light boat”,

“sight”,

“speed”,

“warning”,

“the native people”,

“the sea coast”,

“whereof one party”,

“victuals”,

“centinel”,

“mouth”,

“discovery”,

“Captain Gosnold”,

“these forty discoverers”,

“any high lands or hills”,

“twenty”,

“river”,

“boughs”,

“side”,

“supply”,

“munition”,

“every small defect”,

“doth spring”,

“mountains”,

“lakes”,

“[it ]”,

“the East India Sea”,

“the great and famous rivers”,

“Volga , Tan[a]is and Dwina”,

“joynd”,

“the Caspian Sea”,

“the Euxine Sea”,

“the Paelonian Sea”,

“East India Sea”,

“Paelonian Sea”,

“great care”,

“corn”,

“famine”,

“the country corn”,

“pass”,

“land”,

“hired guides”,

“the great woods or desert”,

“the country people”,

“assault”,

“common men”,

“your sick men”,

“habitation”,

“woods”,

“twenty acres”,

“the good air”,

“swollen bellies and legs”,

“a true sign”,

“a wholesome soil”,

“order”,

“a small kedge”,

“some ill-dispositioned persons”,

“wages”,

“exchange”,

“workmen”,

“building”,

“publick”,

“any private person”,

“any private persons”,

“private men”,

“confusion”,

“a good breadth”,

“square”,

“your market place”,

“market place”,

“a perfect relation”,

“pasport”,

“Lastly and chiefly”,

“good success”,

“our Heavenly Father hath”,

“Heavenly Father”

],

“date”: [

“1606”,

“1570s”,

“1580s”,

“1630”,

“1631”

],

“money”: [

“<”

],

“person”: [

“Gilbert”,

“Richard Hakluyt”,

“Captaine Newport”

],

“phone”: [

“1600-1650″

]

}

}

From the Wayback Machine: Do we need a Systems Analysis of the Digital Humanities?

This post was first published at ideasunderground.com on 24 May, 2009. I’ve reproduced it here partly because that blog no longer exists, partly because it’s a lazy-but-efficient way of offering an idea I’ve been mulling over for some time to a new audience, and partly because I’m (sadly, perhaps) still quite taken with it. It fits well with my belief that scholars – especially in a post Edward Snowden world – need to understand the engineered nature of the tools they use in their work (regardless of whether they want to build digital outputs or not). Achieving a robust level of scholarly self-consciousness in the digital age is a challenge that most people have (I suggest) given up on, in the face of technological advance rather than methodological choice or epistemological orientation. This has huge implications for the integrity of future scholarship, but opens up equally fascinating areas for research and analysis.

I’m working with a team of systems analysts at the moment and it has got me thinking about what kind of ‘business intelligence’ digital humanists have at our disposal. The humanities have developed organically, in a kind of ‘conversational’ or maybe dialectical process that remains opaque and resistant to formal analysis, but I’m wondering if the convergence of our disciplines with technology necessitates more than this. I accept that I’m flirting with nonsense here: technology is just a delivery mechanism and needn’t determine future directions in the humanities, so why waste time analysing it? We should just use it to deliver our products. It’s a fair attitude to adopt if you’re more focussed on what I’ll call ‘traditional products’ (books, journal articles etc), but I think it’s a bit short-sighted for those us interested in pressing the digital humanities towards what they might worthily be.

So what am I suggesting? The idea would be to undertake a formal systems analysis of the engineered sub-structure to our new field, combined with an analysis of the ‘interface’ between traditional humanist outputs and this sub-structure, potential new directions made possible by that sub-structure, and problems imposed by that sub-structure. It would perhaps need to be done at a disciplinary level initially, with those findings combined into a meta-analysis of the entire field when we’d gathered enough information. I am sure, for instance, that an analysis undertaken by an English professor would be quite different to one undertaken by a historian. Sounds like an idea that’s far too heavy to get off the ground, eh.

Ignoring the leaden quality of the idea, though, what do I mean by ’sub-structure’? Commercial systems analysts would probably suggest it’s a woolly idea (as it may well be), but the idea would be to offer digital humanists a basic overview of the underlying ‘wiring’ of the internet, including domain name servers, routers, key data centres and ’suchlike’. The next layer of analysis would perhaps be to describe the governing bodies like ICANN which control (for want of a better word) the flow of information across the internet, and point out their interactions with international law, national governments and, by extension, humanists sitting in their offices producing their digital outputs. Wikipedia offers a decent overview and there will be very good descriptions offered in various other places, but I’m suggesting that humanists need to do it themselves as well, and that a collaborative approach would suit such a project quite well. We all know how descriptive analysis tends to track the interests of the inquirer, and I have a feeling humanists could offer some interesting perspectives.

Part of my interest in this topic stems from a 1960 article by Ernest Nagel in Philosophy and Phenomelogical Research titled ‘Determinism in History’, where he brilliantly explores the parallels between tightly defined scientific articulations of a ’system’, and the way that historians conceive historical ’systems’. The article was written in the heat of the history-as-system debates prompted by Marxist class analyses, which claimed that History was a closed and definable system determined by economic forces. As is well known, the idea that history was determined in this way sparked furious debate and eventually led to an unspoken (if not uniformly accepted) consensus that this couldn’t possibly be the case: it revolted human sensibilities to suggest that our destiny was in the hands of impersonal systemic forces. Nagel’s analysis was brilliant because, although arguing against naive determinism in historical interpretations, he was brave enough to dive right in and argue that, properly conceived, History is indeed systemically determined (just like a chemical reaction, no less) but in an indeterminate way. Building on the very small / very large anomalies observed in the quantum world, Nagel noted that although basically deterministic, historical systems were simply too large (or small in some cases) and complex to identify the deterministic forces at work. The article isn’t particularly well known, but it still stuns me to think that at a time when most mainstream historical theorists were doggedly refusing to acknowledge that History was determined to any degree at all, lest Marxists leverage the gap, Nagel was capable of offering a sophisticated analysis which allowed for what is, after all, a basic truism of all systems – historical or otherwise.

So, to my point: humanists now work in the context of a large, complex system, which determines (a la Nagel) their work and their future to an unknown degree. It’s still very early days in the digital humanities, so it makes sense to engage in a formal systems analysis to start to work out to what degree this system imposes itself on our work. Leaden, I know, but I think potentially fascinating too.

References

Nagel, E., 1960. Determinism in History. Philosophy and Phenomenological Research, XX(3), 291-317.

Digital Humanists as Master Builders

I’ve been thinking about the now relatively long-standing debate in Digital Humanities about ‘who’s in and who’s out’ and wondering if there’s an angle we haven’t been considering (by writing ‘we’ this makes an assumption I’m ‘in’, of course, which I have to admit feels both presumptive given there’s a chance I don’t fit someone else’s criteria and odd given I’m a Senior Lecturer in Digital Humanities). My suggestion is that we need to stop only thinking about the specific technical skills a digital humanist needs, and consider the function the (extra)discipline plays in the broader community, as well as the role(s) it is likely to need to play in the future. It’s becoming clearer, even to a natural pessimist like myself, that humanists are only going to build more digital tools, methods and services as the years progress. I’m contacted fairly regularly by people considering starting digital projects but unsure either how to start or what standards are expected. In New Zealand, if they were building a house, they might contact a Master Builder – a builder accredited by a trade organisation that the customer can trust will either do a good job or be held accountable (to some degree) if they don’t. Often they’ll coordinate subcontractors like plumbers and electricians, to make sure everyone does a good job at the right time, or work with a project manager performing that role. Some Master Builders might be old hands who built houses for years but are now a bit creaky and prefer to manage teams of younger builders who do the work under their guidance. The younger builders (quite possibly accredited themselves) are probably keen on the flash new materials and techniques, and possibly more up to date than their older colleague, but as their elder and better he would probably be the person you’d prefer to go to in a pinch.

So what’s my point? My point is that there might be a layer on top of the list of DH skills you need to display to be ‘in’, and that layer might be almost as important as the list itself. If our practice is to be useful to our peers, they (humanities students, teachers and researchers, and perhaps cultural heritage and library professionals who haven’t done much digital work) need to know i] Who to contact if they’re interested in developing a digital project to good standards and ii] That the person they’re talking to will – if they can’t do it themselves for whatever reason – guide them in the right direction. What characteristics might a ‘card carrying’ (that could be a weird idea) DHer exhibit, in no particular order?

  • Membership in ADHO or a Constituent Organisation, and evidence of participation in ADHO or CO events, THATCamps etc.
  • An awarenes of the range of digital tools, methods and services used across the digital humanities community, and the standards required for them. (The layer underneath this is obviously crucially important, but it will shift over the decades and is best discussed within the community, as regularly happens and will hopefully never stop. Geoffrey Rockwell’s suggestions in a recent edition of Humanist prompted this post to some degree, leading to this list overlapping in places).
  • A degree in the humanities, a record of ongoing engagement with one or more fields, and an understanding of the broad intellectual terrain.
  • Evidence of publication in quality scholarly publications. (This one is difficult. It’s important to indicate an understanding of traditional scholarly norms and an ability to implement them to trade standard, but I don’t want to give the impression I’m referring to a narrow range of traditional analog academic publishing options. It’s also debatable, because I know of several people who I doubt have published but I respect very highly as digital humanists. What I can say is that if it isn’t a current expectation it probably will be in the future, at least for practising within universities.)
  • Experience building and consulting on digital humanities projects.
  • An understanding of project management methodologies and software development best practices.
  • An understanding of the requirements digital archiving, preservation, and data management present.
  • An awareness of local and international funding agencies, and an understanding of how to gain funding for digital projects.
  • An awareness of the history of digital humanities and humanities computing.
  • An awareness of significant digital humanities projects and key people in the local and international communities.
  • Engagement with the digital humanities community via trade publications, social media, and online seminars.
  • Update (a fairly fundamental oversight, actually): Understanding and support for community values like the importance of learning to code, inclusiveness, egalitarianism, and the broader DIY ethos.

Lists like this are sometimes embarassingly self-serving, but you get the idea. I guess it boils down to the fact that I’m more concerned about the type of person someone gets when they make that phonecall to engage a digital humanist than the specific skills that DHer might have. If we (only) try to legislate for a narrow range of skills we’ll find ourselves in the same position computer scientists did in the 1960s, when Ceruzzi notes there were heated debates about who could call themselves a ‘computer programmer’ or not. The rate of change was so great back then that every time they identified a skill requirement another would pop up.Although defining a list of trade skills is crucial for training purposes, it’s possibly more important to ensure people who represent themselves as DHers are capable of guiding our colleagues in the right direction. Let’s face it, it’s quite conceivable that someone could engage an expert in TEI or topic modelling and get very poor advice for their digital project on 3D modelling of paper ships. What would be worse for our community: that someone who doesn’t have a specific skill represents us well, or that someone with it makes us all look bad?

References

P.E. Ceruzzi, A history of modern computing, MIT Press; Cambridge, Mass., 2003, p.105.

Speaking Back to America, Localizing DH Postcolonialism

I’ve put off writing this post for a long time, and I’m still not sure about publishing it because I need to keep thinking it through and catching up on some reading on the topic. Perhaps I should also note that readers from New Zealand, Australia, and other ex-commonwealth nations (as well as Britain) might find it bemusing. The backstory is that I’m writing to a North American audience on a topic specific to the digital humanities. Here goes.

When I saw #dhpoco announced I couldn’t get info about it fast enough. My formative intellectual years were spent saturated in postcolonial discourse and I was eager to see it imported into the digital humanities. New Zealand during the 1980s (my childhood) and 1990s (my undergraduate and postgraduate years) was dominated by it. To me it represents the only weapon I’ve had in arguments with racists who abused indigenous people, who complained about the official acknowledgements of the Treaty of Waitangi, who perpetuated historic wrongs. I eventually wrote about postcolonialism in New Zealand in my doctoral thesis, in a chapter on Keri Hulme, author of The Bone People (1984). Postcolonialism underpins what I think is an increasingly (although by no means completely) honourable relationship between Maori and Pakeha, backed by our political and legal system. It provides the intellectual ammunition required to – ever so slowly and inadequately – begin to roll back 150 years of colonial domination. And the job isn’t finished yet, by a long stretch. It never will be. A professor down the corridor from me has written a postcolonial book called The Long Forgetting, pointing out in ascerbic terms the way we’ve managed as a country to forget what happened on our islands, largely in the interests of middle-class geniality. And of course New Zealanders didn’t come up with this approach to postcolonialism on our own. It was informed by theorists across the postcolonial world.

And so to our beloved postcolonial world. My first reaction to seeing what #dhpoco stood for was – understandably according to the colleagues I’ve spoken to – to recoil in confusion. It didn’t represent anything I understood postcolonialism to be, or speak to the postcolonial world(s) I know. Critical Theory, yes. A powerful melange of identity theory and feminisms, queer theories and activist politics, but not ‘my’ postcolonialism. Frankly, my first thought was ‘The US is colonising postcolonialism!’ In the context of recent US invasions of the (actual geographic) postcolonial world this apparent intellectual appropriation was jarring, to say the least. Coupled with what less charitable people view as a global programme of US cultural colonisation it felt at best wrong-headed and at worst an example of (unconscious) neo-colonialism. The baser side of me initally concluded it was simply the intellectual follow-on to Bush’s adventures; an inevitable smothering of postcolonialism by the only remaining superpower. Domination by intellectual appropriation. Then I wondered if maybe I needed to catch up on my reading, that in my years away from academe postcolonial discourse had radically altered. Wikipedia and fleeting conversations with some random colleagues suggested this isn’t the case, leaving me with the view that all I’m seeing is a provincial (I don’t mean that in a pejorative way) US instantiation of the term that requires localization for New Zealand (and probably Australia and the rest of the postcolonial world). Heavy stuff, then, but not overly problematic. I’d be being dishonest if I didn’t admit that #dhpoco prompted a rather hearty WTF when I first realised what it represented, but after more thought I’ve realised all it might actually show is how ignorant I am about US cultural politics.

So let’s reset things. At the risk of appearing hopelessly passive aggressive, rather than merely confused, I want to go on the record by noting that #dhpoco is a wonderful thing, to be celebrated and encouraged. I agree with its values wholeheartedly. DH is largely populated by white, male, straight people. This has to change. Bravo. And as a Pakeha male what right do I have to either speak for indiginous people in my part of the world, or criticize marginal groups overseas engaged in a struggle for their own empowerment? Moreover, on reflection I can see how the antipodean version of postcolonialism might seem a bit radical, a bit exclusive, from the #dhpoco perspective. When a New Zealander thinks about poco in terms of the US, after all, the only possible beneficiaries are Native Americans. Postcolonialism is their tool, the only real tool they have to empower themselves and gain greater legal and political representation. It is inconceivable, in antipodean terms, that a white woman (for instance) would be able to use that tool for her own purposes. But there’s the rub. The differences between #dhpoco and postcolonialism Down Under hinge around 2 issues: timing and the relationship of postcolonialism to intersectionality. The colonisation process started so much earlier in North America, and was so far advanced by the time postcolonialism appeared in the post World War Two era, that Native Americans (apparently – my ignorance might show here, as elsewhere in this post) weren’t able to claim it for themselves. ‘All’ they seem to have gotten (from our antipodean persepective) was the critical theory that went with it. They have to share the most powerful critical tool available to them, in a radically intersectional political and cultural context. New Zealand intellectuals have resisted going down that path because – even though it makes sense in some ways – it would dilute the movement towards postcolonial empowerment for the people that (our) postcolonialism is designed for: colonised people. While there’s an acceptance that the rights of indigenous people to self-determination and political power intersects in fundamentally important ways with the needs of other marginalised groups, that they are natural allies and in many cases people will be marginalised as a result of more than one aspect of their identity, this does not change the fact that a gay man, say, is one of the colonisers rather than colonised. To assert otherwise would be deeply offensive in anything other than purely intellectual terms, wherein the basic critical precepts that exposed colonialism for what it is are generalised in the service of a broader emancipatory project. But even that wouldn’t be news, or viewed as culturally appropriate in most circles. I can say that with some surity because we’ve been having these discussions since the 1970s and I tackled them in my doctoral thesis in relation to a national debate sparked by Keri Hulme’s assertions of what would now be termed ‘intersectional’ identity politics. The intellectual ground is deep, broad, and riven with hidden chasms.

So, time to get out the toolkit: some localization is needed. In fact, it’s urgently needed because we need to ensure that indigenous peoples in our part of the world get to engage with DH as colonised people and have the full benefit of our local postcolonial discourse(s) when they do. The fact is that we’re far enough behind the rest of the world with DH that our indigenous cultures haven’t had a chance to engage with it yet. Ignoring the work that happens in cultural heritage organisations, it’s still a white middle-class activity. I politely submit, and stand to be corrected by someone more schooled in (non-US) postcolonial politics than me, that we reserve the term ‘postcolonialism’ in our part of the world for the cultures who suffered the effects of nineteenth century colonisation and its ongoing associated processes: give them a chance to engage with the digital world on their own terms and in their own time. Let them own postcolonial DH discourse and determine its future in our part of the world. In fact, why not just park it for a while until more appropriate voices than mine can speak up. Maybe they’ll choose to interpret things differently to me. All I can say is that I’m not entirely confident speaking on behalf of all New Zealanders (and perhaps by implication Australians and people in other postcolonial nations) but am equally uncomfortable with the idea that we might undermine indigenous peoples’ future intellectual (not to mention political and cultural) agency if digital humanities is developed without an effort to localize its #dhpoco aspect.

#dhpoco in the antipodes, then, translates (to me at least) as #criticaldh. I run that code in my head whenever I engage with it now. It allows me to support #dhpoco and to draw out aspects of it that will be of huge value to postcolonial cultures when and if they do begin to engage with DH. I can see fundamentally important areas of mutual interest: Critical examination of video games like Sid Meier’s Civilization, archiving and preservation issues related to precious cultural heritage artefacts, copyright and licensing, encoding standards, data modelling, the use of GIS to aid historical understanding, language localization, the development of AI to represent non-Western consciousness….the list is endless. Many of the skills and attitudes needed to engage with these topics in a sophisticated way are being developed within the #dhpoco movement, and it’s crucially important that work continues, but it’s equally important we develop grassroots expertise in our region so we can contribute our own hard-won perspectives. I’ve more thinking to do about this, and hope to produce something more structured, formal, and code-related but I hope it suffices to add one additional caveat to this post – proof that I’ve been mulling over these issues for some time. Three years ago I wrote a post called The Pacific Node, where I finished with:

There’s an element of post-colonialism in this conception of a Pacific-centered digital humanities node, of course. As much as the international digital humanities community are gregariously interested in new ideas and encouraging of equal participation on all levels and by all groups, they cannot conceive of our local needs, because we haven’t articulated them yet. We need to formulate those needs and then engage with colleagues outside the Pacific to bring a new, digital, vision to reality.

The work to build a Pacific Node has hardly started, but I’d like to think the idea represents what the places Down Under have always offered the northern hemisphere: antipodal perspectives.

Requirements for a New Zealand Humanities eResearch Infrastructure

This is the text of a talk given at eResearch 2013, University of Canterbury, New Zealand, July 03, 2013.

I can only offer a very formative overview of this subject here, but I’m keen to at least put it on the radar. As everyone knows, vast amounts of our cultural heritage are either being digitized and put online or being born online, and this has significant implications for the arts and humanities. In particular, it forces us to start increasing our understanding of, and capability with, the engineered technologies that deliver resources to us online. It will always be difficult getting the balance right – we’re never going to be engineers – but we need to start working through the issues. In this talk I’ll give you a quick overview of the international context, try to convey something about what eResearch in the Humanities actually is, describe where we’re at nationally, and suggest some very formative requirements that might help us work out what direction we need to go in.

I find myself complaining about how far behind the rest of the world we are in New Zealand, but sometimes I think I’m being a bit harsh. Humanities Computing has been around for decades, and New Zealand researchers have never really taken a strong interest in it, but the development of humanities eResearch infrastructure is in its infancy everywhere. That said, the United States, United Kingdom, and to some extent Australia, have been building their capability (and their actual infrastructures) for some time now. New Zealand has basically the same component parts that could be used for a Humanities infrastructure as those countries, but the US, UK and Australia have benefited from ongoing strategic conversations that have allowed them to leverage their assets and develop roadmaps far better than we have.

The conversation in the US was given an additional prompt in 2006 with the American Council of Learned Societies’ report “Our Cultural Commonwealth” , which started to identify areas where requirements for the Humanities and Social Sciences differ from the hard sciences and engineering. This was in the context of the development of the HathiTrust Digital Library and mass digitization efforts by Google, Microsoft and the Internet Archive. In 2006 the US NEH established what came to be known as the Office of Digital Humanities, which has funneled significant amounts of funding into both digital humanities and infrastructure and capability-building projects. Australians often suggest they’re behind the US and UK, but you’ve just seen what fantastic work is happening with the HuNI project [the previous talk was by Richard Rothwell, from the Australian Humanities Networked Infrastructure (HuNI) project]. Very few New Zealand humanists are even aware of HuNI, but I suspect they’ll get a wake-up call when they realize what their Australian colleagues have available, especially if high quality research articles start being produced.

What do humanities researchers need in terms of infrastructure? HuNI provides an excellent model for us, and in many ways I’d suggest we should be modeling requirements for our infrastructure from them, but we need to think about the fundamentals for ourselves as well. This is because in broader terms – and this is why cultural change within our humanities research communities will be needed alongside improved infrastructure – we’re talking about systems capable of supporting a fundamental change in the way *some humanities researchers* approach sources.

In brief, that change involves a shift from thinking about their sources (documents, books, images, audio, video etc) as objects, to viewing them as data. This creates a parallel shift from lone scholars engaging with individual objects one by one, to inter-disciplinary teams of researchers analyzing and making connections between data held in a variety of datasets. Not all humanities researchers will work like this, but we need to accept that programmatic access to our humanities datasets will become more and more important for humanities researchers in the coming decades. Upgrading our infrastructure to allow for this, and providing opportunities for researchers to develop the skills needed to undertake programmatic analysis of large datasets, is something we need to start now.

The Office of Digital Humanities in the US, along with the United Kingdom’s Joint Information Systems Committee and funding agencies in Canada and the Netherlands feel the same way, and established an annual ‘Digging into Data’ event to help surface and resolve the issues involved in this shift. The Digging into Data challenge invites teams to choose a large dataset and engage in humanistic research on it, reporting back on their problems and successes. It’s one important way the humanities community can assess what kind of impact this shift towards programmatic analysis is going to have.

In June of last year a report was published detailing preliminary findings based on early Digging into Data projects. Two findings that struck me relate to the data-centric nature of Humanities eResearch: aside from noting the massive opportunities opened up by eResearch, the report authors noted that

The Digging into Data Challenge presents us with a new paradigm: a digital ecology of data, algorithms, metadata, analytical and visualization tools, and new forms of scholarly expression that result from this research.

They also noted that

It is the combination of algorithmic analysis and human curation of data that helps humanists…

Humanities eResearch occupies a fuzzy place in in the broader eResearch landscape. It is both programmatic and subjective; it involves some researchers with solid programming skills and others who are more comfortable working with word clouds; and most importantly, the data is messy. It isn’t only Google Books OCR that’s the problem – most humanities datasets would make a scientist give up after a first look: nineteenth century ships logs, medical records from early twentieth century leper colonies, you name it, it’ll be hard to read, incomplete and possibly inaccurate. We have traditional methods for dealing with issues like that, but the problems are multiplied exponentially when we port the sources into data formats. Curation, services and tools help a lot, but we also need time to identify and work through the issues, and develop appropriate training schemes for academic staff and their graduate students. So where are we at in New Zealand?

Our biggest problem is awareness. New Zealand academic humanists have had little interest in either humanities computing or digital humanities, and because of this simply aren’t aware what opportunities exist, or even what other countries are doing. Because of this our ability to contribute to a national conversation around eResearch infrastructure is low. I suspect most humanists have out-sourced these issues to scientists, social scientists and engineers without understanding just what a big impact decisions made outside their disciplines could have on them. We do have the component parts for an excellent Humanities eResearch infrastructure, though:

  • DigitalNZ has been the envy of Australian colleagues for years now;
  • The National Digital Heritage Archive is ahead of even the British Library in preservation of our digital heritage;
  • We have a range of large-scale digital assets in the National Library’s Timeframes archive, the Appendices to the Journals of the House of Representatives, and Papers Past.
  • A significant government digital archive is about to come online.
  • [It was noted, quite correctly, afterwards that we have a wide variety of other assets as well. The list above merely offers a snapshot of central government assets].

These could all be leveraged, and added to, to great effect if we can come up with a sensible strategy that connects these central government resources with humanities research communities. So, in short order, what we need is:

  • Education: Efforts are underway to get some regular digital humanities training from a Canadian team who are starting to provide global services, but we also need to increase our efforts to develop digital humanities at undergraduate and graduate level in order to offer the next generation of humanists the skills they need to engage with these issues.
  • Capability building: We need to develop more digital projects, and provide guidance and feedback to humanists who would like to start their own. We also need to develop peer review mechanisms for student projects and PBRF (New Zealand’s ‘Performance Based Research Fund’) so we can begin to understand what a ‘quality’ humanities digital output actually looks like.
  • International collaboration will help us develop more quickly. The international DH community is very welcoming, and many opportunities exist to collaborate.
  • We need to start talking to humanities researchers about what they want from eResearch infrastructures. We could make some educated guesses, but as primary stakeholders they need to be consulted.
  • We also need to work with NeSI [the National eScience Infrastructure team who organized eResearch 2013] and central government to turn any wish-list into strategically aligned and well defined requirements that can realistically be implemented.

Above all else, of course, we need funding. And this isn’t funding only for the final infrastructure but for the education and capability building we need to do first. Compared to the final $ amounts required to implement a Humanities eResearch infrastructure this will be paltry, but it’s important we make a start.

Evaluating Scholarly Digital Outputs: The 6 Layers Approach

The topic of appropriate standards for the evaluation of scholarly digital outputs has come up in conversation at my institution (the University of Canterbury, New Zealand) recently and I’ve realised I haven’t got a ready or simple answer, usually replying that such standards are extremely important because we need to ensure scholarly digital outputs attain to the same standards as, say, monographs, but that they’re evolving. The conversations normally don’t go much further than that. This post, then, is an attempt to get my thoughts down on paper so I can point colleagues to a handy url summarising my thoughts. Much of it will merely repeat common knowledge for digital humanists, but might be of interest.
For a start, as someone employed as a Senior Lecturer in Digital Humanities, standards are of ineffable importance to me. While the ‘big tent’ philosophy of the digital humanities (low barriers to entry and a supportive, welcoming community) is of central importance, it’s becoming almost equally important that we are able to justify our existence within the academy to tenure and hiring committees, and of course funding agencies. For the digital humanities to have a future we need to be able to articulate what is ‘good’ quality. Without that, even ignoring the potential effects on the scholarly reputation of the field, there is a danger that our students will go out into the job market claiming to have digital skills that aren’t recognised as useful, or at the necessary level, by potential employers. In short, while we want to encourage all-comers, we also need to initiate people into an evolving set of standards that mark the difference between ‘welcome contribution’ and ‘scholarly output’. While all content is welcome, and no-one is going to be able to produce high quality scholarly outputs in all the different varieties of the digital humanities, there is an expectation that work claiming to be ‘scholarly grade’ needs to meet certain criteria.
I’ve added some references below to get people started. The special edition of Profession is probably the best place to start. I too have an opinion, though. My feeling is that, in simple terms, there are 5 levels of standards met by most digital humanities projects, and a 6th that doesn’t really make the grade at all. This isn’t a hierarchical scale as much as a classification scheme describing types of projects seen ‘in the wild’. Not all digital humanities outputs are intended to be Category 1, for instance. Some, like this blog post, serve a quite different function. Other projects are produced by people just starting out with a new technology, so there is little chance the product will reach a standard required for tenure or review. They might be experienced digital humanists trying out a new method or experimenting with something likely to fail, or they might a beginner learning the ropes. In short, these are ‘layers’ that all contribute in important ways to the digital humanities ecosystem. Each layer has a function, and is in many ways inter-dependent with the others. To denigrate any layer is to undermine our broader purpose. To paraphrase a friend of mine, ‘they are what they are’.

  1. Category 1: The scholar has built the output themselves, or been a key driver in the technical design and build of it. The output has been driven and project managed by the scholar, often with external funding, including a high degree of technical input in both the design and build phases. The output is complex and/or wide-ranging (either in terms of project scope or technical complexity) and a highly innovative contribution to the field. It conforms to accepted standards in both the digital humanities and computer science. Significant and robust review milestones have been used during all phases of the project, including international feedback. Usage reports (where relevant or possible) indicate high engagement with the output from an international audience. The output has gained wide-spread recognition in both the scholarly and digital humanities communities, and perhaps broader media. It is sustainable, backed up, and controlled by good data management standards.
  2. Category 2: The scholar has built the output themselves, or been a key driver in the technical design and build of it (in this category, because the outputs tend to be of smaller scope than Cat.1, the expectation is really that the scholar has built it themselves, or been an integral part of the team that did) . It either conforms to accepted standards in both the digital humanities and computer science, or provides a conscious and challenging departure from them. The product is of limited scope, but represents an innovative contribution to the field and has gained significant recognition in either the scholarly community, digital humanities community, or the broader media. Usage reports (where relevant or possible) indicate high engagement with the output from an international audience.
  3. Category 3: The output has been built by an external service unit or vendor with no technical input from the scholar, but the scholar has been closely involved in the design and build phases, and contributed high quality content of some form (data or text, perhaps). The product conforms to some standards in either the digital humanities or computer science, but these are loosely applied and/or incompletely implemented.
  4. Category 4: The output has been built by an external service unit or vendor with no technical input from the scholar. It does not conform to generally accepted standards in either computer science or the digital humanities. The scholar, however, has provided high quality content of some form (data or text, perhaps) and the product is of use to general users and researchers.
  5. Category 5: This is a catch-all layer for all the wonderful stuff that the digital world enables – the ephemera of digital scholarship. Examples include blog posts, tweets, small contributions to code repositories etc. It’s also the category that suggests a slightly relativistic attitude is needed when considering my categories, because Cat.5 outputs are incredibly important to the digital humanities. They are our flotsam and jetsam, the glue that keeps the community humming.
  6. Category 6: Rarely seen, and generally politely ignored if they are. This category doesn’t conform to any standards, scholarly or otherwise, indicates little or no understanding of current discourses and practices in the digital humanities, and includes poor quality data or content.

This is only my very broad-brush take on the subject. The reason I present it here is that, like in any other field, the really important thing with digital humanities outputs is that the producer of them understands where their output fits within the broader intellectual context. While this won’t always be the case – we always hope that something will come from left-field – it indicates both an understanding of the field, and respect for it. In general, though, I expect that builders of DH outputs have consciously designed and positioned their product within the broader landscape of DH, and understand that there is a broader matrix of standards and expectations alive in the community. Although I’ve noticed that as the field grows only Cats. 1, 2 and 5 tend to get much airtime, it really doesn’t matter which category the final product falls into….unless it’s Cat.6 and even then people don’t tend to get too bothered: it is what it is. I should also note that I’ve referred to “the scholar” in the singular above, but this is rarely the case in DH projects. For a good example of the growing discourse about collaborative authorship, see http://faircite.wordpress.com/.

For further reading on evaluation of DH projects, and links to other resources, see:
Profession 2011, no. 1 (November 2011).
Modern Language Association, “Guidelines for Evaluating Work in Digital Humanities and Digital Media”, http://www.mla.org/resources/documents/rep_it/guidelines_evaluation_digital.
Modern Language Association, “Guidelines for Editors of Scholarly Editions”, http://www.mla.org/resources/documents/rep_scholarly/cse_guidelines.
U. Nebraska Lincoln, “Recommendations for Digital Humanities Projects”, http://cdrh.unl.edu/articles/best_practices.php.
Todd Presner, Evaluating Digital Digital Scholarship, http://idhmc.tamu.edu/commentpress/digital-scholarship/.

HIST 450 Digital History Seminar – Additional Reading

This list is intended as an additional resource for the University of Canterbury HIST 450: History as a Discipline (Honours) class. The Centre for History and New Media (http://chnm.gmu.edu/resources/essays/) maintain another very useful list, many of which are represented below. Some historical method textbooks will also have sections on computing-related issues. The Zotero Digital History group library is another essential resource.

Books

Cohen, Daniel, and Roy Rosenzweig. Digital History: A Guide to Gathering, Preserving, and Presenting the Past on the Web. University of Pennsylvania Press, 2005. http://chnm.gmu.edu/digitalhistory/.

Coppock, J.T., Information Technology and Scholarship: Applications in the Humanities and Social Sciences. Oxford: Oxford University Press for the British Academy, 1999.

Dougherty, Jack and Nawrotzki, Kristen, eds. Writing History in the Digital Age. Under contract with the University of Michigan Press. Trinity College (CT) web-book edition, Spring 2012, http://WritingHistory.trincoll.edu.

Greengrass, M. and Hughes, L., The Virtual Representation of the Past. Digital Research in the Arts and Humanities. Farnham, England ; Burlington, VT: Ashgate, 2008.

Gunn, S. and Faire, L., Research Methods for History. Edinburgh: Edinburgh University Press, 2011.

Harvey, Charles. Databases in Historical Research: Theory, Methods and Applications. Basingstoke: Macmillan, 1996.

McCarty, Willard. Humanities Computing. Houndmills, Basingstoke, Hampshire ; New York: Palgrave Macmillan, 2005.

Schreibman, Susan, Ray Siemens, and John Unsworth. Companion to Digital Humanities (Blackwell Companions to Literature and Culture). Hardcover. Blackwell Companions to Literature and Culture. Oxford: Blackwell Publishing Professional, 2004. http://www.digitalhumanities.org/companion/.

Turkel, William J. & Alan MacEachern. The Programming Historian, (Network in Canadian History & Environment, 2007-2010). http://niche-canada.org/member-projects/programming-historian/ch1.html.

Articles

Brown, Joshua. “Forum: History and the Web: From the Illustrated Newspaper to Cyberspace: Visual Technologies and Interaction in the Nineteenth and Twenty-first Centuries1.” Rethinking History 8, no. 2 (2004): 253–275.

Bullough, Vern L. “The Computer and the Historian? Some Tentative Beginnings.” Computers and the Humanities 1, no. 3 (January 1967): 61–64.

Burton, Orville Vernon. “American Digital History.” Social Science Computer Review 23, no. 2 (Summer 2005): 206 –220.

Casey, Edward S. “Boundary, Place, and Event in the Spatiality of History.” Rethinking History 11, no. 4 (2007): 507–512.

Cohen, Daniel J. “From Babel to Knowledge.” D-Lib Magazine 12, no. 3 (March 2006).

———. “History and the Second Decade of the Web.” Rethinking History 8, no. 2 (2004): 293–301.

Cohen, Daniel  J. et al. “Interchange: The Promise of Digital History”. The Journal of American History, 95.2.

Corbeil, Pierre. “History and the Computer in Canadian Institutions.” Social Science Computer Review 23, no. 2 (Summer 2005): 181 –189.

Dennis, Brian, Carl Smith, and Jonathan Smith. “Using Technology, Making History: a Collaborative Experiment in Interdisciplinary Teaching and Scholarship1.” Rethinking History 8, no. 2 (2004): 303–317.

Ethington, Philip J. “Placing the Past: ‘Groundwork’ for a Spatial Theory of History.” Rethinking History 11, no. 4 (2007): 465–493.

Hillis, Peter, and Bob Munro. “ICT in History Education— Scotland and Europe.” Social Science Computer Review 23, no. 2 (Summer 2005): 190 –205.

Manning, Patrick. “Gutenberg-e: Electronic Entry to the Historical Professoriate.” The American Historical Review 109, no. 5 (December 1, 2004): 1505–1526.

McLachlan, Robin C. D. “Information Technology and Historians.” Social Science Computer Review 23, no. 2 (Summer 2005): 174 –180.

O’Malley, Michael, and Roy Rosenzweig. “Brave New World or Blind Alley? American History on the World Wide Web.” The Journal of American History 84, no. 1 (June 1997): 132–155.

Phillips, Gervase. “The Historian as Software Engineer.” Alt-J 3, no. 2 (1995): 48.

Rosenzweig, Roy. ‘So, What’s Next for Clio?’ CD-ROM and Historians.” The Journal of American History 81, no. 4 (March 1, 1995): 1621–1640.

———. “The Road to Xanadu: Public and Private Pathways on the History Web.” The Journal of American History 88, no. 2 (2001): 548–579.

———.“Scarcity or Abundance? Preserving the Past in a Digital Era.” The American Historical Review 108, no. 3 (June 1, 2003): 735–762.

Sklar, Kathryn Kish, and Thomas Dublin. “History and New Media: Keeping up with the Web, 1997–2008: Women and Social Movements in the United States.” Perspectives on History (May 2009).

Slatta, Richard W., and E. Kalé Haywood. “Enhancing Latin American History Teaching and Research With Computers.” Social Science Computer Review 23, no. 2 (Summer 2005): 152 –166.

Stam, D. C. “Tracking Art Historians: On Information Needs and Information-seeking Behavior.” Art Libraries Journal 14, no. 3 (1989): 13–16.

Turkel, William. “Intervention: Hacking History, from Analogue to Digital and Back Again.” Rethinking History 15, no. 2 (2011): 287–296.

Walsh, John A, and Wallace Edd Hooper. “The Liberty of Invention: Alchemical Discourse and Information Technology Standardization.” Literary and Linguistic Computing 27, no. 1 (April 1, 2012): 55–79.

Follow

Get every new post delivered to your Inbox.