Another reason to open story archives

Written by Adrian Holovaty on January 25, 2005

Dan Gillmor gives plenty of reasons why newspapers shouldn't charge for archives. Here's another, plain and simple: Open newspaper archives are good for the community.

Forget monetization, forget maintaining newspapers' authority, forget being higher than competitors in search rankings. Journalism exists, in its golden ideal, to spread truth and give people information that helps their lives. Journalists should advance that cause as far as possible.

At the newspaper I work for, for instance, we keep our Web archives open in full, dating back to late 1989. Links don't change, we offer a free archive search and we link to old stories frequently, to provide context.

I bring this up because a few weeks ago I saw a cool example of how our free archives had an effect on people. There's a legally blind homeless man in town, and nobody knew how he became homeless -- until an old friend searched for his name on the Web and found our site's coverage of him.

Pretty cool. And I'm sure this sort of thing happens all the time, thanks to open archives.

Helping the community -- making the world a better place -- is why a lot of people went into journalism, believe it or not. Keeping a newspaper archive open increases the chance that journalists' work has an effect on people's lives.


Posted by Jeremy Flint on January 26, 2005, at 12:25 a.m.:

Most public libraries keep archives of newspapers anyway, and they are free. Why drive people away from your site by trying to charge for something they can get for free elsewhere (although not online).

Posted by Jason Salas on January 28, 2005, at 7:56 a.m.: station's site has a fully searchable database of archived news stories we've produced, that lets users lookup legacy news stories. We've also incorporated a "keyword" search that returns tabular results of multimedia exhibits we've posted online (PDFs, streaming video, audio clips, spreadsheets, etc.). And, I'm finishing work on integrating the next version of our search tool to also return tabular data from our community forums and reporter blogs. So it's a pretty hefty mix of data, all arranged nicely so the user gets an organized gallery of information from which to choose.

My company's main competitor, a local newspaper, requires paid membership to search its archive, which is ridiculous, and is one of the key ways we really are miles ahead of them in the market.

Newspapers with poor business models basically forgo their revenue when they implement open archives, and this is the key: proper planning. It's still a theory of mine (and several others) that newspaper will cease to exist in the next 10 years.

Posted by Colin from Bklyn on February 6, 2005, at 3:25 p.m.:

This issue reminds me of a big story our very specialized trade weekly is covering this year: the proposal to make all market data from U.S. stock exchanges completely transparent, or at least more so. Crudely put, if there's a better price at the NASDAQ for my order, the NYSE would have to let me know that. And in theory you wouldn't have stock analysts manipulating the market in a given security anymore, or have traders timing the market, because we would all be able to know what they know when they know it, including having the same historical perspective that they are able to pull up on their screens.

The harsh truth of the news business, however, is that journalists don't control their own content, and they serve two masters: the public and the suits with the marketing MBAs. In my job, I can't have my freelancers getting together and publishing all their work on the Web because getting access to our archives is a major inducement to pay our hefty annual subscription. We sell insider knowledge that's hard to dig up, that most people wouldn't know what to do with, and that therefore costs a lot of money to collect because the limited size of our specialized audience--which drives higher ad revenues on a "qualified subscriber" model--by definition means we don't get economies of scale.

So we should be careful about generalizing about the business. There are general circulation publications for the public and there are specialized publications, and I think this evolving market we're in increasingly favors the latter. I know I have changed my habits radically from the days I used to just read the L.A. Times. Now I go online and enlist the help of bloggers who share my interests, and other sources, to quickly locate the coverage of those interests in a wide range of publications. I have learned that if I want to read about blogging or Brooklyn or Brazil, the NY Times is not the paper to buy: their coverage of both is dumb. But there are other things they do incredibly well--they have great book reviewers and investigative reporting--and I do like to pick up a copy now and then for train and airplane trips. The problem for the Times is that if I find that coverage in a paper with free access is good enough, I'm going to settle for that rather than paying them $2.50 for an article with better writing but the same basic facts.

So I tend to agree with Jason: syndication and smart aggregation will change the game because they enable content producers to target the mosaic of micromarkets that a given reader inhabits. Personally, I want there to continue to be institutions like the N.Y. Times and the Podunk Herald-Bugle. In the case of Times, having all those smart people in one place pooling resources enriches their coverage. I am still willing to pay a dollar a day to access the parts of it I need, leaving the sports page on the subway seat for someone else to read for free because I don't care about that. In the case of Podunk, who else covers the trash tax I have to pay?

So I agree with you, Mr. H. Archives build your microbrands by allowing people to judge the quality of your coverage on a specific topic over time. That way they know to tune into your syndication channel for specific needs. For us, though, it's something of a catch-22. Paid archives mean you rely on the opposite model: Our coverage of topic B is so good this week that you want to pay for the historical archives as well.

We try to build in as much cross-reference as possible to emphasize the value of our historical coverage: "As we reported back in Oct. 2003, yada. Now, that's all changed." That's why we're trying to work out a multilevel plan for access so it's less of an all or nothing proposition for the customer. We have to offer categorized content streams and make money allowing you to subscribe to just the topics that interest you.

I keep arguing we have to release some of the good stuff into the syndication stream and offer different levels of access for different types of reader if we want to increase subscriptions the way our new owners want. And our Net-exposed product can't just be all tease, either. Readers hate that. It has to be useful in itself. We tend to break our stories down into basic facts and conclusions, then ask you to pay if you want the details on which the conclusions are based, much as market research vendors do. You should be able to rely on us for breaking news and a historical timeline of developments and themes for free, then be able to buy from an a la carte menu or decide that you need complete access, or some plan that gives you a little of both.

Like you say: Planning. Sorry, I rambled a bit. But this is a very useful debate. My point: serving the public now means being able to serve micromarkets. Very original, huh?

Comments have been turned off for this page.