Tuesday, March 16, 2010

A Trick Of The Tail

The New York Times published an article on March 16 titled "Fending Off Digital Decay, Bit by Bit", which talks about the challenges involved in preserving digital source material.  Reading this brought back memories of discussions I have had with several people concerning my misgivings about the enthusiastic embrace of the "long tail" in electronic commerce.

When I first heard about the aforementioned tail used to describe the impending cornucopia of digital goods promised by the Internet, I had some nagging doubts. Perhaps it was my usual reaction to hyperbole, but it might have had something to do with ignored economic realities.  Wikipedia captures the problem concisely:
"Given a large enough availability of choice, a large population of customers, and negligible stocking and distribution costs, the selection and buying pattern of the population results in a power law distribution curve, or Pareto distribution."
Most of the popular "long tail" evangelism focuses on the first part of that sentence, and ignores the next clause: "negligible stocking and distribution costs". The long tail acolytes assumed zero or very low costs because, "bits are cheaper than atoms". Turns out that is only partially true, and completely false if the bits "degrade".

In physical inventories, there is a cost of maintaining a stock of anything. This comprises the cost of the space to house the good and any environmental treatment to maintain it. Then add in the cost of accessing it, packaging it and delivering it (what we all complain about when we see "shipping and handling").

For digital goods, these costs may be lower, but they are not zero. There is the cost of space on a storage medium, and the cost of power to keep that medium active and the bits accessible. Movement to non-rotating storage will reduce the holding cost, but not the access costs. There may be packaging costs (licensing a particular format), and there are delivery costs (currently implicitly, and asymmetrically, shared by consumers and producers, a topic for another post). For most suppliers, these costs are hard to make explicit because consumers assume that bits are free.

As the NYT article points out, digital inventory ages, but in a different way than physical inventory. Bits generally don't become obsolete, but their media, formats and environments do. This is a much more insidious problem than rotting fruit, which can be replaced by a like copy with full confidence that human mouths have not changed. In fact, the aggregate cost of transcoding and migration to current media can make the real cost of older digital content prohibitively expensive, if the price reflects true cost. The only other option is for the vendor to eat the fixed cost and suffer reduced margins. 

Even if this is possible, the bits may not be useable in the current environment. I recently threw away a few hundred CDs of "educational" software that were perfectly readable, but also perfectly useless because it was not economical to recreate the operating system environment they demanded. If I were a vendor, it would be just as silly to keep these CDs in stock, except as drink coasters. More likely I would trash them, and their ripped and rotating counterparts.

Today, if you hear someone gushing about digital content enabling "the long tail", you are probably listening to someone who has never had to manage inventory to make a profit. I would put the long tail diagram in the same category as the Laffer curve - something that looks nice on a napkin, and would be great if only it worked in the real world.