Internet start-ups are now asking one another: “Are you long tail?”

The phrase was popularised by Wired magazine’s editor-in-chief Chris Anderson, who started the ball rolling last year with a series of talks and a long article called The Long Tail. That caught the imagination of the bloggers, who circulated and applied the meme.

The internet is changing the entertainment business from one that is driven by hits to one that will make most of its money from misses. This is good news for consumers, because it means more choice, and we all like things that will never make the best-seller lists for CDs, books or movies. And although it might sound strange, this “new economics of abundance” is already the basis of the net’s most successful companies, such as Amazon, eBay and Google.

“The day I knew I was on to something was when people saw the applicability in places I hadn’t anticipated,” says Anderson. “It’s become, in some sense, an open source meme. They’re doing a lot of research and feeding it back to me. That’s why I started the Long Tail blog and, unbelievably, it works.”

The long tail is named after the type of power law curve you get when you plot the sales of CDs, computer games and other products, or the popularity of websites, or the frequency of word use in a language. This is where George Kingsley Zipf, professor of linguistics at Harvard, observed it.

To put it crudely, there are a small number of words that appear very frequently: the, of, to, and, a, in, etc. After that, there is a steep decline, followed by tens of thousands of words that appear relatively rarely: palimpsest, lapidary, rodomontade, epalpebrate, and so on. When you graph what is now often called the Zipf distribution, these rare words form the long tail that tapers off to the right.

The Zipf curve turns out to be at least as common on the internet as it is elsewhere. You might suppose, for example, that since all websites are more or less equally accessible, visitors would be evenly distributed. That is not the case. Research by Lada Adamic and Bernardo Huberman showed that, in fact, “the distribution of visitors per site follows a universal power law, characteristic of winner-take-all markets”.

In other words, a small number of sites have tens of millions of visitors each month – Google, Yahoo, MSN, etc – while millions of sites attract only a handful each. Counter-intuitively, the more choices there are, the more extreme the curve becomes. Life isn’t fair. This knocks most naive ideas of competition on the head.

What interested Anderson, however, was what happens when something moves from the physical world of atoms to the digital world of bits. That’s where the “long tail” effect starts to make a real impact.

Anderson points out that, in the real world, there are physical limits on how many titles a shop can stock, or a cinema can show. These are the economics of shortages. For example, an average American movie theatre can only show films that will attract 1,500 people over two weeks, and almost all of those will live within a 10-mile radius. It is hardly surprising that relatively few movies ever get a showing, and cinema distribution has almost no tail at all: you go straight from hero to zero.

More here.