Forget squeezing millions from a few megahits. The future of entertainment is in the millions of niche markets at the shallow end of the bitstream.

IN 1988, A British mountain climber named Joe Simpson wrote a book called Touching the Void, a harrowing account of near death in the Peruvian Andes. It got good reviews but, only a modest success, it was soon forgotten. Then, a decade later, a strange thing happened. Jon Krakauer wrote Into Thin Air, another book about a mountain-climbing tragedy, which became a publishing sensation. Suddenly Touching the Void started to sell again.

Random House rushed out a new edition to keep up with demand. Booksellers began to promote it next to their Into Thin Air displays, and sales rose further. A revised paperback edition, which came out in January, spent 14 weeks on the New York Times bestseller list. That same month, IFC Films released a docudrama of the story to critical acclaim. Now Touching the Void outsells Into Thin Air more than two to one.

What happened? In short, Amazon.com recommendations. The online bookseller's software noted patterns in buying behavior and suggested that readers who liked Into Thin Air would also like Touching the Void. People took the suggestion, agreed wholeheartedly, wrote rhapsodic reviews. More sales, more algorithm-fueled recommendations, and the positive feedback loop kicked in.

Particularly notable is that when Krakauer's book hit shelves, Simpson's was nearly out of print. A few years ago, readers of Krakauer would never even have learned about Simpson's book—and if they had, they wouldn't have been able to find it. Amazon changed that. It created the Touching the Void phenomenon by combining infinite shelf space with real-time information about buying trends and public opinion. The result: rising demand for an obscure book.

This is not just a virtue of online booksellers; it is an example of an entirely new economic model for the media and entertainment industries, one that is just beginning to show its power. Unlimited selection is revealing truths about what consumers want and how they want to get it in service after service, from DVDs at Netflix to music videos on Yahoo! Launch to songs in the iTunes Music Store and Rhapsody. People are going deep into the catalog, down the long, long list of available titles, far past what's available at Blockbuster Video, Tower Records, and Barnes & Noble. And the more they find, the more they like. As they wander further from the beaten path, they discover their taste is not as mainstream as they thought (or as they had been led to believe by marketing, a lack of alternatives, and a hit-driven culture).

An analysis of the sales data and trends from these services and others like them shows that the emerging digital entertainment economy is going to be radically different from today's mass market. If the 20th-century entertainment industry was about hits, the 21st will be equally about misses.

For too long we've been suffering the tyranny of lowest-common-denominator fare, subjected to brain-dead summer blockbusters and manufactured pop. Why? Economics. Many of our assumptions about popular taste are actually artifacts of poor supply-and-demand matching—a market response to inefficient distribution.

The main problem, if that's the word, is that we live in the physical world and, until recently, most of our entertainment media did, too. But that world puts two dramatic limitations on our entertainment.

The first is the need to find local audiences. An average movie theater will not show a film unless it can attract at least 1,500 people over a two-week run; that's essentially the rent for a screen. An average record store needs to sell at least two copies of a CD per year to make it worth carrying; that's the rent for a half inch of shelf space. And so on for DVD rental shops, videogame stores, booksellers, and newsstands.

In each case, retailers will carry only content that can generate sufficient demand to earn its keep. But each can pull only from a limited local population—perhaps a 10-mile radius for a typical movie theater, less than that for music and bookstores, and even less (just a mile or two) for video rental shops. It's not enough for a great documentary to have a potential national audience of half a million; what matters is how many it has in the northern part of Rockville, Maryland, and among the mall shoppers of Walnut Creek, California.

There is plenty of great entertainment with potentially large, even rapturous, national audiences that cannot clear that bar. For instance, The Triplets of Belleville, a critically acclaimed film that was nominated for the best animated feature Oscar this year, opened on just six screens nationwide. An even more striking example is the plight of Bollywood in America. Each year, India's film industry puts out more than 800 feature films. There are an estimated 1.7 million Indians in the US. Yet the top-rated (according to Amazon's Internet Movie Database) Hindi-language film, Lagaan: Once Upon a Time in India, opened on just two screens, and it was one of only a handful of Indian films to get any US distribution at all. In the tyranny of physical space, an audience too thinly spread is the same as no audience at all.

The other constraint of the physical world is physics itself. The radio spectrum can carry only so many stations, and a coaxial cable so many TV channels. And, of course, there are only 24 hours a day of programming. The curse of broadcast technologies is that they are profligate users of limited resources. The result is yet another instance of having to aggregate large audiences in one geographic area—another high bar, above which only a fraction of potential content rises.

The past century of entertainment has offered an easy solution to these constraints. Hits fill theaters, fly off shelves, and keep listeners and viewers from touching their dials and remotes. Nothing wrong with that; indeed, sociologists will tell you that hits are hardwired into human psychology, the combinatorial effect of conformity and word of mouth. And to be sure, a healthy share of hits earn their place: Great songs, movies, and books attract big, broad audiences.

But most of us want more than just hits. Everyone's taste departs from the mainstream somewhere, and the more we explore alternatives, the more we're drawn to them. Unfortunately, in recent decades such alternatives have been pushed to the fringes by pumped-up marketing vehicles built to order by industries that desperately need them.

Hit-driven economics is a creation of an age without enough room to carry everything for everybody. Not enough shelf space for all the CDs, DVDs, and games produced. Not enough screens to show all the available movies. Not enough channels to broadcast all the TV programs, not enough radio waves to play all the music created, and not enough hours in the day to squeeze everything out through either of those sets of slots.

This is the world of scarcity. Now, with online distribution and retail, we are entering a world of abundance. And the differences are profound.

TO SEE HOW, meet Robbie Vann-Adibe, the CEO of Ecast, a digital jukebox company whose barroom players offer more than 150,000 tracks—and some surprising usage statistics. He hints at them with a question that visitors invariably get wrong: "What percentage of the top 10,000 titles in any online media store (Netflix, iTunes, Amazon, or any other) will rent or sell at least once a month?"

Most people guess 20 percent, and for good reason: We've been trained to think that way. The 80-20 rule, also known as Pareto's principle (after Vilfredo Pareto, an Italian economist who devised the concept in 1906), is all around us. Only 20 percent of major studio films will be hits. Same for TV shows, games, and mass-market books—20 percent all. The odds are even worse for major-label CDs, where fewer than 10 percent are profitable, according to the Recording Industry Association of America.

But the right answer, says Vann-Adibe, is 99 percent. There is demand for nearly every one of those top 10,000 tracks. He sees it in his own jukebox statistics; each month, thousands of people put in their dollars for songs that no traditional jukebox anywhere has ever carried.

We’re obsessed with hits, but “misses” make money, too.

People get Vann-Adibe's question wrong because the answer is counterintuitive in two ways. The first is we forget that the 20 percent rule in the entertainment industry is about hits, not sales of any sort. We're stuck in a hit-driven mindset—we think that if something isn't a hit, it won't make money and so won't return the cost of its production. We assume, in other words, that only hits deserve to exist.

But Vann-Adibe, like executives at iTunes, Amazon, and Netflix, has discovered that the "misses" usually make money, too. And because there are so many more of them, that money can add up quickly to a huge new market.

With no shelf space to pay for and, in the case of purely digital services like iTunes, no manufacturing costs and hardly any distribution fees, a miss sold is just another sale, with the same margins as a hit. A hit and a miss are on equal economic footing, both just entries in a database called up on demand, both equally worthy of being carried. Suddenly, popularity no longer has a monopoly on profitability.

The second reason for the wrong answer is that the industry has a poor sense of what people want. Indeed, we have a poor sense of what we want. We assume, for instance, that there is little demand for the stuff that isn't carried by Walmart and other major retailers; if people wanted it, surely it would be sold. The rest, the bottom 80 percent, must be subcommercial at best.

But as egalitarian as Walmart may seem, it is actually extraordinarily elitist. Walmart must sell at least 100,000 copies of a CD to cover its retail overhead and make a sufficient profit; less than 1 percent of CDs do that kind of volume. What about the 60,000 people who would like to buy the latest Fountains of Wayne or Crystal Method album, or any other nonmainstream fare? They have to go somewhere else. Bookstores, the megaplex, radio, and network TV can be equally demanding. We equate mass market with quality and demand, when in fact it often just represents familiarity, savvy advertising, and broad if somewhat shallow appeal. What do we really want? We're only just discovering, but it clearly starts with more.

TO GET A sense of our true taste, unfiltered by the economics of scarcity, look at Rhapsody, a subscription-based streaming music service (owned by RealNetworks) that currently offers more than 735,000 tracks.

Chart Rhapsody's monthly statistics and you get a "power law" demand curve that looks much like any record store's, with huge appeal for the top tracks, tailing off quickly for less popular ones. But a really interesting thing happens once you dig below the top 40,000 tracks, which is about the amount of the fluid inventory (the albums carried that will eventually be sold) of the average real-world record store. Here, the Walmarts of the world go to zero—either they don't carry any more CDs, or the few potential local takers for such fringy fare never find it or never even enter the store.

An entire Tower Records devoted to ‘80’s hair bands!

The Rhapsody demand, however, keeps going. Not only is every one of Rhapsody's top 100,000 tracks streamed at least once each month, the same is true for its top 200,000, top 300,000, and top 400,000. As fast as Rhapsody adds tracks to its library, those songs find an audience, even if it's just a few people a month, somewhere in the country.

This is the Long Tail.

You can find everything out there on the Long Tail. There's the back catalog, older albums still fondly remembered by longtime fans or rediscovered by new ones. There are live tracks, B-sides, remixes, even (gasp) covers. There are niches by the thousands, genre within genre within genre: Imagine an entire Tower Records devoted to '80s hair bands or ambient dub. There are foreign bands, once priced out of reach in the Import aisle, and obscure bands on even more obscure labels, many of which don't have the distribution clout to get into Tower at all. Oh sure, there's also a lot of crap. But there's a lot of crap hiding between the radio tracks on hit albums, too. People have to skip over it on CDs, but they can more easily avoid it online, since the collaborative filters typically won't steer you to it. Unlike the CD, where each crap track costs perhaps one-twelfth of a $15 album price, online it just sits harmlessly on some server, ignored in a market that sells by the song and evaluates tracks on their own merit.

What's really amazing about the Long Tail is the sheer size of it. Combine enough nonhits on the Long Tail and you've got a market bigger than the hits. Take books: The average Barnes & Noble carries 130,000 titles. Yet more than half of Amazon's book sales come from outside its top 130,000 titles. Consider the implication: If the Amazon statistics are any guide, the market for books that are not even sold in the average bookstore is larger than the market for those that are (see Anatomy of the Long Tail).

In other words, the potential book market may be twice as big as it appears to be, if only we can get over the economics of scarcity. Venture capitalist and former music industry consultant Kevin Laws puts it this way: "The biggest money is in the smallest sales."

The same is true for all other aspects of the entertainment business, to one degree or another. Just compare online and offline businesses: The average Blockbuster carries fewer than 3,000 DVDs. Yet a fifth of Netflix rentals are outside its top 3,000 titles. Rhapsody streams more songs each month beyond its top 10,000 than it does its top 10,000. In each case, the market that lies outside the reach of the physical retailer is big and getting bigger.

When you think about it, most successful businesses on the Internet are about aggregating the Long Tail in one way or another. Google, for instance, makes most of its money off small advertisers (the long tail of advertising), and eBay is mostly tail as well—niche and one-off products. By overcoming the limitations of geography and scale, just as Rhapsody and Amazon have, Google and eBay have discovered new markets and expanded existing ones.

This is the power of the Long Tail. The companies at the vanguard of it are showing the way with three big lessons. Call them the new rules for the new entertainment economy.

Rule 1: Make everything available

IF YOU LOVE documentaries, Blockbuster is not for you. Nor is any other video store—there are too many documentaries, and they sell too poorly to justify stocking more than a few dozen of them on physical shelves. Instead, you'll want to join Netflix, which offers more than a thousand documentaries—because it can.

Such profligacy is giving a boost to the documentary business; last year, Netflix accounted for half of all US rental revenue for Capturing the Friedmans, a documentary about a family destroyed by allegations of pedophilia.

Netflix CEO Reed Hastings, who's something of a documentary buff, took this newfound clout to PBS, which had produced Daughter From Danang, a documentary about the children of US soldiers and Vietnamese women. In 2002, the film was nominated for an Oscar and was named best documentary at Sundance, but PBS had no plans to release it on DVD. Hastings offered to handle the manufacturing and distribution if PBS would make it available as a Netflix exclusive. Now Daughter From Danang consistently ranks in the top 15 on Netflix documentary charts. That amounts to a market of tens of thousands of documentary renters that did not otherwise exist.

There are any number of equally attractive genres and subgenres neglected by the traditional DVD channels: foreign films, anime, independent movies, British television dramas, old American TV sitcoms. These underserved markets make up a big chunk of Netflix rentals. Bollywood alone accounts for nearly 100,000 rentals each month. The availability of offbeat content drives new customers to Netflix—and anything that cuts the cost of customer acquisition is gold for a subscription business. Thus the company's first lesson: Embrace niches.

Netflix has made a good business out of what's unprofitable fare in movie theaters and video rental shops because it can aggregate dispersed audiences. It doesn't matter if the several thousand people who rent Doctor Who episodes each month are in one city or spread, one per town, across the country—the economics are the same to Netflix. It has, in short, broken the tyranny of physical space. What matters is not where customers are, or even how many of them are seeking a particular title, but only that some number of them exist, anywhere.

As a result, almost anything is worth offering on the off chance it will find a buyer.

This is the opposite of the way the entertainment industry now thinks. Today, the decision about whether or when to release an old film on DVD is based on estimates of demand, availability of extras such as commentary and additional material, and marketing opportunities such as anniversaries, awards, and generational windows (Disney briefly rereleases its classics every 10 years or so as a new wave of kids come of age). It's a high bar, which is why only a fraction of movies ever made are available on DVD.

‍

That model may make sense for the true classics, but it's way too much fuss for everything else. The Long Tail approach, by contrast, is to simply dump huge chunks of the archive onto bare-bones DVDs, without any extras or marketing.

Call it the Silver Series and charge half the price. Same for independent films. This year, nearly 6,000 movies were submitted to the Sundance Film Festival. Of those, 255 were accepted, and just two dozen have been picked up for distribution; to see the others, you had to be there. Why not release all 255 on DVD each year as part of a discount Sundance Series?

In a Long Tail economy, it's more expensive to evaluate than to release. Just do it!

The same is true for the music industry. It should be securing the rights to release all the titles in all the back catalogs as quickly as it can—thoughtlessly, automatically, and at industrial scale. (This is one of those rare moments where the world needs more lawyers, not fewer.) So too for videogames. Retro gaming, including simulators of classic game consoles that run on modern PCs, is a growing phenomenon driven by the nostalgia of the first joystick generation. Game publishers could release every title as a 99-cent download three years after its release—no support, no guarantees, no packaging.

All this, of course, applies equally to books. Already, we're seeing a blurring of the line between in and out of print. Amazon and other networks of used booksellers have made it almost as easy to find and buy a second-hand book as it is a new one. By divorcing bookselling from geography, these networks create a liquid market at low volume, dramatically increasing both their own business and the overall demand for used books. Combine that with the rapidly dropping costs of print-on-demand technologies and it's clear why any book should always be available. Indeed, it is a fair bet that children today will grow up never knowing the meaning of out of print.

Rule 2: Cut the price in half. Now lower it

THANKS TO THE success of Apple's iTunes, we now have a standard price for a downloaded track: 99 cents. But is it the right one?

Ask the labels and they'll tell you it's too low: Even though 99 cents per track works out to about the same price as a CD, most consumers just buy a track or two from an album online, rather than the full CD. In effect, online music has seen a return to the singles-driven business of the 1950s. So from a label perspective, consumers should pay more for the privilege of purchasing à la carte to compensate for the lost album revenue.

Ask consumers, on the other hand, and they'll tell you that 99 cents is too high. It is, for starters, 99 cents more than Kazaa. But piracy aside, 99 cents violates our innate sense of economic justice: If it clearly costs less for a record label to deliver a song online, with no packaging, manufacturing, distribution, or shelf space overheads, why shouldn't the price be less, too?

Surprisingly enough, there's been little good economic analysis on what the right price for online music should be. The main reason for this is that pricing isn't set by the market today but by the record label demi-cartel. Record companies charge a wholesale price of around 65 cents per track, leaving little room for price experimentation by the retailers.

That wholesale price is set to roughly match the price of CDs, to avoid dreaded "channel conflict." The labels fear that if they price online music lower, their CD retailers (still the vast majority of the business) will revolt or, more likely, go out of business even more quickly than they already are. In either case, it would be a serious disruption of the status quo, which terrifies the already spooked record companies. No wonder they're doing price calculations with an eye on the downsides in their traditional CD business rather than the upside in their new online business.

But what if the record labels stopped playing defense? A brave new look at the economics of music would calculate what it really costs to simply put a song on an iTunes server and adjust pricing accordingly. The results are surprising.

Take away the unnecessary costs of the retail channel—CD manufacturing, distribution, and retail overheads. That leaves the costs of finding, making, and marketing music. Keep them as they are, to ensure that the people on the creative and label side of the business make as much as they currently do. For a popular album that sells 300,000 copies, the creative costs work out to about $7.50 per disc, or around 60 cents a track. Add to that the actual cost of delivering music online, which is mostly the cost of building and maintaining the online service rather than the negligible storage and bandwidth costs. Current price tag: around 17 cents a track. By this calculation, hit music is overpriced by 25 percent online—it should cost just 79 cents a track, reflecting the savings of digital delivery.

Putting channel conflict aside for the moment, if the incremental cost of making content that was originally produced for physical distribution available online is low, the price should be, too. Price according to digital costs, not physical ones.

All this good news for consumers doesn't have to hurt the industry. When you lower prices, people tend to buy more. Last year, Rhapsody did an experiment in elastic demand that suggested it could be a lot more. For a brief period, the service offered tracks at 99 cents, 79 cents, and 49 cents. Although the 49-cent tracks were only half the price of the 99-cent tracks, Rhapsody sold three times as many of them.

Since the record companies still charged 65 cents a track—and Rhapsody paid another 8 cents per track to the copyright-holding publishers—Rhapsody lost money on that experiment (but, as the old joke goes, made it up in volume). Yet much of the content on the Long Tail is older material that has already made back its money (or been written off for failing to do so): music from bands that had little record company investment and was thus cheap to make, or live recordings, remixes, and other material that came at low cost.

Popularity no longer has a lock on profitability.

Such "misses" cost less to make available than hits, so why not charge even less for them? Imagine if prices declined the further you went down the Tail, with popularity (the market) effectively dictating pricing. All it would take is for the labels to lower the wholesale price for the vast majority of their content not in heavy rotation; even a two- or three-tiered pricing structure could work wonders.

And because so much of that content is not available in record stores, the risk of channel conflict is greatly diminished. The lesson: Pull consumers down the tail with lower prices.

How low should the labels go? The answer comes by examining the psychology of the music consumer. The choice facing fans is not how many songs to buy from iTunes and Rhapsody, but how many songs to buy rather than download for free from Kazaa and other peer-to-peer networks. Intuitively, consumers know that free music is not really free: Aside from any legal risks, it's a time-consuming hassle to build a collection that way. Labeling is inconsistent, quality varies, and an estimated 30 percent of tracks are defective in one way or another. As Steve Jobs put it at the iTunes Music Store launch, you may save a little money downloading from Kazaa, but "you're working for under minimum wage." And what's true for music is doubly true for movies and games, where the quality of pirated products can be even more dismal, viruses are a risk, and downloads take so much longer.

So free has a cost: the psychological value of convenience. This is the "not worth it" moment where the wallet opens. The exact amount is an impossible calculus involving the bank balance of the average college student multiplied by their available free time. But imagine that for music, at least, it's around 20 cents a track. That, in effect, is the dividing line between the commercial world of the Long Tail and the underground. Both worlds will continue to exist in parallel, but it's crucial for Long Tail thinkers to exploit the opportunities between 20 and 99 cents to maximize their share. By offering fair pricing, ease of use, and consistent quality, you can compete with free.

Perhaps the best way to do that is to stop charging for individual tracks at all. Danny Stein, whose private equity firm owns eMusic, thinks the future of the business is to move away from the ownership model entirely. With ubiquitous broadband, both wired and wireless, more consumers will turn to the celestial jukebox of music services that offer every track ever made, playable on demand.

Some of those tracks will be free to listeners and advertising-supported, like radio. Others, like eMusic and Rhapsody, will be subscription services. Today, digital music economics are dominated by the iPod, with its notion of a paid-up library of personal tracks. But as the networks improve, the comparative economic advantages of unlimited streamed music, either financed by advertising or a flat fee (infinite choice for $9.99 a month), may shift the market that way. And drive another nail in the coffin of the retail music model.

Rule 3: Help me find it

IN 1997, AN entrepreneur named Michael Robertson started what looked like a classic Long Tail business. Called MP3.com, it let anyone upload music files that would be available to all. The idea was the service would bypass the record labels, allowing artists to connect directly to listeners. MP3.com would make its money in fees paid by bands to have their music promoted on the site. The tyranny of the labels would be broken, and a thousand flowers would bloom.

Putting aside the fact that many people actually used the service to illegally upload and share commercial tracks, leading the labels to sue MP3.com, the model failed at its intended purpose, too. Struggling bands did not, as a rule, find new audiences, and independent music was not transformed. Indeed, MP3.com got a reputation for being exactly what it was: an undifferentiated mass of mostly bad music that deserved its obscurity.

The problem with MP3.com was that it was only Long Tail. It didn't have license agreements with the labels to offer mainstream fare or much popular commercial music at all. Therefore, there was no familiar point of entry for consumers, no known quantity from which further exploring could begin.

Offering only hits is no better. Think of the struggling video-on-demand services of the cable companies. Or think of Movielink, the feeble video download service run by the studios. Due to overcontrolling providers and high costs, they suffer from limited content: in most cases just a few hundred recent releases. There's not enough choice to change consumer behavior, to become a real force in the entertainment economy.

By contrast, the success of Netflix, Amazon, and the commercial music services shows that you need both ends of the curve. Their huge libraries of less-mainstream fare set them apart, but hits still matter in attracting consumers in the first place. Great Long Tail businesses can then guide consumers further afield by following the contours of their likes and dislikes, easing their exploration of the unknown.

For instance, the front screen of Rhapsody features Britney Spears, unsurprisingly. Next to the listings of her work is a box of "similar artists." Among them is Pink. If you click on that and are pleased with what you hear, you may do the same for Pink's similar artists, which include No Doubt. And on No Doubt's page, the list includes a few "followers" and "influencers," the last of which includes the Selecter, a 1980s ska band from Coventry, England. In three clicks, Rhapsody may have enticed a Britney Spears fan to try an album that can hardly be found in a record store.

Rhapsody does this with a combination of human editors and genre guides. But Netflix, where 60 percent of rentals come from recommendations, and Amazon do this with collaborative filtering, which uses the browsing and purchasing patterns of users to guide those who follow them ("Customers who bought this also bought ..."). In each, the aim is the same: Use recommendations to drive demand down the Long Tail.

This is the difference between push and pull, between broadcast and personalized taste. Long Tail business can treat consumers as individuals, offering mass customization as an alternative to mass-market fare.

The advantages are spread widely. For the entertainment industry itself, recommendations are a remarkably efficient form of marketing, allowing smaller films and less-mainstream music to find an audience. For consumers, the improved signal-to-noise ratio that comes from following a good recommendation encourages exploration and can reawaken a passion for music and film, potentially creating a far larger entertainment market overall. (The average Netflix customer rents seven DVDs a month, three times the rate at brick-and-mortar stores.) And the cultural benefit of all of this is much more diversity, reversing the blanding effects of a century of distribution scarcity and ending the tyranny of the hit.

Such is the power of the Long Tail. Its time has come.

‍