stevenberlinjohnson.com

The Long Tail

Finished Chris Anderson's The Long Tail this morning, and while I've been following the discussion around these issues (and chiming in occasionally) since his original essay was published, I still found the book completely stimulating and fun to read. In addition to the original research that drives so much of his argument, Chris has also included some fascinating stories: the creation of the Sears-Roebuck catalogue; the pro-am movement in astronomy, etc. It also has a very persuasive critique of one of my pet peeves -- the polarization hypothesis generally associated with Cass Sunstein. (Related to our epic serendipity debate from earlier this year.)

There was a review in the Wall Street Journal this weekend that chided Chris for sharing some of the techno-utopian tendencies of writers like Steven Johnson and James Surowiecki, which is damning criticism indeed. And it's true The Long Tail -- like a certain book of mine -- does generally hold to an enthusiastic tone about the trend it is describing. But in Chris's case at least, I find that kind of sweeping criticism irritating. I suspect Chris does believe the that the trend towards long tail distributions in culture is, on the whole, a substantial improvement over the top heavy mass media model that has dominated the twentieth century. But that hardly makes him a techno-utopian. It means that in this one realm of technological development, Chris thinks things are getting better. For all we know, there may be other technological trends that Chris considers more problematic or troubling -- the point is, The Long Tail is not a book about those trends, nor is it a general statement about technology (the way, I imagine, Kevin Kelly's next book will be.)

When I was heavy in the promotion of Everything Bad, I often had people generalize out from my endorsement of popular culture and say things like: "Well, you think everything's just cheery in American society right now..." And I'd invariably have to explain that this wasn't true: I think there are plenty of problems in America right now (the usual suspects: wealth inequality, global warming, our President); I just happen to think pop culture is not one of those problems. So I wrote the book partially to say to our politicians and other cultural authorities: stop worrying about video games -- you've got real problems to deal with. Maybe you should focus on them for a change.

Reading The Long Tail actually made me think that I should have added one additional factor in my description of the forces behind The Sleeper Curve, the trend towards increased pop culture complexity I described in Everything Bad. One of the puzzling things about the Curve that readers occasionally had trouble with is that the trend is towards increased complexity, but not necessarily elevated artistic or intellectual achievement. The content can be silly or gratuitously violent, but the formal techniques used to convey the content have grown, on average, more complex. There's more information conveyed in shorter amounts of time, with less hand-holding from the creators. It occurred to me reading The Long Tail that the general trend from mass to niche can explain some of this increased complexity: niches can speak to each other in shorthand; they don't have to spell everything out. But at the same time, the niche itself doesn't have to become any more aesthetically or intellectually rich compared to what came before. If there's a pro wrestling niche, the creators don't have to condescend to the non-wrestling fans who might be tuning in, which means that they can make more references and in general convey more information about wrestling -- precisely because they know their audience is made up of hard core fans. But it's still pro wrestling. The content isn't anything to write home about, but the form grows more complex. In a mass society, it's harder to pull that off. But out on the tail, it comes naturally.

Fantastic, and dead-on post, from Chris Anderson on "media elitism and the 'derivative' myth":

What are most people actually talking about? Mostly themselves, their friends, their family and things that are more interesting to them and their daily lives than whatever we in the media choose to focus on with our limited resources and space.

I've almost finished The Long Tail, which I've found very interesting and entertaining -- I'll post about it sometime in the next day or so.

Great to see Kevin Kelly's cover story -- "Scan This Book!" in the Times Magazine. It's effectively two essays: one about the future of electronic text in the age of a universal library, and one about the Google book scanning controversy. Kevin's great on the latter, but because I'd been following that story fairly closely already, I almost wish they'd given him a whole feature to riff on what kinds of writing and reading practices will emerge as all these books take on new digital lives. This was just a teaser:

Yet the common vision of the library's future (even the e-book future) assumes that books will remain isolated items, independent from one another, just as they are on shelves in your public library. There, each book is pretty much unaware of the ones next to it. When an author completes a work, it is fixed and finished. Its only movement comes when a reader picks it up to animate it with his or her imagination. In this vision, the main advantage of the coming digital library is portability — the nifty translation of a book's full text into bits, which permits it to be read on a screen anywhere. But this vision misses the chief revolution birthed by scanning books: in the universal library, no book will be an island. Turning inked letters into electronic dots that can be read on a screen is simply the first essential step in creating this new library. The real magic will come in the second act, as each word in each book is cross-linked, clustered, cited, extracted, indexed, analyzed, annotated, remixed, reassembled and woven deeper into the culture than ever before. In the new world of books, every bit informs another; every page reads all the other pages.

Hey, that reminds me a of a certain serendipity conversation I was just having the other day...

Can We Please Kill This Meme Now

Jason linked to this op-ed on the "endangered joy of serendipity":

Serendipity is defined as the ability to make fortunate discoveries accidentally. There's so much of modern life that makes it preferable to the vaunted good old days - better hygiene products and power steering leap to mind - but in these disposable days of now and the future, the concept of serendipity is endangered. Think about the library. Do people browse anymore? We have become such a directed people. We can target what we want, thanks to the Internet. Put a couple of key words into a search engine and you find - with an irritating hit or miss here and there - exactly what you're looking for. It's efficient, but dull. You miss the time-consuming but enriching act of looking through shelves, of pulling down a book because the title interests you, or the binding... Looking for something and being surprised by what you find - even if it's not what you set out looking for - is one of life's great pleasures, and so far no software exists that can duplicate that experience.

I find these arguments completely infuriating. Do these people actually use the web? I find vastly more weird, unplanned stuff online than I ever did browsing the stacks as a grad student. Browsing the stacks is one of the most overrated and abused examples in the canon of things-we-used-to-do-that-were-so-much-better. (I love the whole idea of pulling down a book because you like the "binding.") Thanks to the connective nature of hypertext, and the blogosphere's exploratory hunger for finding new stuff, the web is the greatest serendipity engine in the history of culture. It is far, far easier to sit down in front of your browser and stumble across something completely brilliant but surprising than it is walking through a library looking at the spines of books. With music blogs and iTunes, I've discovered more interesting new bands and albums in the past year than I did in all of my college years. I know radio has gotten a lot worse, but really -- does anyone actually believe that radio was ever more diverse and surprising in its recommendations than surfing through the iTunes catalog or the music sites? It's no accident that BoingBoing is the most popular blog online -- it's popular because it's an incredible randomizer, sending you off on all these crazy and unpredictable paths.

I mean, look at what's on the front door of Kottke this morning: soccer jersey fonts, debate over travel time to JFK, best American fiction poll, funny t-shirt joke, new Google software, Richard Feynman video, Tufte design riff, etc. What's the organizing principle? There is none -- other than Jason's quirky taste -- and that's precisely why so many of us visit his site every day. It takes me thirty seconds to make all those connections by reading Jason's blog. I defy McKeen to walk into a library and find so many weird and diverse and interesting things in an hour of staring at bindings.

Updated a few hours later: Alan Jacobs writes in to the comments section:

My particular situation is that of a scholar, and I think Steven is -- what's the technical term? -- nuts to think that I now have more serendipity than I did before. When I used to rely on print dictionaries or encyclopedias, I would very often forget what I was looking for because, in thumbing the pages, I would stumble across all sorts of interesting words or topics, which would lead me to look up other interesting words or topics, along the way to which I would be distracted by yet other words or topics that I had never seen before.

This is a perfect example of what I was talking about in my original post. There are two primary elements to this situation: first, the random stumbling across of something unexpected thanks to the alphabetical organization of the reference book (and the lack of digital search tools), and then this process of being led to other interesting words or topics thanks to that original swerve. It's true enough that Web's structure has eliminated the first kind of "discovery" -- you don't see alphabetical or Dewey Decimal "neighbors" the way you do walking through the stacks or reading book index. But the Web is far better at the second part of Alan's scenario: following a trail of associations from some original starting point. Pick a given entry in the print Britannica and a given entry in Wikipedia, and try to find ten interesting loosely associated articles published in other venues around the world. It's a thousand times easier to do this on the Web, obviously.

So the question is: is there anything in the online experience that compares to the random discoveries of alphabetical or Dewey Decimal exploration. I would say -- nuts or not -- definitively yes. I read regularly about 20 different blogs or other filters, and each day through them I'm exposed to literally hundreds of articles and clips and conversations and songs and parodies that I had no idea about when I woke up that morning. Many of them I just skim over, but invariably a handful of them will send me off on some crazy expedition from site to site, ushered along with the help of other bloggers, Google, del.icio.us, wikipedia, etc. I'm constantly stumbling across random things online that make me think: what is the deal with that anyway? And then an hour later, I'm thinking: how did I get here? I can't tell you how many ideas that eventually made it into published books and articles of mine began with that kind of unexpected online encounter.

Yes, those initial starting points are filters defined by my initial tastes. But my taste is for surprise and novelty -- and that's what they deliver. Serendipity is not randomness, not noise. It's stumbling across something accidentally that is nonetheless of interest to you. The web is much better at capturing that mix of surprise and relevance than book stacks or print encyclopedias. Does everyone use the web this way? Of course not. But it's much more of a mainstream pursuit than randomly exploring encyclopedias or library stacks ever was. That's the irony of the debate: the thing that is being mourned has actually gone from a fringe experience to a much more commonplace one in the culture. Boingboing has a million readers, for crissakes! Right now, on their front door, we have a study of monkey drinking habits, a roadsite alert sign hacking project, a "news of the weird" story about a German would-be suicide, a re-writing of Robinson Crusoe, a collection of vintage cartoons, a digital mapmaking tool, and so on and so on. And this eclecticism is what you get every day there -- which is precisely why it is the most linked-to blog in the world. Now factor in reading a dozen other blogs of comparable range. This is now the daily information intake for millions and millions of people around the world. Why isn't that an increase in serendipity?

I was pointed to this exchange between Vinton Cerf and NEH Chairman Bruce Cole because it has a nice little back-and-forth about Everything Bad, but the rest of it is even more interesting.

Cerf: I stand in the middle of my personal library trying to remember which book it is that had a particular phrase and I have no way of physically going through these books in the library to figure out which one it was. Even if I couldn't read them online--I mean we're concerned about intellectual property protection, as we should be--but even if I couldn't read them online, if I could just find which one it was and what page it was on, I would be happy to go turn to my personal library or the public library or the bookstore to get the copy of the book. Having things online is vitally important for making knowledge accessible to everybody.

This is exactly I how I feel right now, trying to put the endnotes together for The Ghost Map.

I love this NPR commentary from Dave Weinberger about the etiquette of not keeping up with other people's blogs, and feeling guilty about it when they say, "Hey, why are you asking about how my new car is working out -- didn't you see my post from last week?"

It's not that I never read your blog -- I read it occasionally, intermittently, from time to time. It might be once a week, though it's probably more like once a month, or three months. And I skim to see what's important in your life, and, well, to see if you've mentioned my blog. But I still get a sense of what's going on with you. I just don't want to be quizzed on it!

Following on the heels of last week's roundtable discussion on the future, I make another appearance in Time Magazine this week, this time in the form of an essay defending what they're calling Generation M (presumably for "multitasking.") It's all part of a cover package they've put together on both the perils and promise of kids growing up in the age of iPod, MySpace, et al. I think in some ways it marks a bit of a turning point -- it's precisely the sort of cover package that a few years ago would have been full of scaremongering about the dangers of all these new technologies. But this issue is quite balanced; they certainly voice some concerns about the impact of 24/7 multitasking, which of course is entirely appropriate, but they also talk about the positive aspects. And they gave me an entire page to do my riff.

Great summary by Kevin Kelly on what he calls "Consensus Web Filters". This captures the appeal of these sites perfectly:

How I use these consensus tools: By scanning these lists daily I get a fantastic sense of what the web is reading, and an early glimpse of what will reach the MSM in the next day or so. But most important for me is the large volume of very interesting news that will not become "news." This is the kind of material that is more interesting than random pages but which lacks an appealing hook to place it on the front page of a magazine or even a news website. Often these items are timeless; they don't make the front page because they could be run at any time. But they are more valuable than odd curiosities. Because of the voting, tagging, bookmarking process enough people find the item worthwhile that they rise to notice. What emerges for me is a delightful counter-news, or what we used to call at CoEvolution Quarterly, "news that stays news." I have encountered no other process in the world that is better at surfacing "news that stays news" and "news that will be news" better than these collaborative filtering sites.

Here's a question/idea. I'd really like some automatic way of listing my upcoming appearances (speeches, media happenings, etc) in the sidebar here, but I'd like there to be close to zero maintenance time, since every time I try to do it by hand, it ends up being hopelessly out of date after about two months, because I'm incompetent. Basically, I'd like to be able to enter some event with a date, title, and place, and then have it automatically appear in the sidebar, until the date passes, at which point it would be automatically removed. Upcoming should be able to do this, but it doesn't seem to offer the right combination of public/private events that I'd need. Any other ideas? I would think this kind of calendar plug-in would be extremely useful to lots of people/organizations with blogs, no?

Update: Thanks to Rod for the 30Boxes suggestion. I'd looked at it before when it was in super-early beta, and forgotten to check back. It's perfect for my needs right now -- I'm porting over the upcoming appearance schedule; you can see the results in the sidebar already. Sweet.

Wow. Folksonomies rule. Here are some of the tags that people have used to categorize Everything Bad Is Good for You via Amazon's newish tagging feature:

pop culture (1), td123 (1), not worth Clooneys weight gain (1), gillian (1), 235555555 (1), gggillef (1), t03131235 (1), 23456h74 (1), temple of doom (1), turqoise accessories (1), 3231 (1), dimitris arvanitis (1), 35223 (1), translations for Fever 1793 (1), manhattan project (1)

I think they really nailed it, right? Particularly "turqoise accessories."