The Pothole Paradox

Why Building The Geographic Web Is Hard, and Why It's Worth Doing

(It's been almost exactly a year since we first put up the prototype for outside.in, and this week we've just launched a major new iteration of the site. It seemed like a fitting occasion to write out in longer format some of my thinking about where all this is heading.)

A few months ago, old friends (and neighbors) of mine decided to embark on what was originally going to be a small renovation of their basement. Somehow, over the course of the construction work, the foundation of their 19th-century brownstone was compromised -- to the point where one structural engineer advised that they move out of the house until the problem had been fixed. There were a few days of panic, but after the contractor did some work shoring up the basement, my friends were told that the house had been temporarily stabilized -- assuming that there were no abnormal vibrations or earthquakes in the vicinity. (Living in Brooklyn, earthquakes were not exactly in the forecast.) And in another month or two, when the final work was done, the house would actually be more stable than it had originally been.

But then, just the other day, while chatting with a neighbor, my friend learned that the city was days away from starting a major sewage pipe replacement project on their very block, closing down the street with jackhammers and diggers carving out a ten foot hole in the pavement. In other words, a veritable symphony of abnormal vibrations thundering through their house for six straight weeks.

There's a moral to this story -- and not just the one about the importance of hiring the right contractor. What I think is so instructive in this sequence of events is the news value of that information about the city's construction plans for the street. For my friends, that little nugget of information was arguably the single most vital headline they could have possibly read that week, far more important than anything going on in Iraq, or in the U.S. campaign season, much less in Britney Spears' custody battle. It was news that had significant financial and safety implications for their entire family. And yet despite its urgency, the news had arrived on their doorstep via the word-of-mouth network of two neighbors gossiping with each other.

All of this would seem to reinforce an observation Chris Anderson (of Wired and "The Long Tail" fame) made a few months ago on his blog, in what he called the "vanishing point theory of news":

... our interest in a subject is in inverse proportion to its distance (geographic, emotional or otherwise) from us. For instance, the news that my daughter got a scraped knee on the playground today means more to me than a car bombing in Kandahar.

You would think that I would whole-heartedly endorse Chris's vanishing point theory, having spent the last year of my life helping to build outside.in, a site dedicated to hyperlocal news and conversation. And in many ways I do -- the passion and expertise that we all have for the things geographically close to us has always been the animating idea at outside.in.

But I think there's a complicating factor here, one that explains why hyperlocal has up to now been a difficult problem to solve. (And it explains some of the choices that we've made over the past year at outside.in.) I started calling it "The Pothole Paradox" earlier this year, using another street construction metaphor, well before my friends ran into trouble with their foundation and the jackhammers. The Pothole Paradox goes like this:

1. Say you've got a particularly nasty pothole on your street that you've been scraping the undercarriage of your car against for a year. When the town or city finally decides to fix the pothole, that event is genuinely news in your world. And it is news that you'll never get from your local paper, or TV affiliate, or radio station.

Obviously this is a great opportunity for a site like outside.in, where news of pothole repairs might easily trickle up from neighborhood bloggers. But it's not that simple, alas -- there's a flip side to the pothole paradox:

2. News about a pothole repair just five blocks from your street is the least interesting thing you could possibly imagine.

To use Chris's vanishing point framework, my kid scraping his knee at the playground might well be news in my household (though knowing my boys' habits at the playground, it would be pretty repetitive news) but news about Chris' kid scraping her knee is as boring to me as, well, pothole repair five blocks away from my house.

The pothole paradox plays out with any number of different topics. The delicious Indian place that at long last opens up in your neighborhood; the creepy science teacher who finally retires at the local public school; the come-from-behind victory staged by the middle-school lacrosse team -- all of these are potentially exciting events if they are happening in the communities you inhabit, but they are mind-numbingly dull if they're one county over -- much less on another coast.

The other complication here is that the correct scale of hyperlocal news varies depending on the nature of the news itself. Pothole repair may die out beyond a few blocks, but many happenings -- crimes or political rallies or controversial real estate development -- reverberate more widely. Going local sometimes requires that you zoom in all the way to the block level, even all the way to the individual address. But sometimes you need to zoom out too.

So how do you get around the pothole paradox? At outside.in, we believe the answer is to build an information system modeled not after traditional newspapers or search engines, but rather the way that people intuitively think about the communities they live in. First, people have an extraordinary innate capacity for organizing their world spatially, which is precisely why pothole repair five blocks away is not interesting to us. And part of that spatial organization involves anchoring people and events in specific places. Think about the people you know socially, and the implicit place-based social networks that you carry around in your head: these are the folks I know from the local school, and these are the ones I know from the coffee shop, and these are the ones from my office...

So our guiding assumption with outside.in has been that a successful hyperlocal service has to be structured from the ground up around those two spatial properties:

1. Every single piece of data in the system has to be associated with some kind of machine-readable location, as precisely as possible.

2. Where appropriate, data should be associated with a human-readable place (a bar, school, playground, park, or even neighborhood.)

The idea of requiring geographic metadata for information might strike some people as excessive, but I suspect in a few years we will look back at the first decade of the web and be amazed that we went for so long without it. Think about it this way: both email and the Web depend on standardized location information embedded in every document -- we call them email and Web "addresses" for a reason. It's a virtual location, of course, but without that universally recognized location data, the last fifteen years of online innovation would have never happened.

We are also increasingly standardizing metadata for time. One of the things that is not commonly said about the blogging revolution is that it has introduced machine-readable time stamps for billions of web pages. One of the things that made Blogger such a breakthrough product was not that it made it easy to put up a web page and publish your thoughts -- home page building tools had been doing that for years -- but rather the fact that it let you publish a reverse chronological list of your thoughts.

So we have widely adopted metadata for virtual location and for time. We just haven't made the same breakthrough for real-world location. This has resulted in a strange imbalance in the way we interact with information on our screens, and in our expectations about what should be readily available to us. Thanks to Technorati (and, during book review season, Google News) I regularly sit down at my browser and issue this command: Show me everything that has been said online about "The Ghost Map" over the past two weeks. We now take it for granted that this kind of query will generate comprehensive results, though it would have been unthinkable just ten years ago.

We're able to take such miracles for granted in part because we have standardized virtual locations and time stamps. But because we don't have the equivalent for geography we're not able to ask what should be much simpler questions:

Show me everything that has been said about schools within one mile of my house.

Or, in the case of my friend with the compromised foundation:

Show me everything that has been said about construction projects on my block.

Geocoding lets you get around the limitations of the pothole paradox, because you can adjust the geographical scale of your query with ease. And, of course, you can do a thousand other things that are impossible to do in systems that lack proper geographic metadata. The new design we launched this week at outside.in includes a Places tab on the front door of each major city in the U.S.; click on it and you see an overview of the most-discussed places in that city, with sparkline graphics showing the post activity over the past 30 days. You can subscribe to an RSS feed for all stories tagged "schools" in your neighborhood. Or you can just move from neighborhood to neighborhood exploring the latest buzz in each community as you go. You can imagine the roadmap we have planned for the next few months at outside.in: exploring social networks by place, mapping discussions, triggering email alerts by proximity, mobile applications, and so on. Once you establish that geographic database, and arrive on a set of standards for geo-tagging, there's almost no limit to the number of interesting new interfaces you can build.

I started writing this little essay in Brooklyn, but I'm finishing it in Barcelona. The day I arrived I wandered out into the Born neighborhood, a kind of Soho grafted onto a medieval street layout. There's a distinctive feeling you get walking around a new city on your own -- the guide books and review sites can tell you where the best restaurants and bars are, and give you the architectural history. But there's always a feeling that you're missing something, that the neighborhood is filled with another kind of data: all the debates and rumors and breaking news that make up the real social information of a community, from the true experts. Right now that layer is almost inaccessible to us -- assuming we can't always sit down and talk to an actual neighborhood maven. We can search a million servers scattered across the globe for a specific text string and get results within seconds. But we can't do a search that tells us what people are saying about the street we're currently standing on. It's about time we changed that.

Comments

[John Beaty]

part of the problem is that it's not intuitive what "near" means. In the case of the pothole, 5 blocks may indeed be too far, unless it's on my route home. And your daughter scraping her knee is essentially irrelevant, unless I'm collecting stories about hurt children to help the parks dept. fix a problem. So, I need to be able to ask "What affects me" and not restrict TOO much the range of the question. But I love the idea, and signed up today for outside.in.

Best wishes,

John

[Zelnox]

Interesting. I think most of what gets twittered fall in the pothole paradox until there are like minded people that follow each other. The relevancy may not be geographical.

There was a twitterquake last week in the wake of a real earthquake in California. I live pretty far from California, but I cared, b/c I follow these folks and read their blogs. It was also a bit amusing how some people mentioned it was their first earthquake experience. Had I read about the earthquake via a mainstream news outlet the next morning, I think it would be complete pothole paradoxy to me.

[Miquel]

@John:

I think what you mean was told by Steven at his recent conference in Barcelona.

The relevance of the information is not only perceived by geographical distance, but also by "emotional distance".

Both concepts mixed gives a more accurate view of what is important.

[Michael Wood-Lewis]

Excellent post Steven. We've been using the same example of potholes here in Burlington, VT for some time now to explain why so many people have subscribed to our service and use it to post to their neighbors (more than 25% of Burlington uses Front Porch Forum).

Which raises a question about your thoughts... who supplies all this geo-specific data? Currently, it seems to be a narrow slice off the top... tech savvy, young, online extroverts, etc. Not too many everyday folks. It's a version of the 1% Principle.

We're thrilled with our early results in this arena too... in one sample FPF neighborhood forum, a full 50% of the participants posted in the past six months.

Congratulations on Outside.in's progress!

[Gary Cook]

Interesting post. I was thinking about folksonomy clouds and how an individual builds up a cloud of their links, interests etc. If people built up custom clouds relevant to their usual locations, adding their interests, maybe importing clouds from other people, you news could be filtered according to those parameters. Obviously this would never actually work, I am, in fact, wasting three minutes typing this, I just had to type it to see how ridiculous the idea was.

[Robert]

Interesting stuff. I spend my time working at a wireless sensor networking startup in the bay area. I get a distinct feeling that the worlds of sensor networks and hyperlocal information will collide in the next few years. With today's technology I could deploy a sensor network in an hour to sense just about anything - but it would be pretty expensive 'industrial grade' gear. Tomorrow I may be able to walk down to Home Depot and buy a network that gives me remote awareness of what's happening on my street, in my garage, or in my neighborhood. It can get sorta creepy if misused, but sites like outside.in would hopefully let the technology serve the good of the local community.

[Hector]

Outside.in featured Digg/Stumble for some extra exposure!