The Game Theory of Google Reader

Google Reader is one of top five most-used websites. I literally always have a reader tab open in Chrome. So when I heard that Reader is shutting down on July 1st, I was like

alt text

But after a while I was like

alt text

And now I’m like…

alt text

…because I’ve concluded that I’m glad Google is shutting down Reader.

Why, dear reader? Because of Game Theory (obviously)! By shutting down Reader, Google is breaking a bad Nash Equilibrium which has been hobbling the progress of web content distribution.

Whoa, what does that mean?

Well, “Game Theory” is simply a mathematical examination of the ideal strategy for playing games. Unsurprisingly it often applies to business and other competitive situations.

Let’s consider the canonical example Game Theorists love to use: “the prisoner’s dilema”. In this scenario, the cookie monster and his brother (Charleston) have both been arrested for, you guessed it, home invasion and murder. The cops put them into two separate cells and start questioning them.

“Now look, cookie monster, I’m not going to lie. Our evidence here is weak. If neither of you confess then we’re going to have to take this to trial, and you might both get off. But I’ll offer you a deal. If you testify against Charleston, we’ll give you immunity and five cookies. Unless, of course, he testifies against you in which case you’ll be learning to count to twenty to life.”

The police say the same thing to Charleston. So what is the cookie monster to do? He has two choices: “cooperate” with Charleston and go to trial, or “defect” and rat him out in exchange for sweet cookies (or long jail time if Charleston also defects.) Charleston has the same choices.

To figure out what both should do, we can make a simple matrix:

alt text

Cookie Monster is Player A, Charleston is Player B, and the numbers in each square are the number of cookies each gets in each situation, respectively. (No, the numbers don’t match up; I’m borrowing the diagram.)

Obviously, both monsters are better off if they both cooperate. But that won’t happen, at least not assuming that both care exclusively about maximizing their individual cookie intake.

Why not? Remember that our monsters are in separate rooms and cannot coordinate. Also, notice that each monster is individually best off in the situation where the other monster cooperates but he defects. So if either monster suspects that the other one will be generous and cooperate, his individual narrowly rational best option is to defect. And so he will.

alt text

Now the phrase “Game Theory” might quite appropriately make you think of the movie “A Beautiful Mind”. And if you saw that movie, you might think that Russell Crowe (a.k.a. “John Nash”) got academically famous for figuring out how to use game theory to pick up women in bars. But actually he got famous for inventing the concept of a “Nash Equilibrium”.

To over-simplify, a Nash Equilibrium is a set of player choices in a game (e.g. “Cookie monster -> defect”, Charleston -> defect" in the prisoner’s dilema) that will produce an outcome which neither player can improve by changing his choice unilaterally.

Take another look at the prisoner’s dilema diagram. The bottom right square is a Nash Equilibrium because, even though it is the worst outcome it’s the best that either player can do for himself. If Charleston thinks cookie monster is going to defect, he makes himself even worse off by cooperating. Same for cookie monster.

So in games where a Nash Equilibrium exists, we can generally expect that it will come to pass.[1][2]

So how does this all apply to Google Reader? Well, the existence of Google Reader has created and trapped us in a bad Nash Equilibrium. In this case, Google isn’t actually a player. Google is literally and figuratively out of the game – they haven’t cared about Google Reader in years and have let it wither on the vine.[3]

Instead, the players are consumers of RSS content and developers of alternative RSS readers.

The developers have two choices,

  • develop a new, better reader
  • do nothing, and let people continue to use Google

And the consumers have two choices:

  • switch to an alternative
  • keep using Google reader

alt text

We end up with something that looks an awful lot like the prisoner’s dilema, but with a few key differences.

It’s still the case that everyone is better off if consumers and developers cooperate on moving to a new reader. But it’s no longer the case that one side gains because the other loses. This is actually a game with two Nash Equilibria (develop, switch) and (don’t develop, don’t switch.) And instead of being played once in a jailhouse, this decision process is repeated every day.

Games with multiple equilibria (which I am of course oversimplifying here) introduce a fascinating question: “given that there’s a better state which both sides can agree is better for them individually how can move to that better world?” Remember that in a Nash Equilibrium, neither side can improve her outcome by changing her move unilaterally. But if the sides can coordinate then it’s as simple as agreeing to change their decisions at the same time.

But in the real world of RSS readers, developers can’t practically coordinate en masse with consumers. Each has to speculate in isolation about what the other will do. So we end up in a world where fear reigns and consumers don’t investigate alternative RSS readers because they fear that the other options are even more primitive. And although developers could make something much better, they don’t want to risk investing the development time and having no one show up (since GR is “good enough” for most people.)

And so for years we’ve been muddling along in the bad Nash Equilibrium of continuing to use GR and letting RSS become “uncool” relative to social sharing (despite being, IMHO, about 10x more useful.)

But when Google shuts down Reader, the whole game changes. Suddenly the bad Nash Equilibrium square is out of play. That means that coordination between developers and consumers is no longer necessary because “staying put” is no longer an option.

The immediate aftermath might be painful, but I predict that we’ll all end up with a much better system.

Also, note that this dynamic is hardly exclusive to GR. It’s all over the tech world: some company develops a product which gets popular; they lose interest and stop improving. But they keep the product alive, and the network effects and switching costs keep people using it even though much better mechanisms are theoretically possible. The two worst offenders I can think of are:

  1. Craigslist
  2. Javascript

Both are terrible but both are extremely widely used. And so we’re stuck in a bad Nash Equilibrium where everyone suffers because it’s too hard to get people to sell their broken furniture on a new website when they can just use Craigslist and it’s too risky for Mozilla to implement Python in the browser when developers might just keep kludging JS.

If Craigslist would just take a lesson from Google and shut down completely, it would be one of the best days ever in tech.

Can you think of better examples of “okay” products that could help the world by just disappearing? Please comment and let me know!

[1] Game theory is complicated, and there are a lot of exceptions depending on circumstances. But simplification usually gets us close enough.
[2] The prisoner’s dilema is one of the saddest abstract constructs I know. In a painful nutshell, it explains why “we can’t all just get along.”
[3] Not because they don’t care per se, but because it’s just not impactful enough relative to their other lines of business

LinerNotes Liveblog #7 — SXSWi

Things in Austin got weird. There was carousing, there was trespassing,  and yes, there was even laser tag. But after seven crazy days in Austin, my SXSW adventure is over. Instead of sharing a small Austin house with ten people (including two different bands), I’m now just sharing a house with my parents as I visit the Bay Area for some meetings.

SXSW was definitely fun, but it was also probably a waste of time.[1] I probably could have seen that coming – one of the clearer lessons of my time working on LinerNotes is to be very skeptical of conferences. I’ve been to several of the bigger ones (mostly because I managed to snag free tickets), and none of them have delivered enough value to clearly justify the time it took to attend, much less the the jaw-dropping ticket prices (two thousand dollars for TC Disrupt? Seriously?)

Still, out of all the hundreds of tech conferences SXSW retains a somewhat mystical reputation for “serendipity,” i.e. that simply by walking around and talking to people one might encounter the investor or business partner one needs to “kick it up a notch.” That’s a tempting offer, since I do believe in serendipity (or rather the darker flip-side that many endeavors will be destroyed by the absence of serendipity.)

My skepticism almost made me follow Danielle Morrill’s advice and skip SXSW, but a few things fell in place at the last minute (i.e. I found a place to stay for free) and so I decided to take the plunge.

And…I was disappointed.

Serendipity did not strike. I’m sure that it did for some people, but I didn’t meet even one new person with whom I plan to stay in close contact. To be fair, I didn’t do much to maximize my luck surface – I only went for the last two days, I didn’t buy a conference badge ($800?!) and I only struck up conversations with 20% of the people I randomly stood next to in lines instead of 100%. Still, the conversations I did have didn’t give me the sense that my digital soul mate was just one forced conversation away.

The basic problem is with the odds. When SXSWi was smaller, it may have been viable to talk to a large percentage of attendees and either meet someone useful or meet someone who could introduce you to someone useful they just met. But now there are 30,000 people at SXSWi. Let’s assume that’s 1.5% of the entire startup world, and let’s assume that the population is representative of the general startup population.  [2]

What are my (very rough) odds of meeting someone useful? Well, I’d estimate that there are probably less than 500 viable co-founders and 500 viable investors for me in the whole world, and LinerNotes is not really at a point where I’d benefit much from any business development relationships or marketing hires. If 1.5% of those people are at SXSW, that’s approximately 30 people at SXSW worth meeting.

If I’m there for two days, I probably come into physical proximity with 300 people in a day or 600 in total. That’s 4% of the conference. 4% of 30 is 1.2, meaning that in the whole two days, I was probably near someone useful about twice.

Now how many conversations with strangers can I reasonably have in a day? If I’m really forcing it, maybe 20. So that’s (approximately) 40 dice rolls with a 599/600 odds per roll of not meeting someone useful. That’s an 93% chance of failure, which implies a 7% chance of meeting someone useful, if I’m aggressively social and if the population is really representative (which it probably isn’t).

So after all that expense, travel, and foregone working time, the odds of serendipity striking are about 1 in 15. Compared to everything else I could do with those resources, going to SXSW for networking is pretty clearly not worth it.

So why does SXSW continue to have a reputation for magical serendipity?

Simple, because only the 1 in 15 people who benefit bother to write about it or tell their friends. No one likes a party pooper and so other than me and Danielle Morrill, few people want to loudly admit to wasting their time in Austin.

To be clear, I don’t regret going. The main purpose of my trip was actually to go to SXSW Music and do market research for LinerNotes, and it was worth sacrificing a weekend of work to have some fun and confirm my suspicions about SXSW. And I did meet up with a few old friends at SXSWi and got to strengthen those relationships.

There are some people for whom SXSW still does make sense (specifically startups that sell to marketers or app makers, or VC’s and later stage startup employees who just want to relax and party.) But now I can say from experience that there are better ways to make new friends than shouting at strangers at a loud, crowded Taco Bell sponsored party.

 [1]  This post only applies to SXSW Interactive. The dynamics of SXSW Music were much different and probably need their own blog post.

[2] This is obviously a rough calculation. These numbers are mostly made up with the goal of being ballpark correct and having errors cancel out. And even very generous assumptions don’t change the overall picture much.

LinerNotes beta launch day 5 – ‘What is LinerNotes?’

Sorry for the hiatus; it was a busy weekend. I participated in the NYC Monthly Music Hackathon, continuing work on my project Videodrome (a HypeMachine/Exfm for music videos.) I’m aiming to release it publicly next month.

Much more importantly, I pressed the button on LinerNotes.com on Saturday night. The beta is now live, publicly accessible, and has gone four days without exploding.

There was a bunch of interesting process involved that I’d like to blog about, but I need to write something else first. I realized over the weekend that I obviously should have started this series by answering one simple crucial question…what the heck is LinerNotes, anyway?

Actually there’s more like three questions:

1)   What is LinerNotes right now?

2)    What will LinerNotes become?

3)    Why bother building something like this in the first place?

I’m going to answer in reverse order.

Why?

This deserves it’s own blog post and I’ll probably write one soon. But the short answer is very simple – because I’m a curious person. I genuinely love learning things. And right now, it’s too hard for me to learn about and find music (and a great many other things.)

Many people are satisfied with whatever pops out of Google or Wikipedia or Pandora. Well, I’m not. I want more. I want a lot more. If you don’t then LinerNotes is probably not for you. At least not today.

LinerNotes is hand-tailored to the way I want to listen to music. I want my experience to be fluid, effortless, and richly contextualized. I want an endless stream of music that really affects me emotionally, and I want to know how that music and the artists who make it fit into the broad and wonderful history of music. I want to spend my time listening and learning and ruminating, not seeking and scrounging and clicking. The more I know about my music, the more I like it. And as hanging out with my friends constantly reminds me, there’s an awful lot I don’t know.

What will LinerNotes become?

I want LinerNotes to bring all information about music together in one place. All of it. But I want to keep it organized enough that you can find any fact or song just by describing it, e.g. “bands from Liverpool in the 50’s.”

LinerNotes is going to become the Virgil to your musical Dante, guiding you from the hell of tired silence up into the golden light of musical rapture. LinerNotes is going to become the Mrs. Robinson to your musical Benjamin, guiding you from music novice to musical master and then going straight up crazy when you try to date our musical daughter.

LinerNotes will be there for you on every step of your music experience, from helping you learn about and remember what you already love to helping you find what you’re going to love next. LinerNotes isn’t loyal to any label or big corporation – we’re only loyal to you.

What is LinerNotes today?

LinerNotes.com is a very early beta right now. It’s not the realization of my grand vision –  it’s just the first version I can show to people without cringing from embarrassment.

Right now, LinerNotes is basically just an overlay on top of Wikipedia, but with some really useful you-friendly enhancements: 

  • It’s condensed. Because we love the Beatles, but we don’t five-pages-of-unbroken-text love the Beatles.
  • We’ve got complete discographies for nearly every artist, including bootlegs and rarities. Plus pages for every song and album, often complete with credits.
  • Our dangerously-advanced search system lets you search using concepts like “people who play the ukulele” or “bands who’s hometown is Manchester”. Press the “hard mode” button on the homepage to give it a try.
  • You can browse genres and record labels to find new bands. Have you seen all the great acts on Jagjaguwar?
  • You can start listening immediately using Spotify or YouTube (click the “Videos” tab.)

We’ve got an extremely long list of additional features we’re going to add on top of all that, but we need your help to decide what to build first.

So welcome to LinerNotes. Please give it a try and leave feedback telling us what you like, what you don’t like, and what your musical experience today is missing.

“Broken Tools” — Live-blogging LinerNotes.com, Day 4

“Equipment in action operates in an inconspicuous usefulness, doing its work without our noticing it. When the tool fails, its unobtrusive quality is ruined. There occurs a jarring of reference, so that the tool becomes visible as what it is:”—Graham Harman [1]

Last live-blog, I made a plan to launch. Just the day before, I talked about how often planning goes astray. And unsurprisingly, it’s only taken two days for my plan to get off track.

This plan is about broken tools. But more broadly it’s about being sucker punched by the unexpectable. Luckily I got off pretty easy this time, but the punches come hard and fast, and they will for as long as I’m in this business.

Why write about it now? Because a lot of stuff has broken in the last few days. I’ve had to spend so much time putting out fires that my self-imposed deadline of “pushing the button” by tomorrow is probably not going to happen.

Some of the breakage has been minor and some has been major. All of it has been completely typical.

Minor example: I got a letter (delayed in the mail) urgently requiring me to scan and email a document. My scanner broke that same day. So I had to waste 20 minutes on the phone with Lexmark’s absolutely dismal customer before tracking down a fax machine in a bodega.

Major: The virus I thought I beat last week has returned, leaving me feverish and mealy-minded. So I’ve been operating at 25% productivity today as I try to force my mind to think enough thoughts to figure out why ElasticSearch has abruptly stopped working.[2]

Both of these inconveniences were effectively unpredictable.  Both siphoned off real time. And like a running back’s shoe falling off mid-stride, they made themselves perilous to ignore and shattered the precious “flow” where technical productivity hides out.

As you saw on Tuesday, I tried to make conservative estimates for how long each step of preparation would take. But I’ve already handily blown through my margins. The problem is that the time a task requires is not normal distributed – it’s heteroskedastic. In other words, nine out of ten times it will take me 15 minutes to get to Union Square. But the tenth time the train will break and it will take an hour. So when I’m deciding how early to leave for a meeting, my only options are to leave an hour early (and spend hours each month alone in Starbucks, guarding my seat from 14 year old girls with their cinnamon spice lattes), or to accept that once in a while I’ll be offensively late.

Doing the former is the equivalent of buying so much health insurance that you can’t afford to eat. It’s just not practicable. On the other hand, the person you’re late for will really be offended: people tend to weight one negative interaction about as much as a dozen positive ones. I once heard noted VC Brad Feld say that he would not invest in a company unless every interaction was better than the previous one.

But think about the paradox there: the only teams that never fail are either much too conservative, absurdly lucky, or outright lying.

For me, lying is automatically off the table. And my luck is stubbornly resistant to dark magic. But I haven’t figured out another good solution. It pains me to miss my launch target, especially so soon after I’ve posted it publicly. But the alternatives (don’t be transparent, or power through and make myself sicker) seem even worse.

Entrepreneurs build with fragile tools. They’re all we can afford. But earning the confidence of employees and partners seems to require projecting a superhuman façade.

How do you reconcile that tension?

[1] I wanted to start this post off with a quote from philosopher Martin Heidegger, since I’m stealing his idea about tools. But I couldn’t find a direct quote that would require less than 500 words of jargon unpacking, so enjoy this secondary source.

[2] I skipped yesterday; these posts are taking about 75 minutes / per, so a rigid daily cadence is clearly not going to happen.

[3] Backward incompatible version upgrade, plus forgetting to update the cluster name in code upon spawning a new cluster.