Bitly News

bitlynews.com was shut down on October 15, 2012 after two years of continuous operation. Here’s a screenshot of what it looked like on its last day. Thanks to everyone who used the site. Read the original blog post below for an explanation of what it was all about.

I’m a believer in doing side projects as a way to avoid burnout. I think of them as being like Google’s twenty percent time, but at an individual scale.

Recently I was spending some of my side project time tinkering with news.arc, the open source software by Paul Graham that powers Hacker News. I’m not a Lisp hacker and I don’t know the Arc language, so I wasn’t coding new features or doing anything novel—just trying to get my own hosted news.arc instance up and running on Amazon EC2. (Which I did eventually, after tracking down the right version of mzScheme and getting a refresher on the Unix screen commmand.)

Around the same time, bit.ly announced it was running another bit.ly API contest. The contest was open to any submission that made use of the bit.ly API. 1st Prize: a Makerbot Thing-O-Matic 3-D Printer.

Sounded to me like a prime opportunity to hack up something fun.

Automated News from Bit.ly Data

The outcome was a mashup of bit.ly and news.arc: Bitly News.

Bitly News is a news site powered by bit.ly clicks. It works by automatically discovering bit.ly links on Twitter in real time, then posting stories with high bit.ly click activity. Each story’s score is periodically updated with recent stats from bit.ly, so the most clicked-on stories bubble to the top.

As I see it, the primary value of Bitly News is its acute real time nature. Rankings are derived from trailing 30-minute click counts, so the top stories on the front page should be breaking news. Not breaking news in the New York Times sense, but whatever the masses of users clicking on bit.ly links collectively deem to be “breaking”. Seeing what everyone is clicking on right now is just inherently interesting.

This idea is similar but not the same as community-voting news sites like Reddit, Digg, and Hacker News. This article in TechCrunch about Bitly News draws a direct parallel between Bitly News and Hacker News, but I don’t think the comparison is quite apt. (I definitely invited the comparison by using news.arc as the platform for Bitly News, though.) Whereas Hacker News is intensely community driven, my goal with Bitly News was to remove the influence of a specific community from the equation entirely. Instead I wanted a parallel site where news discovery was more decentralized and where the community could be the entire web—or if not the whole web then its viceroy, Twitter.

As it stands, Bitly News is a decent Minimum Viable Product but it has plenty of flaws. One is the lolcats effect. Looking on the front page right now I see celebrity interviews, shopping promos, sports news and the occasional viral photo—exactly the kinds of links that zillions of average Twitter users out there are sharing, but not quite a “Hacker News for the rest of us.” In retrospect, this pop-culture focus was the inevitable result of honing in on popular, trending links, I just didn’t expect the content to be so all over the place. WikiLeaks next to the Red Sox. Tony Hawk side by side with CSI Miami.

Improvements

How can Bitly News be improved?

  • Categories. Separate news stories into topic verticals (Entertainment, Technology, etc). This should help mitigate the lolcats effect. The challenge will be automating the categorization with high accuracy.
  • Other sources of real-time engagement. Expand the story ranking metric to include data points beyond just bit.ly clicks.
  • Social graph weighting. Allow users to login with their Twitter accounts, then overweight stories that have been retweeted by people by they follow.

Thanks

My sincerest thanks to bit.ly for running the API contest. Bitly News was lucky enough to win the robot.