Last Friday, The Simple Dollar was lucky enough to have a post receive a front page posting on digg, an extremely popular tech link aggregation site (here’s the actual digg post itself, currently at 2,051 diggs). Subsequently, it spent a day as the most popular link on del.icio.us (a social bookmarking site) and then it appeared on the front page of Lifehacker, an extremely popular blog focusing on personal productivity. Needless to say, as a nearly anonymous site, the attention and subsequent surge in traffic was a pleasant surprise and I really appreciate it.
Since then, I’ve been asked by many people what exactly the traffic surge was like, how it affected my site, and what lessons I learned from the experience. Even though this is pretty far from the topic of personal finance, I thought I would answer all of these questions with a detailed post reviewing the experience.
First of all, a graph:
This graph shows the accesses of The Simple Dollar, starting at 12:01 AM on December 1 (last Friday) and ending at midnight on December 4 (last Monday). The overall graph indicates total file accesses, while the red color indicates page accesses and the blue portion indicates other file accesses (images, CSS, and so forth). The point marked A indicates the first hour during which my post appeared on digg; point B indicates approximately when my post popped up on del.icio.us, and point C shows the point when my post topped the front page of Lifehacker.
If you would like to download the raw numbers for yourself, you can download this data here.
Before the boom As you can see, my site was quite obscure before the linkage. I was receiving at best 1,000 accesses an hour prior to the appearance of the site on digg, which is admittedly a pittance. Thus, the first hour saw a roughly 8,000% increase in typical hour-by-hour traffic to the site, an incredible amount.
The digg spike At about 6 PM on Friday evening, my post appeared on the front page of digg. This single posting saw my page accesses increase from 291 in the hour before the linkage to 19,315 in the hour following the linkage, a 66,000% jump in traffic. Needless to say, this just about killed my web service, something which was only averted because I had some mechanisms in place to automatically protect myself in such an event (discussed below). However, the initial digg spike was just that, a spike. Each subsequent hour saw a 25% decrease in traffic over the previous one for about ten hours.
The del.icio.us / diaspora effect At this point, I actually began to see a slight increase in traffic, which I attribute to the appearance of the site on the del.icio.us popular links page, as well as to the appearance of the site on an array of different blogs all over the internet. This effect was more gradual than the pretty clear spike I got from digg, though as Saturday wore on, the accesses slowly headed down again, but only at about a 4% decrease per hour rate. This is more typical of the decay one would expect from a reasonably popular link spread across a wide variety of blogs. I expected this rate of decay to continue and eventually level off, leaving me at a somewhat higher rate of readers than I had before.
The Lifehacker boom However, this all changed at roughly 8 AM Sunday morning, when this same post appeared on Lifehacker. While this spike was not nearly as strong as the digg spike, it was much more sustained (aided in part by the residuals from other blogs, but still quite noticeable). Throughout Sunday, following the spike, the access rate dropped about 8% an hour.
How digg and Lifehacker users are different The usage spike from digg was extremely sharp on its own, indicating that digg’s core user community reads the front page there quite frequently. Their audience is large, but their power comes from their loyalty to the site: they visit it many times a day. This is different from the other blogs that linked to the post over the last few days, particularly Lifehacker. It appears from these numbers that Lifehacker may have a larger readership, but it is less passionate than digg’s readership; the users that came from Lifehacker numbered higher than those from digg, but the Lifehacker hits were much more spread out.
Warped site feed statistics I use Feedburner to manage my feeds, and this usage spike has caused some very interesting skewing of the statistics there. Feedburner is indicating that The Simple Dollar has thousands of subscribers, a fantasy that I would like to believe, but I know better. The truth is that this spike in my feed subscribers came from the viewers from the spike, as described at blogtrepreneur.
Not quite a server meltdown Obviously, the usage spike caused my li’l blog to undergo some serious stress. I was lucky enough not to suffer a server meltdown, but I was unable to add any updates to the site most of the time throughout the weekend without incurring errors of various kinds.
The one thing that saved my server from completely melting down was my use of the wp-cache plugin. WP-Cache is a plug-in for WordPress that caches page accesses; in effect, it stores any page that is accessed as a static file for a specific length of time. When these spikes occurred, WordPress was not generating a new page from scratch each time; instead, it would just deliver a static page to the browser. If my site had been doing dozens of database queries for each page access during the spike, it would have melted down.
In short, if you suspect that your site is going to appear on a popular site like digg or lifehacker, you need to be delivering static pages. If you’re a blogger, you need to look into whether or not your blog authoring package makes it possible to make static pages out of your posts and, if it does, have this option available. If you don’t, an unexpected popularity surge will cause your site to go *poof* when you need it to be up.
Advertisements Although I am not at liberty to discuss actual dollar amounts, I can discuss trends, and I can say that although the spike did increase advertisement income, it was not in proportion to the increase in readers. Why? The incoming readers, especially the digg readers, did not click on any ads. These readers are very savvy and are looking for content; many of them were actually blocking the ads and not many of the others paid any attention to them. The Lifehacker readers were much more open to advertisements, but still not as forgiving as my usual readership.
I hope this post has answered any and all questions you might have about the effects of a digg or a Lifehacker linkage on an obscure blog. If you have any further questions about this spike in usage, please ask them in the comments so that others can be informed.