Regardless of how you may feel about about the pervasive coverage of the passing of Michael Jackson, this event has served as yet another reminder of just how dominant a role the internet now plays in the distribution of news and media. It’s the first channel many people turned to to find out what was happening as that story rapidly developed.
Ironically, these types of events also remind us of the many limitations that still remain around web content delivery, and the broad challenges the web will face in supporting the kinds of things we are expecting it to support one day.
So how did the web hold up?…
As events unfolded, the LA Times – who broke the Jackson story – had its website crash after several million visitors hit it in less than an hour. Many other major news sites slowed down significantly due to high volume. Even Google, a company used to dealing with access on a massive scale, had problems handling the load of people searching on its Google News section for information about Jackson’s death. And Twitter, a site that has effectively become the web’s real-time “newswire”, was running at least 5 minutes behind in getting tweets posted. While not an infrastructure disaster, this event certainly pushed most news/gossip sites close to the edge of their capacity, and not many had a graceful way to degrade.
Problems seemed to be even worse for the mobile web. Unaware of what was going on that night, I had posted the following on Twitter:
This wasn’t the first time I have had problems connecting to the web wirelessly through AT&T, but clearly it was more than just typical AT&T issues that ended up causing it this time. In general, I believe the adoption of web enabled mobile devices is outpacing even the fairly aggressive growth in mobile data capacity. Combine that with both a spike in demand and many unresponsive news sites, and the results were no doubt frustrating for many others as well.
The other big traffic spike happened this past Tuesday.
Many sites decided to set up live streams of the memorial service held for Michael Jackson. Though it didn’t go off completely trouble free, Akamai, the leading video distribution/streaming provider on the web, ended up serving about 20 million live video streams during that time frame. That is nearly 10 times the number of streams they typically handle – by any measure a huge spike. While this number is way short of the 100 Million+ viewers that watch live events like the Superbowl, it still represents around 3 times the audience that watches a typical top rated TV show every week. This was impressive.
Unlike the bit starved mobile web, the wired web didn’t seem to have an issue with overall bandwidth or routing. There were no reports of general slowdowns or serious bottlenecks occurring because of this event, which is great news. What didn’t seem to scale up as well were individual sites. Some of those issues could probably be addressed with a more aggressive adoption of cloud based site deployments. If designed correctly, cloud-based deployments could help these types of sites scale capacity dynamically to better handle unexpected spikes in demand. In effect, this is exactly what all of the major news organizations did by using Akamai to deliver their video, an aspect of their delivery that seemed to work pretty well. They produced and packaged the content itself and then leveraged Akamai’s shared global infrastructure to handle delivery – something they would never be able to do well on their own.
We need to start thinking differently about how to build out the web going forward, especially around the optimal use of shared vs proprietary resources. I also think we are probably getting to the point where we should look more closely at what role fundamental network management technologies like QOS, packet prioritization, deep packet inspection and traffic shaping should play on the internet, and how we can make sure they aren’t abused. This is an issue I am somewhat torn about. I am a big proponent of network neutrality, but recognize the very real negatives of an ‘all packets are equal’ approach to managing traffic. Though it tends to be an issue that evokes passion from all sides, we will need to have a rational, dispassionate discussion about it if we are serious about making the web into a truly global media backbone – something it has the potential to become.
Events like these remind us that we still have a lot more to do.