April 3rdTechnical Problems Rock WebSideStory
Christopher Kenton
WebSideStory, one of the Web’s leading site analytics providers, has been hammered over the past two days by technical problems that have shut down both the public corporate site and the hosted application servers. Customers trying to access the popular HitBox analytics service have found the servers down, including all reports as well as online customer support. Apparently calls into the telephone support line haven’t fared any better.
As of Tuesday evening, the application server is nominally back online, but the navigation and login appear corrupted. A notice on the site states that problems have persisted for more than 18 hours, and two days worth of reports will not be delivered. It’s unclear whether that means “never” or just not immediately. If that sounds trivial, consider the case of a friend who first reported the problem to me. They just launched the first major product release for their company and gained press attention in some of the leading media channels. A big day for marketing. When they go to measure the impact of their campaign, sorry, site is down and we can’t tell you if you’ll ever get your data. Not good.
 I’ve got a call in to WebsideStory to see if there are any more details available. Â
System issues 2007 Apr 2-
2007 Apr 3 Tue 1435 PDT - Since Apr 2 Mon ~0700 PDT we are experiencing system issues. All presentation interfaces were slow and occasionally unavailable until ~1245 PDT today. The corporate web site and support portal were slow and occasionally unavailable until ~1400 PDT today. Most scheduled reports were not delivered yesterday and none will be delivered today. There are several remaining issues: Report scheduling services are not available. Session (global and active segment) data processing is currently 30 hours behind, so scheduled reports may not be delivered tomorrow. We deeply apologize for this disruption. WebSideStory takes uptime very seriously, and we are in the process of further upgrading our core systems to reduce the chance of any such occurrence in the future. We are carefully monitoring all systems and will update this advisory when further status is available.
WebSideStory, one of the Web’s leading site analytics providers, has been hammered over the past two days by technical problems that have shut down both the public corporate site and the hosted application servers. Customers trying to access the popular HitBox analytics service have found the servers down, including all reports as well as online customer support. Apparently calls into the telephone support line haven’t fared any better.
As of Tuesday evening, the application server is nominally back online, but the navigation and login appear corrupted. A notice on the site states that problems have persisted for more than 18 hours, and two days worth of reports will not be delivered. It’s unclear whether that means “never” or just not immediately. If that sounds trivial, consider the case of a friend who first reported the problem to me. They just launched the first major product release for their company and gained press attention in some of the leading media channels. A big day for marketing. When they go to measure the impact of their campaign, sorry, site is down and we can’t tell you if you’ll ever get your data. Not good.
 I’ve got a call in to WebsideStory to see if there are any more details available. Â
System issues 2007 Apr 2-
2007 Apr 3 Tue 1435 PDT - Since Apr 2 Mon ~0700 PDT we are experiencing system issues. All presentation interfaces were slow and occasionally unavailable until ~1245 PDT today. The corporate web site and support portal were slow and occasionally unavailable until ~1400 PDT today. Most scheduled reports were not delivered yesterday and none will be delivered today. There are several remaining issues: Report scheduling services are not available. Session (global and active segment) data processing is currently 30 hours behind, so scheduled reports may not be delivered tomorrow. We deeply apologize for this disruption. WebSideStory takes uptime very seriously, and we are in the process of further upgrading our core systems to reduce the chance of any such occurrence in the future. We are carefully monitoring all systems and will update this advisory when further status is available.
In this new age of hosted services, sometimes people forget that people have outsourced very mission critical applications and your job as a vendor is to ensure 99.999% ( five nines) of reliability.
Web analytics is not mission critical? Well to the VP of marketing it is certainly mission critical. Your story about the new product launch hitting at the same time you had no analytics highlights that.
Hopefully this will be a lesson for WebSideStory and they will use it as an opportunity to improve.
PS - I fully believe in hosted services (my company sells a hosted service!) and think they usually provide more uptime than applications that companies buy and run themselves. The difference is just that it is more public when hosted services fail and you have to yell at your vendor, not your VP of IT.
April 4th, 2007 at 4:47 am
Mr. Volpe’s response is a bit confusing.
1. Only 10% of businesses online today utilize an enterprise caliber on-demand analytics application - so even the online world as a whole does not see it as a mission-critical app.
2. If a commerce site’s shopping cart doesn’t function, they lose money, i.e. cart=mission critical. Access to an analytics UI go down for an hour, with NO data loss,….is this the failure of a mission critical app?
3. VP’s of Marketing rarely look at web analytics on a daily basis, unless they are at a start-up. Most companies have managers and analysts that look at the numbers and DO generate daily reports for VP’s, though in reality, how often do the VP’s look at them? …and keep in mind, only 10% of business online are even doing this.
4. WebSideStory has been, actually, very reputable in terms of uptime. I believe SEMPhonic was the latest firm to tout this in their report released last summer, ‘HBX vs. SiteCatalyst’
5. Mike’s company, HubStop.com has no enterprise on-demand analytics implemented at this time, per their source code, not even free google.
April 5th, 2007 at 6:41 am
Steve–
You make some good points. There’s no question that a shopping cart is significantly more mission-critical than an analytics application. But I don’t think that warrants writing off the incident as a non issue. If analytics were unimportant, there wouldn’t be such a compelling business case for applications like HitBox and HBX. If you’re the marketer responsible for reporting on the effectiveness of a major campaign–not to mention justifying the monthly budget for the application–a significant outage like this at the wrong time simply sucks.
You’re absolutely right–WebSideStory has been very reputable, and one of the pioneers of online analytics. That’s what makes this incident troubling. When a firm of their caliber takes such a signficant hit–and its support systems appear to be in disarray in responding to the problem–it’s disconcerting.
Granted, it’s not the same as losing revenue, but I don’t think that gives them a bye on this.
April 5th, 2007 at 7:10 am