Friday, December 26, 2008

How “Real” is Real-Time Web Analytics?

The nuances of real-time Web analytics are elusive, underused and often misinterpreted. Yet, real-time Web analytics is a feature that several vendors are seeking to fulfill. The way I see it, real-time has two components; real-time reporting and real-time processing. Real-time reporting makes Web analytics data visible within an interface instantaneously (or within 10 seconds). Real-time processing is the ability to create reports or segments and view data instantaneously (or again, within 10 seconds or less).

Real-Time Reporting
The last time I checked (back in May 2008), only Coremetrics, IndexTools, Nedstat and Omniture claimed to have real-time data reporting capabilities. At that time, Unica could report metrics within 15 minutes and Google Analytics shortly thereafter provided same day reporting (although they make no promises). So, overall the major Web analytics players are pretty fast about reporting, yet not impervious to flaws [I welcome comments on real-user data delay horror stories]. Of course, log files can typically be processed in real-time as well, yet don’t necessarily provide the slick interfaces of the previously mentioned vendors.

Now don’t get me wrong, I believe that Web analytics data should be reported as quickly as possible. I for one like to gauge metrics the same day I post or make changes to my site, however, it’s my opinion that real-time reporting is a luxury that many want but few properly exercise. To obtain real-time reporting so that a neurotic analyst can watch metrics as they accrue is not a good use of real-time reporting. However, those that use real-time data to populate content on pages or shift messaging to segments…now that’s interesting. But there’s a distinction here, where real-time automation is different than real-time decision-making. Decision-making based on data rarely (if ever), happens in real-time. Some might argue that alerting functionality provides the ability to make real-time decisions, but c’mon…is that really real-time?

    Alert -> Think [allot appropriate time for comprehension, protocol and red-tape] -> Take Action.
A time lapse view of this process will likely reveal less than optimal Alert-to-Action reflex.

Real-Time Processing
The second component of real-time is processing. This facet of real-time serves the instant gratification need in all of us. A typical scenario might unfold like this:
    Idea -> Create report to illustrate idea -> Processing… -> Insight!
Again, going back to my research from May 2008, I asked about each vendors’ ability to create segments in real-time with no delay in data representation. Most vendors did offer this functionality, with only Clicktracks and Nedstat reporting that they did make you wait (Omniture can perform real-time segment processing within Discover and Google Analytics didn’t have custom segments at that time, but they can do it now). Once data appears within an interface, analysts can slice and dice to their heart’s content. Processing times will vary based on complexity, the amount of information being processed and hardware and/or bandwidth available. But this really has nothing to do with real-time. Rather, it’s the ability to create a new report, segment or data view and gain access to it without having to wait.

Real-time automation of data – using rules-driven processes, event triggering and content delivery that occurs in real-time based on data derived from analytics does elevate the need for real-time data. While the applications that enable this type of real-time activity may be independent of Web analytics, there is an opportunity to improve – dare I say – “optimize” the process with real-time capabilities.

So, I’m really wondering if people are using real-time data in other ways that provide tangible value to their organizations? Beyond real-time dashboards that make great corporate lobby eye-candy are businesses making real-time decisions that require real-time Web analytics?


Daniel Tunkelang said...

The main use case I've seen for real-time Web analytics is to trigger alerts for anomalous behavior that might require urgent intervention.

For high-traffic sites, real-time analytics could be leveraged to perform rapid iterations experimentation. But I suspect this is a rare use case. Few sites have the requisite traffic volumes, and those that do tend to be risk-averse.

Anonymous said...

Hi John,

Very interesting post. To me, real time analytics in WA is B***S***, or at best a myth. Very few, I mean very few, organizations can make good use of it. The rest of us don't really need it, and I wonder why vendors push that button.

Well, in fact I know: I have seen that "need" so many times in my practice. I usually defuse it when I ask clients this test question:"If I told you something at 11AM, what would be your ability to react within the same day?", and of course, the typical answer is that they couldn't do anything.

I don't say real-time web analytics is 100% irrelevent, all I'm saying is the number of businesses that can really put it to good use is very low (for technical, process flexibility, and business model reasons).

Anonymous said...

Nice article. Real time or not, web analytics is key to measure success.