Wednesday, October 1, 2008

Escaping Web Analytics Hell

At the annual summit in Vegas, I presented Escaping Web Analytics Hell: A Strategy for Attaining Paradise by Avoiding Eternal Damnation. Based on a strong response from the crowd and numerous follow-up questions, I decided to share some of my insights from this presentation here on my blog. I originally delivered a version of “Web Analytics Hell” at eMetrics in San Francisco, but modified the one to include commentary from Brain Elliott from Albris and John Lazarchic from Petco. I’m told that the full replay of the presentation will be available for download on the site soon.

The concept draws on my literary background, which I exercised by extending an analogy for Web analytics that parallels Dante’s Divine Comedy. The Divine Comedy is a trilogy written in the 14th century describing the horror and punishment delivered in hell (Inferno), enduring penance in Purgatory (Purgatorio), and the ascension into Paradise (Paradiso) for the truly virtuous. Playing the part of Virgil (Dante’s guide through the underworld and purgatory), I began by describing for my audience how the depths of Inferno align with common challenges for Web analytics practitioners and their organizations. Consequently, climbing from the depths of Hell requires strong Web analytics practices and only truly evolved analytics tactics will lead you to Paradiso. The following is what I relayed to the audience:

Dante’s Nine Stages of Inferno and the Corresponding Analytics Resolve

Stage 1) Limbo:

Web analytics limbo exists when sites have unclear goals, poorly defined measurement practices or no analytics evangelist/champion within the organization. Nothing truly gets accomplished and Web analytics data often suffers from neglect.

Sites that escape Limbo are those that establish process within their organizations as it relates to Web analytics. This involves several components including:
• Strategic resources to define business objectives and establish data collection needs
• Analytical team to train end users, build reports – dashboards and KPI’s – and most importantly, automate reporting
• Tactical support to react to analytics reports, implement changes and build a process of continuous improvement through measuring the effects of change

Stage 2) Lust:

Web analytics lust occurs when practitioners begin to question whether an alternative tool would perform better for their organization. If only we had [insert Web analytics vendor name here] we could achieve so much more. This fallacy perpetuates when Web analytics is viewed as the resolution to a problem rather than a diagnostic tool. Analytics tools are a starting point and not a means to an end.

Sites that overcome Lust cultivate expertise internally and seek resources to improve their analytical capabilities. Often times, these resources come from external sources in the form of vendors, authorized consultants or marketing strategists. While in-house expertise may be difficult to acquire, the analytics consulting industry is burgeoning and contains plentiful help in technical and strategic support. Yet, the overarching objective must be to nurture in-house talent to attain a sustainable program of analytics.

Stage 3) Gluttony:

Online marketers currently exist in a rampant state of Data Gluttony. This is exemplified by the fact that 39 percent of site operators measure multiple aspects of online visitor behavior, but do not use this data in any way. Data gluttony is a travesty of resources enabled by readily available analytics solutions and a dearth of actionable insight. Organizations must resist the urge to consume all data without cause or purpose.

Defeating Gluttony requires managing data intake and aligning business objectives with measurement goals. Sound logical? Ironically, many sites simply collect data without acknowledgement of how that information will be used or leveraged to improve business initiatives. Not that each measure must have immediate impact, rather sites should understand their data needs and how they might effectively collect and analyze data so that it contributes to overarching goals.

Stage 4) Avarice:

Also known as Greed, Avarice (in analytics terms), is the failure to disseminate information for fear of misinterpretation or misrepresentation of the data. Often times analytics data is not shared across organizations and insight and intelligence remain siloed in disparate parts of the organization. The inability or unwillingness to disseminate data profoundly limits a holistic view of behavior and performance. The antithesis of Avarice is distributing too much data, such that reports are meaningless and recipients fail to identify or take note of important information.

Site operators that quell Avarice learn to delegate data responsibilities throughout their organizations. This includes distribution of information in an efficient and responsible manner (i.e., multiple report formats distributed to different stakeholders). Further, reporting that is inclusive of analysis is exponentially more valuable than raw metrics. The opportunity to include analyst insight within reports empowers those closest to the data to identify meaningful opportunities and forewarn dangers. Further, annotations within reporting minimize misinterpretation of data and align recipients.

Stage 5) Wrath:

When translated to analytics-speak, Wrath equates to an inability to take action from data. This can result from a lack of unified efforts, when goals or KPI’s go unchecked or when data is simply ignored or devalued within an organization. The wrathful are a dangerous bunch because despite their efforts no results evolve. Sites with wrathful analytics practitioners should heed this warning; fear the loss of your valuable analysts for they will soon flee for organizations that value their work.

Empowered analytics practitioners are happy practitioners and empowerment is achieved within data driven organizations. Although using data to drive change is no easy task and one that requires commitment from the top echelons of an organization. A strong foundation for data comprehension and analysis therein lies. Only then can analytics data be used to drive rules-based processes, automate content delivery and dynamically effect change. Sites that overcome wrath are using data from multiple campaigns and opportunities to fuel optimization in other parts of their organizations. Learnings from the online channel, email or search efforts can be harnessed for new initiatives.

Stage 6) Heresy:

Within web analytics heresy killed the HiPPO by challenging the status quo. Effective marketers are open to change and do not remain content once they think they know their customers. In reality customer behavior, attitudes and opinions shift rapidly and relying on static marketing tactics is a flawed strategy. Challenging preconceived notions of who target customers are, where they come from and how they behave on sites and within campaigns is infinitely traceable using Web analytics. Sites that don’t challenge conventional wisdom risk customer abandonment.

Respectful Heresy can be accomplished through A/B and multivariate testing. Using technologies fueled by analytics data, site operators can effectively test ideas, concepts and designs using widely available testing technologies. These processes are a logical extension to Web analytics data collection and segue to more sophisticated analytical practices. While too few sites are currently utilizing testing, this is the next frontier for achieving incremental optimization with clearly demonstrable results. Sites that aren’t performing tests to identify optimization opportunities should question why they don’t have a program in place.

Stage 7) Violence:

Analytics data should unite organizations rather than cause conflict between them. However, often times data resides in silos making it challenging to access or share data between business units. According to 27 percent of executives surveyed, one of their greatest challenges for their organizations is that data use and analysis is conducted independently within different business units. Disjointed analysis of this nature has the potential to influence flawed assumptions and misguided strategies, which could escalate to violent situations.

The solution to mitigating violence caused by siloed analysis of analytics data is to democratize access to Web analytics tools and provide various levels of access to individuals within your organization. By enabling all interested parties access to data, information has a much greater opportunity to provide a foundation for the data driven organization. Yet, not all stakeholders will show interest or have the inclination to access primary tools via an interface and therefore data must be socialized throughout the organization in reports customized for individual stakeholders.

Stages 8 & 9) Fraud:

The eighth and ninth stages of the Inferno represent the worst offenders in the category of Fraud. These carnal sinners include: Panderers, Seducers and Traitors. The corresponding offenses for analytics practitioners can be aligned to those introducing doubt regarding data accuracy concerns, those seduced by multiple tools resulting in double tagging of pages and finally, no single version of truth or loyalty to metrics which diminishes the ability to make data driven decisions.

Fraud can be overcome by instilling a process for measurement and communication of information as it relates specifically to business goals. This requires a commitment from multiple levels within an organization and often times trickles down from the top in large companies. Yet, the reality is that over one quarter of executives reported that Web analytics are not ingrained within their corporate cultures and even more distressing is that these individuals feel that analytics is something that they could do without. A travesty!

If you made it through my extended analogy either you’re a curious literature buff in a quandary over my stretched analogy, an analytics aficionado looking to glean a gem or idea, or simply a glutton for punishment. In any case, I’d love to hear your thoughts on ways to get into analytics hell, or better yet, methods to get out.

Stay tuned for my next post…Attaining Web Analytics Paradise (Methods for Attaining Paradise by Avoiding Eternal Damnation).


HomeProspect said...

Excellent points! I suppose my comment counts as user satisfaction and return rate possibility. Cept' I forgot to hit the dang RSS feed. So much info so little time. I'll be back.


John Lovett said...

Thanks for your feedback John, you're welcome back anytime ;)