Analysts: Put up or shut up time!This blog is centered around creating incredible digital experiences powered by qualitative and quantitative data insights. Every post is about unleashing the power of digital analytics (the potent combination of data, systems, software and people). But we've never stopped to consider this question:
What is the return on investment (ROI) of digital analytics? What is the incremental revenue impact on the company's bottom-line for the investment in data, systems and people?
Isn't it amazing? We've not pointed the sexy arrow of accountability on ourselves!
Let's fix that in this post. Let's calculate the ROI of digital analytics. Let's show, with real numbers (!) and a mathematical formula (oh, my!), that we are worth it!
We shall do that in in two parts.
In part one, my good friend Jesse Nichols will present his wonderful formula for computing ROA (return on analytics).
In part two, we are going to build on the formula and create a model (ok, spreadsheet :)) that you can use to compute ROA for your own company. We'll have a lot of detail in the model. It contains a sample computation you can use to build your own. It also contains multiple tabs full of specific computations of revenue incrementality delivered for various analytical efforts (Paid Search, Email Marketing, Attribution Analysis, and more). It also has one tab so full of awesomeness, you are going to have to download it to bathe in its glory.
Bottom-line: The model will give you the context you need to shine the bright sunshine of Madam Accountability on your own analytics practice.
Ready? (It is okay if you are scared. :)).
Here's part one, let me hand you over to Jesse…
______________________________________________________
Hello my dear, dear friends fighting the good fight of analytics. I felt compelled to write about this topic because I, too, am an analyst to the core. However, I've long felt the unsettling sensation of having a tremendous impact on the business, but still having to fight for attention and resources, and always wondered why that is.
After being an actual analyst for a while, I'm now managing the Google Analytics Certified Partner program, and I now realize that it wasn't just me, our entire INDUSTRY is affected by this issue.
Why is it that we analysts feel like we have some amazing untapped ability that could revolutionize any business we touch, and yet we have to fight to be included in strategic conversations where we could do the most good, and we have to fight not to be ignored when we have something important to say?
Why is it that most analytics departments are constantly under-funded and under-staffed compared to the budget-hogs in Marketing or the herds of tech workhorses in IT?
I would venture to say it’s because we’ve made an awfully poor case for the value of what we do. Businesses, by and large, don’t understand the ROI of analytics… the Return on Analytics, if you will.
Everyone else seems to get an ROI calculation, but not us. Marketing dollars (hopefully!) get measured by their Return on Ad Spend. Product improvements are quantified in incremental sales. Even internal tools are evaluated by work hours saved. Yet the analytics team rarely has its costs measured in terms of impact on The Company.
‘But we measure (and hopefully, improve!) the ROI of other things’, you say. 'The impact of analytics is the impact we have on other teams.'
Exactly! And therein lies the problem! The absolute best case scenario is that we spend all our time making everyone else look better, only to let them take all the credit.
Do we really think that if our executives believed that every dollar invested (properly) in analytics would result in ten dollars back for the company, that we would still face the massive hurdles that so many of us deal with daily? Heck no!
However, it’s true, analytics is worth at least 10x what you invest into it. These successes are ours to claim, and it’s high time we started claiming them.
So, what is ROI?
The ROI calculation is cliché and overused because it’s so simple, even a child could do it:
* How much did you invest?
* How much did you make in return?
* Was the latter greater than the former? (And enough so that it was worth the effort?)
With that context in mind, here’s an equation I drew up to quantify your impact as the “Return on Analytics”:
Don’t worry, its bark is worse than its bite. All it’s asking you to do is put the ROI calculation in terms of how analytics works. The formula accommodates for the critical need to compute incremental impact from deployment of an analytics practice.
You see, the challenge with analytics is that you can’t just say “how much did you make in return?” because you were (likely) already going to make something in return. So we have to figure out the impact – the incremental return that you wouldn’t have had otherwise.
So what I’ve done here is:
* Highlight the “full incremental return” within a discrete time unit (such as a day, week, month, whatever) by first subtracting the improved ROI thanks to analytics (Ra) from the original ROI that you were getting before (Rm)
* Then multiplied that impact by the duration of those time units that it will last: (d)
* Finally divide that by the costs it took to get to that impact: Ia
The end result:
‘But it’s so much more complicated than this!’, you argue. Yes. Yes it is. But so is any computation of ROI, if you really want to be honest. What this does is to package your impact into a (relatively) easy to understand way.
Let’s take an example.
Say you’re a mid-sized company who sells hubcaps. Your digital marketing team has a monthly budget of $30k, and the company sees $120k in monthly sales from it. 400% ROI, not bad.
You then hire on an analysis ninja and pay them $5k per month to “fix” your analytics (a bargain, if you ask me), and after 6 months of data-driven improvements to campaigns & landing pages, all of a sudden the same marketing costs are bringing in $180k in sales per month, a success rate which continues on for 12 months (until a new line of hubcaps come out).
Clearly some of the credit for this goes to your marketing team. But before you jump to “my marketing is now making a 600% ROI, that’s fantastic!” and then promptly give the Marketing team more money, it is important to realize that none of this would have happened were it not for the analyst who took the holistic approach to identify the optimization opportunities.
So, let's go back to our formula (above). Punch in some of the relevant numbers and you'll see this:
Holy guacamole! You’ve hit a gold mine! If every dollar you’ve invested in this team paid off even half as much, then your company would be the #1 hubcap dealer in the world in no time!
This is the potential power of calculating your ROA. Attributing success where it’s due so that you can fuel the true driver of growth.
Once you’ve taken a hard look at what your investment in analytics (everything from tools to people to professional services) has produced in terms of real business results, ask where you need to invest more in order to get to a positive ROA … and not just a positive one, but the one we all imagine ourselves to be capable of.
______________________________________________________
Simple and amazing, right?
Here's the really key part… Businesses often don't understand the ROI of analytics. In fact it is not uncommon that they often don't even understand what analytics is! Here's the hidden awesomeness of computing ROA: If we can prove that there is ROI, they don't need to understand what we do as Analysts! Just like other professions (say, Accounting – what is it that they really do? :)), the analytics practice, and Analysts, will earn the right to be left alone to add value because in a very compelling way Businesses and leaders, through ROA, will know that we are adding value!!
Yes. I hear you (and Jesse acknowledged this as well).
This part of the blog post is to deliver the specificity that you've come to expect on Occam's Razor. Practical examples and specific guidance that will give you a leg up if you are convinced that bringing accountability to your analytics effort is a good thing.
The guidance is going to come to you via a customizable model in a handy dandy spreadsheet. So to speed up your ROA computation, download: Return on Analytics Calculation Model.
The model has a summary tab, a tab full of awesome specific guidance on how to compute incrementality with pitfalls and caveats, and finally a whole bunch of tabs with sample computation of incrementality across various analytical efforts.
Let's walk through the model in detail.
Tab one contains the model for an actual client from whom you can find inspiration. The first thing you'll notice is that the formula has already been created for you. No need to touch this.
The second key element is the annual revenue for your company prior to the implementation of analytics (or a major expansion of your analytics practice). We are trying to establish a baseline. Type it in.
The third element is to calculate the total cost of ownership. Your cost! Ok, ok, you plus the hardware, software, army of consultants and BFFs. :)
Here's what you'll see for that element when you open the model…
The numbers are realistic but by no means reflect what they might look like in your company. I've typed in as many things as I could think about connected to having a web analytics practice (i.e. Total Cost of Ownership).
So say you have Adobe's SiteCatalyst. You have a fixed fee you have to pay. You have a variable cost. You have a hourly support contract. You have an external agency helping you with implementation and online support. You have other software deployed, like tag management (all in vogue now and you know what, it costs money!) and specialized PPC tools and email or other software.
It also includes the important bits, ones we often overlook when creating an analytics strategy: The cost of people inside your company whose primary analytics job is implementation/IT (tagging, retagging, etc.), people whose primary job (greater than 70%) it to provide data but without insights or recommendations ("reporting squirrels," a necessary expense in any large company) and finally people whose primary job (greater than 70%) is to do analysis (and hence not data puke but provide insights and recommendations only).
And the $50,000 for IT resource and $25 for an analysis resource is just a joke I desperately hope is not true in your company (big or small, remember the 10/90 rule for incredible digital analytics success).
Not every single one of these rows will apply to your business. Say you use Yahoo! Web Analytics, the first two rows disappear, the third might not apply, but the rest might. Say you are a medium-sized company using WebTrends or Omniture, the first two rows might not be 100k/50k rather be 1,000k/350k. If you are a larger-sized company, well, you know the drill. If you are a large company you might have an army of consultants, if you are a small business that might be the free time you are getting from your cousin Ali.
So adapt the model, type in your actual costs. Calculate your digital analytics total cost of ownership. It will be revealing. I promise.
Then comes the magical part. What does your company get for all this investment?
The structure is simple, you identify the change you drove and then identify bottom-line impact of the aforementioned change after implementation of your data-influenced recommendation.
Here's what the various bits of impact look like the ROA computation model you've downloaded…
There are literally n number of things you could be driving inside your company.
In the model there are three clusters: 1. Media Optimizations 2. Content / Website Optimizations 3. Product / Company Optimization.
In each case, as you'll note above, there are examples of the type of activity that data might have informed and an example of the incremental impact.
By the way, incremental means incremental. The analytics team found an insight via their data analysis (at this moment you'll really, really regret if the primary function of your analytics practice is to data puke), that insight bundled with a specific recommendation for action was communicated effectively to the senior management, they in turn ensured it was implemented, and revenue went up.
At this point let me say something immensely important. We (Analysts) are NOT trying to claim credit for the entire uplift. We found the insight in the data and recommended an action, but many people are involved from that point on. Your marketing team went and got it implemented. Your copywriter created new copy. Your designer created new graphics. And so on and so forth. We are not trying to say here that we were singularly responsible for the incremental revenue.
We are just trying to say that that incremental revenue came from an insight produced by data analysis. So we are trying to give credit to the data. We are NOT trying to steal credit or undermine the team effort it takes to get things done in every company.
I sincerely hope that this section of the model serves as an inspiration of sorts for the vast net that data can cast in terms of driving change.
You'll see reduction of checkout abandonment rates from quantitative analysis, you'll see impact from improving task completion rate from qualitative analysis (which might drive offline conversions), you'll see impact from technical improvements, you'll see impact on the company's long-term value by improving brand perception or social media presence.
Let your mind roam wild. Look in every nook and cranny. And if your analytics practice is not focused on everything listed in this section (why not?), there is a lot of upside for you!
At least at the moment, not all the rows will apply to your business. That is ok. Fill out the ones that do. Improve over time.
Right now you are surely wondering: "Wait, what about that incremental bit? You ran over that pretty fast. That is hard stuff! "
: )
No. Did not forget that!
First, identifying incrementality is an incredibly difficult challenge. While getting perfect answers is nothing short of a life time effort, getting a good enough answer does not have to be very difficult.
So why not start there?
In the model you'll be delighted to discover a number of examples of how to compute incrementality. For example here's a screenshot of identifying incremental impact from your email marketing program.
The first thing you'll notice is that you can do this exercise in layers.
You can start with something simple. Let's say the analytics team does analysis of current email marketing metrics and identifies improvements to how your company structures the emails that go out. The recommendations are implemented and that drives an additional 100k clicks from the email campaigns. Assuming that nothing else was changed, it is now easy to measure the incremental impact of these changes.
Or maybe nothing was changed in the campaigns, but conversion rate was improved from 2% to 5% by changing the checkout conversion process for email campaigns. Well, it is easy to calculate that impact.
Or maybe you have an advanced analytics team with lots of senior management support and are able to improve the email copy and calls to action, the checkout process and do much better cross-sells and upsells and improve average order value. Well, that third cluster shows you how your computation might look.
Is it a perfect approach? Almost. Does it get you going in the right direction? Emphatically, yes!
As Voltaire put it: "Le mieux est l'ennemi du bien." (The best is the enemy of the good.)
There are other examples in the spreadsheet that should serve as guidance/inspiration for approaches you can take when you compute incrementality of the impact you deliver via your analytics practice.
Here's the section on computing value delivered by your investment in software to do multi-channel attribution modeling and the person you hired specifically to do that work…
From an impact computation perspective you can see how brutally simple the process is. Either you delivered revenue increase, or you did not.
Multi-channel attribution modeling is not easy. It has an astounding track record of failure. Identifying which model to use to attribute credit for a single conversion across multiple media channels is immensely difficult. Yet calculating whether it improved the bottom-line, whether it delivered positive ROA, is simple. You fill out the blue cells. You look at the row called Incremental Revenue. If there is something there, your digital analytics investment is worth it. If you have nothing there … well, you know … let's figure out how to say data is always worth investing in. :)
There are a few more examples I wanted to insert to really make this concrete. We cover how to compute incrementality from improving conversions, but also how to do that for the micro conversions and capture the impact of the long term impact on the business by tracking micro conversions.
Here's an excerpted version of that section…
Excited? I hope so. I was giddy as a teenage school girl just creating these for the model!
There is also a tab to help you identify the incrementality from landing page optimization, and from improvements you make to the cart and checkout process. (You know my obsession with both, see best digital marketing experiences post.)
And we can't do anything related to data driven improvements without helping you compute the incrementality from insights we identify for our Paid Search campaigns.
I'll let you be delighted about both those tabs when you look at the model, and not spoil your surprise by posting images here.
The model contains one last present for you. Checkout the tab titled General Impact Analysis.
If you are new to the field you are perhaps wondering what kinds of actions you could be taking for each focus area (PPC, Email, Display, etc.). You'll find that in this tab. Column B provides description and examples of the types of outcomes you might drive in each initiative, Column D sheds light on the implementation difficulty of various types of analyses, Column E helps you understand the difficulty you'll face when computing incremental return and finally a reality check under the column titled validity of incremental return .
You are now all set to go!
Here's the link again: Download: Return on Analytics Calculation Model.
Closing Thought #1: "I ain't got no incrementality!!"
It is entirely possible that at the end of looking at all the tabs in the spreadsheet you have nothing to type into the ROA computation model proposed by Jesse. A likely reason for that is that you were unable to identify any action taken as a result of your analytics practice.
There might be a simple causal factor for that. Your analytics practice is focused on DC and DR. And it turns out that you need to obsessively focus on DA for your analytics practice to have an impact on the company's bottom-line.
DC, DR & DA are three key components of any analytics practice. Data capture, data reporting, and data analysis.
I discussed this framework extensively in a recent blog post: Web Analytics Consulting: A Simple Framework For Smarter Decisions.
As you'll note in the DC, DR, CA framework post, most analytics efforts (especially web analytics), consulting or in-house, are focused on collecting ever more data and in figuring out how to puke an ever-increasing amount of it in the form of standard reports via as much automation as possible. Sadly this rarely leads to the recipients gleaning any insights. Which in turn ensures that the organization is data-rich, but action-poor. Which, heartbreakingly, does result in zero actual impact on the company's bottom-line.
Hence your inability to type anything into the column titled Incremental Revenue/Impact.
So if you don't have anything to type into the various tabs in the spreadsheet I encourage you to read the DC, DR, DA post for specific guidance on what is contained in each area and how to ensure you have a better balance (egregiously focused on DA) for your analytics practice.
More investment in analytics (and your salary) will come from an ability to clearly demonstrate impact on the bottom line; otherwise, we will remain third-class citizens of the business world. The model outlined in the spreadsheet could possibly be a diagnostic tool in helping identify problems with your analytics practice (big data or small data) and figure out how to create a practice that is focused on ensuring incremental impact.
Closing Thought #2: Inspiration wrapped inside an exhortation!
You'll fail to attract investment in analytics inside your company (and a higher salary for yourself) if you are unable to show an impact on the company's bottom-line. You'll fail to show an impact on the company's bottom-line if you don't recommend actions your executives should take. You'll fail to recommend actions without an obsession on analysis of data. And yes, you'll fail to analyze data without collecting it.
If your analytics practice is not producing any actionable insights (hence no ROA) then it might be because the analytics practice is not focused on what's important to the business (advice: Biggest Mistake Web Analysts Make… And How To Avoid It!), or focused on reporting and not analysis (advice: Difference Between Reporting And Analysis), or perhaps needs a crash course in how to do better analysis (advice: Beginner's Guide To Web Data Analysis), or perhaps just needs to extract more value from the tool you have (advice: Google Analytics Tips: 10 Data Analysis Strategies That Pay Off Big! ). Identify and fix the problem. Promise me you are not going to settle for a lower salary and a boring job!
I wish you all the very best.
Before we go, my deepest thanks to Jesse Nichols for contributing to this post and inspiring a discussion that has been a long time coming.
As always, it is your turn now.
Does your company compute the incremental impact of its big data, digital analytics efforts? Is there a part of your effort that you are able to identify incremental impact for most easily? What are the biggest challenges you've faced to justify return on analytics? The model is centered on ecommerce/digital type businesses, what unique challenges do you face as a non-ecommerce/non-primarily-digital business? Do you have suggestions for improvements to Jesse's ROA formula? What are some salient hidden dangers we might be overlooking?
Please share via comments.
PS: An Ask from You: I feel that the model could use more tabs of incremental computation guidance. Can you help me create more tabs for various online or offline marketing initiatives powered by analytics? If yes, could you please create additions and email them to me? I'll be immensely grateful, and I'll add it as a tab to the model in this post (and of course credit it to you in the model, with a link to your blog / twitter profile / company). Please consider helping the community.
Thank you.
Excellent Analytics Tip #22: Calculate Return On Analytics Investment! is a post from: Occam's Razor by Avinash Kaushik
DIGITAL JUICE
No comments:
Post a Comment
Thank's!