MeaningfulNoise.com

A client gave me the ominous duty of creating a model for proving ROI on paid search. ‘Omnious’ because the client’s paid search agency has been asked repeated to do it over the course of the last twelve months and they have yet to produce one. Now, to be fair, I don’t think they couldn’t come up with it. I just seriously doubt they even tried. Alas, organizations are asking for estimated returns on spend. [Soon] the days of increased online spend without objectives, measures, and success criteria, and attributable ROI, will be gone. And that’s where the actionable web strategist comes in.

“Attributable” ROI is the key here, i.e. I know people come to our site, do some things, and I know we generate revenue… but can I link them together? Many of the implementations I’ve seen around online marketing initiatives just aren’t setup to track an entry all the way through conversion. This is a problem especially with paid search. You dump budgets into it, your website makes money, but never the two shall meet - money for nothin’ and clicks for free. But, again, those times are coming to an end.

In creating this model, I started with the obvious questions… what are the objectives of paid search, what are the KPIs that were set, where’s the data, and how much is there?

The business objective is the obvious one, right? Wrong - talk to three people involved with the SEM program and you’ll get three different answers (as I did). The KPIs are page views… or should it be visitors, or maybe just referrals? Since they didn’t predefine the success metric, they wanted whatever had the best story. And the answers to where and how much data was relative to the multiple source systems used to manage the program and the constantly changing campaigns. This is not a great start - ominous, dubious, and unbounded.

So, let’s say what we’re starting with is a clean slate. The trap therein is picking something that works and tells a positive story so the executives can have some level of confidence that it’s been money well spent. It’s a trap because metrics are dynamic and have dependent relationships. As an analyst, you’re going to be confronted with this situation a lot. Here are a couple things to remember:

Any model is better than no model
This is from the Eric Peterson school of thought. You can sit there thinking about how hard it is and do nothing because you don’t want to be wrong, or you can get something down and watch it change. Don’t let the fear of being wrong stop you. If you are wrong, you now know one way that it doesn’t work. And that’s actually key. When creating your model, record your definitions and the findings of those methods that don’t cut it. This’ll save time for the next fellow.

Scope the model
This basically goes back to just trying something and not falling into analysis paralysis. When you set out to capture the events and context of a situation, plan it out from beginning to end and be as detailed as possible. But know, too, that you don’t know what you don’t know. Some influencer may get left out and you won’t know it until data starts coming in and it’s not following forecasts. Then you adapt the model. BTW, always work with the vendor or system administrator that manages the tracking system - you must fully understand the criteria for events being recorded. Don’t assume a ‘click-through’ is a ‘page view’ is an ‘impression’.

Test, revise, test, revise, test, revise
And this is the third affirmation for ‘just do it.’ Assume things will have to be tweaked from the beginning. I, in fact, always build in a data validation and revision into my timelines. The one thing clients hate is surprises. If you account for these things up front - no surprises, happy client. As a side note, if you have several date sets on-hand and you’re not sure which to include, run a regression on the data sets to determine which ones have influence (or dependency). This will allow you to expand or contract the set.

Forecast your baseline
Estimating performance based on the model is integral in the life-cycle evaluation of the model. There are things that will come up and potentially derail the linear trend. But those things are the ‘unknown’ variable I’ve talking about in the last three paragraphs. Once you see a deviation, you can evaluate the model and account for it if need be. I say “if need be” because some things you can just accept as anomalous. I like to include multiple arrays in my calculations to account for changes over time and smooth the anomalies within the calculations.

Hard returns, soft returns
There are always those little incidental things along the way you just can’t affix a dollar amount to… build it in as soft returns or potential gains. Just like with assumptions, a time may come when you can track them. Then you’ll have data to correlate and firm up the model.

The search model I built for the client was limited by the information available, but showed an estimated return given a set of assumptions. I even built in soft returns that could later become directly attributable to return. But the process gave the executives an idea of what was going on. An ‘idea’ that they prevously couldn’t have even guessed at. So, although limited in scope, the model helped. And with the flexibility and proper explanation of the assumptions and soft returns built in, they were very happy that this patient would only get better.