Why Analytics (Mostly) Don’t Matter

Posted on | June 24, 2009 | View Comments

Analytics are important to me. I’m a child of the Internet. A Web nerd. A baseball geek. I love me some numbers.

But numbers aren’t all they are cracked up to be particularly online and really particularly when you’re trying to build a business around them. Any salesperson will tell you that.

There is information to be gleaned from them, though. But I’ve found that most folks have a rudimentary understanding of them when it comes to editorial development (and let me also say that when it comes to the high level math that runs the Web – e.g. Googlenomics — I am equally baffled).

One of my first tests, when I talk with traditional media companies, is this: I ask them how much an ad on a particular page costs and how much it costs to produce that page; and then ask them the same for their website.

There’s usually an answer to the first one and a blank stare with the second. Which brings me to the analytics problem:

When I was at MIT (and actually every place before and after that where I had access to the numbers), I break my analytics down into 5 layers, each which has a value but which ultimately allows me to judge what is working and what isn’t, then make adjustments.

Layer 1:

These numbers are the basics: page views, unique users, time on page, advertisements per page, CPM (and ACPM). Essentially these are numbers that use raw figures and maybe some rudimentary averages.

These are good to judge an overall health (PVs and Uniques), decisions on sales (what’s our percentage inventory sold, what are our overall ACPMs). These are large, systemic answers.

The problem comes when you begin to make specific tinkers — we must tweak this section or raise ad prices — based upon the global picture. That’s a bad idea and leads, quite often, to things like feature creep or ever-changing designs because those numbers have to go up….immediately.

Layer 2:

These are usage numbers where you begin to analyze where people are going, why they are going, how long they are staying in places, and bounce rates. These are metrics that — like Layer 1 — give you an overall picture of heath.

It’s a bit more specific than Layer 1 stats because you can begin to target parts of the site; however, you are still analyzing the site as a whole. You haven’t started to parse through the content (not the sections mind you) so whatever numbers you get are more of a global health, even if it’s a bit more targeted.

Essentially, Layer 2 — for me — is anything that is below the site level but above the self-labeled content level. These helps in mapping trends.

Layer 3:

This layer comes with some voodoo. It’s at this point where you need to begin to classify your content, which requires that you rethink your entire editorial structure. This happens in a couple ways:”

At MIT, I went through 10 years of content and grouped content into the largest possible buckets that we could (at the time: Info, Nano and Bio; we eventually settled on Energy too). Every piece of content we did — since that was in our mission statement — had to be grouped into that category.

If you find you have lots of content that doesn’t fit, you may not be doing what you think you are. (Editor-types are particularly prone to arguing on this point). Either way, you need to wrestle your content into LARGE buckets. If you start parsing down into very small spaces, you will have other problems (e.g. navigation, selling ads in those section, ect).

Once we broke down the large buckets, then we began to dig into the tagging to find out what we were covering. This was for editorial purposes — again, so we knew exactly what it was that we were doing (and not just what we said we were doing).

Now you can begin to template the Layer 1 and Layer 2 onto these Layer 3 statistics, which allows you to get very specific information (what is the ACPM for a section? is that section to small to sell? are users leaving these sections? where are they going after? how much does it take to produce these sections? what ad prices do you need to sustain them? is this small enough that it’s a loss leader but important enough to keep? how much do you have to make up in other sections to keep this going?)

Layer 3 analytics, which is voodoo because you are creating the sections based upon your best analysis, are where the business model comes from. It’s also where you can begin to have real discussions with people on what needs to be done with content.

(I can say, though, content creators I have worked with ignore this level, preferring to have some fictionalized argument about “the way things should be”, which then leads to things like newspapers that make no money online and decry how hard it is to do so).

Layer 4:

From there, you can begin to look at the navigation structure. You now know what your big buckets are, you know exactly what content drives traffic, what ads are selling, where your people are going. If you have a smart boss, you can make a logic, numbers-based argument about the design and navigation based upon what your mission statement is.

What is your goal as an organization + deep analytics = your design.

Epilogue:

Now you have the ability to step back from the day-to-day, second-to-second analytics that so many people get caught up following. Instead, you have a well reasoned way to examine what is happening on your site.

You have levers you can pull (this section, while important, isn’t making money so we need to find ways to augment that with other content) to keep your operation in the black. Instead of making wholesale design, content, product changes, you are making long-term, strategic decisions based upon what your company does.

This is hard to do. It requires a tremendous attention to the analytical detail. It requires that you don’t over-categorize your content. That you resist the pull of those who won’t dig into the numbers to see the reality of what you do.

But to build a successful operation around analytics, you need to find a way to communicate the story of the site in business terms.

Comments