Comments on: Why Web analytics won't help interaction design

By: Paula Thornton

Paula Thornton — Mon, 06 Aug 2007 20:42:29 +0000

The traffic has to be tied to the behaviors…one without the other is only half the story. If you don’t know what people came to do and whether or not they felt like they were successful in their attempts, you know absolutely nothing.

By: Webanalyticsbook » Web analytics vs. design

Webanalyticsbook » Web analytics vs. design — Sat, 04 Aug 2007 17:01:01 +0000

[...] Here the thoughts of the Design Research blog: [...]

By: dpascoe

dpascoe — Fri, 03 Aug 2007 16:52:43 +0000

Sam,
Pretty brave of you to step out there with this opinion – some might think it’s heresy

I particularly love your jewelry clerk analogy. Way too much is assumed based on the data that web analytics collects. There are gaping holes, including:

1. Web analytics only captures what people did, not what they didn’t do, or what they started to do, but lost patience with before they finished doing it. To understand why, it is not sufficient to ask people, because they can only comment on what they did and saw during the visit, a visit which will only consume a fraction of the site. Owners also have to be able to perform complete site discovery, so they can understand the environment they have created, that visitors are reacting to. They need to be able to know with certainty everything is working like it should, scripts are executing, links are working, pages are being served, forms are advancing to the next level. They need to be able to drill down on the data in multiple ways, slicing and dicing to find that needle in the haystack when the situation requires it. And they need the ability to adjust their queries of the data in fast and flexible way, as their needs change.

2. Web analytics is prone to systemic errors – untagged pages, beacons that are not firing. Lots of WA pundits argue that accuracy doesn’t matter and that getting a sample is enough. That MAY be true if the sample is truly random. Systemic error means that the data about a particular action – page visit, for example – is missing 100% of the time. This leads to incorrect assumptions and bad decisions. Without validation of the implementation, site owners can have no confidence in the decisions they are making because they can have no confidence in their data.

Complete site discovery, full functional verification, the ability to perform multi-variate searches of the site structure, and the ability to respond quickly to changes in the market and the business environment are crucial to online success. The evaluation of structure, correct and complete web analytics data, AND the opinions of users are all vital elements in achieving online success.

By: sladner

sladner — Wed, 01 Aug 2007 14:37:27 +0000

You’re right, Dave, a person is always involved. But the problem comes when people too much faith in the number spouted out in analytics tools, even when they’re reviewed by an experienced analyst.

My point is that there are fundamental limitations to the data you get in these tools, and they are no substitute for in-person observation.

By: Dave Hamel

Dave Hamel — Wed, 01 Aug 2007 13:17:47 +0000

I’m don’t agree. Web analystic tools have their place but don’t forget the actual analysis is done by a person. That person usually comes with some form of experience.