Which Comes First, The Test or the Metric?

A Web marketer commented to me recently that he thought his firm should focus first on basic “blocking and tackling” before moving on to more sophisticated efforts. When I asked what he considered basic, he said analyses like understanding customer value. He considered the advanced projects to be things like site optimization and multi-variate testing.

My initial reaction was that he had it backwards. To me, it seems harder to assemble accurate customer data across multiple channels and systems than to implement off-the-shelf testing software.

But maybe he had it right. Even though assembling comprehensive customer data may be much more challenging, optimization without a proper measurement capability can lead to bad decisions. So even though it might be quicker and easier to set up the testing systems, it’s might be best to concentrate on assembling the customer data.

As a devotee of lifetime value, I find this a comforting notion. But the hard-nosed businessperson in me says testing will yield immediate benefits even if it uses short-term measures like response rate. You need to look at the back end too, and as quickly as possible. But building a comprehensive measurement system may take a long time. Not testing until that's done means deferring the benefits that testing can provide. Unless testing is likely to lead to harmful choices, it's worth moving ahead even with imperfect measurements.