Webinar, December 18: How Marketers Can (Finally) Get Good Customer Data

Let’s face it: no real work will get done next week, what with all the holiday parties and caroling and so forth. So you might as well set aside 2:00 to 3:00 p.m. Eastern time on Wednesday, December 18 and register for the Webinar I’m co-presenting with RedPoint Global on Customer Data Platforms.

In addition to uncovering the secret relationship between cuneiform and Justin Bieber,

you’ll learn about our latest discovery: a new species of Customer Data Platform, bringing the known total to four. We’ll even provide a handy field guide to identifying which is which. Join us, and gain enough new information to fuel your party conversations for the rest of the holiday season!

Read More

New Study: Three Types of Customer Data Platform Address Cross-Channel Marketing Needs

My detailed study of Customer Data Platforms should be released next week. Now that the information is assembled, I can at last pull back and get a good overview of what I’ve found.

Perhaps the most interesting discovery has been that the CDP vendors cluster into three main groups.

• B2B data enhancement. These build a large reference database of companies and employees, which they match against records imported from their clients. They generally return corrected and enhanced data and lead scores based on models built from the client’s customer files. Their reference databases are built from multiple public, commercial, and proprietary sources, and are assembled using sophisticated matching engines. Most also perform their own scans of Web sites and social networks to extract sales-relevant information such as technology use and changes that suggest buying opportunities. These vendors vary considerably in the data they return, ranging from lead scores only to recommended marketing treatments to full customer profiles. Some also provide prospect lists of companies that are not already in the client’s own database. CDP vendors in this group include Infer, Lattice Engines, Mintigo, and ReachForce.

These systems compete with non-CDP products which also add or enhance prospect records but do not maintain a database with their clients’ customers. These include Web scanning systems such as InsideView, LeadSpace, and SalesLoft, and general data compilers including NetProspex, Demandbase, Data.com, ZoomInfo, and OneSource. The predictive modeling features also compete to some degree with end-user-oriented marketing analytics and modeling software such as Birst, GoodData, Cloud9 Analytics, AutoBox, and Predixion Software. Data cleansing competitors include services from firms such as D&B, as well as data management software for technical users such as Informatica, Experian QAS, and FullContact.

• Campaigns. These systems build a multi-source marketing database from the client’s own data and either recommend marketing treatments to execution systems or execute marketing campaigns directly. These are primarily used for consumer marketing although they also have B2B clients. Most have sophisticated matching capabilities. This group includes Silverpop with its Universal Behavior feature, NICE’s Causata, AgilOne, and RedPoint.

This group competes with conventional consumer marketing automation products, which provide similar campaign management abilities but lack the CDPs' database flexibility, database management, and customer matching features.

• Audience management. These systems build a database of customers and their responses to online display advertisements. They then build models that predict the customers’ probability of responding to future advertisements and provide recommendations for how much to bid and which content to display. These systems perform the same basic functions as standard online audience management systems (Data Management Platforms, or DMPs) and provide the same very quick responses needed for real time bidding (usually under 100 milliseconds). The major difference is that they also recommend messages in other channels, such as Web site personalization or email campaigns. Like DMPs, they work primarily at the Web cookie level, can link cookies known to relate to the same customer, and can be linked to actual customer names and addresses in external systems. This group includes IgnitionOne, [x+1], and Knotice.

This group overlaps with recommendation and ad targeting engines and DMP systems. Those products provide similar functions but do not track identified individuals and are often limited to single channel executions.

Given that each group addresses a different business need, you might wonder why I think they should all be lumped together under the CDP label. Quite simply, it’s because they are all addressing a portion of the same larger problem, which is how marketers can get a complete view of their customers and use that view to coordinate treatments across channels. What marketers truly need is a combination of the features from each group: data enhancement from external sources, for consumers as well as B2B; sophisticated customer matching and treatment selection; and integration of online advertising audiences with traditional customer databases. Each of these systems has the potential to grow into a complete solution, and the normal dynamics of software industry growth will push them towards pursuing that potential. So I expect the categories to overlap increasingly over the next few years and eventually merge into complete Customer Data Platforms as I envision them.

Incidentally and tangentially related: I'll be giving a Webinar with ReachForce on October 2 on Data Quality for Hipsters, a name that started as a joke but does make the point that data quality is essential for cutting-edge marketing.  YOLO, so you might as well attend.  I'm already working on the mustache.

Read More

What's New at DataFlux? I Thought You'd Never Ask.

What with it being Valentine’s Day and all, you probably didn’t wake up this morning asking yourself, “I wonder what’s new with DataFlux?” That, my friend, is where you and I differ. Except that I actually asked myself that question a couple of weeks ago, and by now have had time to get an answer. Which turns out to be rather interesting.

DataFlux, as anyone still reading this probably knew already, is a developer of data quality software and is owned by SAS. DataFlux’s original core technology was a statistical matching engine that automatically analyzes input files and generates sophisticated keys which are similar for similar records. This has now been supplemented by a variety of capabilities for data profiling, analysis, standardization and verification, using reference data and rules in addition to statistical methods. The original matching engine is now just one component within a much larger set of solutions.

In fact, and this is what I find interesting, much of DataFlux’s focus is now on the larger issue of data governance. This has more to do with monitoring data quality than simple matching. DataFlux tells me the change has been driven by organizations that face increasing pressures to prove they are doing a good job with managing their data, for reasons such as financial reporting and compliance with government regulations.

The new developments also encompass product information and other types of non-name and address data, usually labeled as “master data management”. DataFlux reports that non-customer data is is the fastest growing portion of its business. DataFlux is well suited for non-traditional matching applications because the statistical approach does not rely on topic-specific rules and reference bases. Of course, DataFlux does use rules and reference information when appropriate.

The other recent development at DataFlux has been creation of “accelerators”, which are prepackaged rules, processes and reports for specific tasks. DataFlux started offering these in 2007 and now lists one each for customer data quality, product data quality, and watchlist compliance. More are apparently on the way. Applications like this are a very common development in a maturing industry, as companies that started by providing tools gain enough experience to understand how the applications commonly built with those tools. The next step—which DataFlux hasn’t reached yet—is to become even more specific by developing packages for particular industries. The benefit of these applications is that they save clients work and allow quicker deployments.

Back to governance. DataFlux’s movement in that direction is an interesting strategy because it offers a possible escape from the commoditization of its core data quality functions. Major data quality vendors including Firstlogic and Group 1 Software, plus several of the smaller ones, have been acquired in recent years and matching functions are now embedded within many enterprise software products. Even though there have been some intriguing new technical approaches from vendors like Netrics and Zoomix, this is a hard market to penetrate based on better technology alone. It seems that DataFlux moved into governance more in response to customer requests than due to proactive strategic planning. But even so, they have done well to recognize and seize the opportunity when it presented itself. Not everyone is quite so responsive. The question now is whether other data quality vendors will take a similar approach or this will be a long-term point of differentiation for DataFlux.

Read More

Low Cost CDI from Infosolve, Pentaho and StrikeIron

As I’ve mentioned in a couple of previous posts, QlikView doesn’t have the built-in matching functions needed for customer data integration (CDI). This has left me looking for other ways to provide that service, preferably at a low cost. The problem is that the major CDI products like Harte-Hanks Trillium, DataMentors DataFuse and SAS DataFlux are fairly expensive.

One intriguing alternative is Infosolve Technologies. Looking at the Infosolve Web site, it’s clear they offer something relevant, since two flagship products are ‘OpenDQ’ and ‘OpenCDI’ and their tag line is ‘The Power of Zero Based Data Solutions’. But I couldn't figure out exactly what they were selling since they stress that there are ‘never any licenses, hardware requirements or term contracts’. So I broke down and asked them.

It turns out that Infosolve is a consulting firm that uses free open source technology, specifically the Pentaho platform for data integration and business intelligence. A Certified Development partner of Pentaho, Infosolve has developed its own data quality and CDI components on the platform and simply sells the consulting needed to deploy it. Interesting.

Infosolve Vice President Subbu Manchiraju and Director of Alliances Richard Romanik spent some time going over the details and gave me a brief demonstration of the platform. Basically, Pentaho lets users build graphical workflows that link components for data extracts, transformation, profiling, matching, enhancement, and reporting. It looked every bit as good as similar commercial products.

Two particular points were worth noting:

- the actual matching approach itself seems acceptable. Users build rules that specify which fields to compare, the methods used to measure similarity, and similarity scores required for each field. This is less sophisticated than the best commercial products, but field comparisons are probably adequate for most situations. Although setting up and tuning such rules can be time-consuming, Infosolve told me they can build a typical set of match routines in about half a day. More experienced or adventurous users could even do it for themselves; the user interface makes the mechanics very simple. A half-day of consulting might cost $1,000, which is not bad at all when you consider that the software itself is free. The price for a full implementation would be higher since it would involve additional consulting to set up data extracts, standardization, enhancement and other processes, but cost should still be very reasonable. You’d probably need as much consulting with other CDI systems where you'd pay for the software too.

- data verification and enhancement is done by calls to StrikeIron, which provides a slew of on-demand data services. StrikeIron is worth knowing about in its own right: it lets users access Web services including global address verification and corrections; consumer and business data lookups using D&B and Gale Group data; telephone verification, appends and reverse appends; geocoding and distance calculations; Mapquest mapping and directions; name/address parsing; sales tax lookups; local weather forecasts; securities prices; real-time fraud detection; and message delivery for text (SMS) and voice (IVR). Everything is priced on a per use basis. This opens up all sorts of interesting possibilities.

The Infosolve software can be installed on any platform that can run Java, which is just about everything. Users can also run it within the Sun Grid utility network, which has a pay-as-you-go business model of $1 per CPU hour.

I’m a bit concerned about speed with Infosolve: the company said it takes 8 to 12 hours to run a million record match on a typical PC. But that assumes you compare every record against every other record, which usually isn’t necessary. Of course, where smaller volumes are concerned, this is not an issue.

Bottom line: Infosolve and Pentaho may not meet the most extreme CDI requirements, but they could be a very attractive option when low cost and quick deployment are essential. I’ll certainly keep them in mind for my own clients.

Read More

Using Lifetime Value to Measure the Value of Data Quality

As readers of this blog are aware, I’ve reluctantly backed away from arguing that lifetime value should be the central metric for business management. I still think it should, but haven’t found managers ready to agree.

But even if LTV isn’t the primary metric, it can still provide a powerful analytical tool. Consider, for example, data quality. One of the challenges facing a data quality initiative is how to justify the expense. Lifetime value provides a framework for doing just that.

The method is pretty straightforward: break lifetime value in its components and quantify the impact of a proposed change on whichever components will be affected. Roll this up to business value, and there you have it.

Specifically, such a breakdown would look like this:

Business value = sum of future cash flows = number of customers x lifetime value per customer

Number of customers would be further broken down into segments, with the number of customers in each segment. Many companies have a standard segmentation scheme that would apply to all analyses of this sort. Others would create custom segmentations depending on the nature of the project. Where a specific initiative such as data quality is concerned, it would make sense to isolate the customer segments affected by the initiative and just focus on them. (This may seem self-evident, but it’s easy for people to ignore the fact that only some customers will be affected, and apply estimated benefits to everybody. This gives nice big numbers but is often quite unrealistic.)

Lifetime value per customer can be calculated many ways, but a pretty common approach is to break it into three major factors:

- acquisition value, further divided into the marketing cost of acquiring a new customer, the revenue from that initial purchase, and the fulfillment costs (product, service, etc.) related to that purchase. All these values are calculated separately for each customer segment.

- future value, which is the number of active years per customers times the value per year. Years per customer can be derived from a retention rate or a more advanced approach such as a survivor curve (showing number of customers remaining at the end of each year). Value per year can be broken into the number of orders per year times the value per order , or the average mix of products times the value per product). Value per order or product can itself be broken into revenue, marketing cost and fulfillment cost.

Laid out more formally, this comes to nine key factors:

- number of customers

- acquisition marketing cost per customer
- acquisition revenue per customer
- acquisition fulfillment cost per customer

- number of years per customer
- orders per year
- revenue per order
- marketing cost per order
- fulfillment cost per order

This approach may seem a little too customer-centric: after all, many data quality initiatives relate to things like manufacturing and internal business processes (e.g., payroll processing). Well, as my grandmother would have said, feh! (Rhymes with ‘heh’, in case you’re wondering, and signifies disdain.) First of all, you can never be too customer-centric, and shame on you for even thinking otherwise. Second of all, if you need it: every business process ultimately affects a customer, even if all it does is impact overhead costs (which affect prices and profit margins). Such items are embedded in the revenue and fulfillment cost figures above.

I could easily list examples of data quality changes that would affect each of the nine factors, but, like the margin of Fermat’s book, this blog post is too small to contain them. What I will say is that many benefits come from being able to do more precise segmentation, which will impact revenue, marketing costs, and numbers of customers, years, and orders per customer. Other benefits, impacting primarily fulfillment costs (using my broad definition), will involve more efficient back-office processes such as manufacturing, service and administration.

One additional point worth noting is many of the benefits will be discontinuous. That is, data that's currently useless because of poor quality or total absence does not become slightly useful because it becomes slightly better or partially available. A major change like targeted offers based on demographics can only be justified if accurate demographic data is available for a large portion of the customer base. The value of the data therefore remains at zero until a sufficient volume is obtained: then, it suddenly jumps to something significant. Of course, there are other cases, such as avoidance of rework or duplicate mailings, where each incremental improvement in quality does bring a small but immediate reduction in cost.

Once the business value of a particular data quality effort has been calculated, it’s easy to prepare a traditional return on investment calculation. All you need to add is the cost of improvement itself.

Naturally, the real challenge here is estimating the impact of a particular improvement. There’s no shortcut to make this easy: you simply have to work through the specifics of each case. But having a standard set of factors makes it easier to identify the possible benefits and to compare alternative projects. Perhaps more important, the framework makes it easy to show how improvements will affect conventional financial measurements. These will often make sense to managers who are unfamiliar with the details of the data and processes involved. Finally, the framework and related financial measurements provide benchmarks that can later be compared with actual results to show whether the expected benefits were realized. Although such accountability can be somewhat frightening, proof of success will ultimately build credibility. This, in turn, will help future projects gain easier approval.

Read More