Marketing Automation News from Dreamforce: B2B More Integrated, B2C Stays Separate

I spent the early part of this week at Salesforce.com’s annual Dreamforce conference. Here are my observations.

The big news was for geeks. The main theme of the conference was Salesforce1, a new set of technologies that make it vastly easier to deliver and integrate mobile versions of Salesforce-based applications. It is apparently a major technical accomplishment and at least one of my technical friends was hugely impressed. But I can’t say I personally found it all that exciting. Perhaps we’ve reached the point where we expect technology to do pretty much everything, so the line between what's already available and what's new is only visible to experts.  Any way you slice it, focusing on platform technology is much less exciting than last year's vision of "social enterprise".

The bad news was for B2B marketing automation. Conference presentations confirmed that Pardot, the B2B marketing automation system that Salesforce acquired as part of its ExactTarget acquisition, has been separated from the rest of ExactTarget and made part of the Sales cloud. There, Pardot is described only as providing lead scoring and nurture programs, which ignores landing pages, behavior tracking, and other features that B2B marketing automation usually provides (and Pardot includes). In terms of infrastructure, Pardot will eventually work directly from the CRM data objects, rather than maintaining its own synchronized database. (Data outside the CRM structure, such as detailed Web behaviors, will remain separate.)

What this means is that Salesforce sees B2B marketing automation as just an appendage of sales automation.  This is pretty much the same constricted view of marketing automation that Salesforce management has held all along.  The logical consequence is to make lead scoring and nurture campaigns standard features within the Sales offering and discard Pardot as a separate product.  I should stress that no one at Salesforce said this was their plan, but it seems inevitable. If and when that does happen, only the most demanding companies will purchase a separate B2B marketing automation product.

To put a more optimistic spin on the same news: Salesforce will continue to let independent B2B marketing automation apps synch with Sales.  If Salesforce does merge Pardot features into its core Sales product, then marketers who have a more expansive view of B2B marketing automation functions (or who simply want a system of their own) will be forced to buy from someone else.

The interesting news was that B2C marketing automation remains separate. Salesforce’s list of business groups includes the Sales Cloud, Service Cloud, and ExactTarget Marketing Cloud. Did you notice that just one of these has its own brand? As this suggests, and conference presentations confirm, Salesforce has kept B2C marketing distinct from its Sales and Service businesses, most importantly at the data and platform levels. The ExactTarget Marketing Cloud does now include Salesforce’s previously-purchased social marketing components, Radian6 social monitoring and Social.com social advertising. It also includes the iGoDigital predictive personalization technology that came along with the ExactTarget acquisition.

Salesforce did announce some plans to integrate the Marketing cloud with Sales and Service, but they are pretty much arm’s length: Marketing can receive alerts about changes in Sales (and I assume Service) data, even though that data remains separate; Sales and Service can send emails through the ExactTarget engine; Sales and Service can receive content recommendations from the Marketing predictive modeling tool. As near as I can tell, this is the same type of API-level integration available with any third-party system. For what it’s worth, the ExactTarget Marketing Cloud APIs are also part of Salesforce1, but don’t confuse that with sharing the same underlying platform.They don't.

The good news is the B2C marketing vision. It’s not really surprising that Salesforce kept its B2C platform separate, since Salesforce's core technology isn’t engineered for the massive data volumes and analytical processing needed for B2C in general and consumer Web marketing in particular. Happily, this technical necessity is accompanied by what strikes me as a sound vision for customer management.  ExactTarget framed this around three goals: single view of the customer; managing the customer journey; and personalized content across all channels and devices. It described major features for each of these: a unified metadata layer to access (and optionally import) data from all sources; a “customer journey” engine to manage multi-step, branching flows; and predictive modeling to select the best offers and contents across email and Web messages.

This felt like a more coherent approach than Salesforce described for the Sales cloud, where external data and predictive modeling in particular were barely mentioned (or, more precisely, are still being left to App Exchange partners). The ExactTarget cloud still lacks tools to associate customer identities across email, phone, postal, social, and other systems, although there are plenty of partners to provide them. I didn’t get a close look at the details of the ExactTarget functions, which will really determine how well it competes with other customer management platforms. But the general approach makes sense.

News of the revolution may be exaggerated. Salesforce argued during the AppExchange Partner keynote that the AppExchange and Salesforce platform have created a “golden age of enterprise apps” by enabling small software developers to sell to big enterprises. One part of the argument is that the platform itself lets small vendors break through the credibility and scalability barriers that have historically protected large enterprise software vendors. The other is that end-users can purchase and deploy apps without involving the traditional gatekeepers in enterprise IT departments. A corollary to this is that end-users have different priorities than IT buyers – in particular, end users care more about ease of use – so successful software will be different.

Of course, this is exactly what the AppExchange partners wanted to hear and exactly the strategy behind Salesforce’s platform approach in the first place. But that doesn’t necessarily make it untrue: and, if correct, it would indeed be a revolution in the enterprise software industry.

But some revolutions are bigger than others.  Even in an app-based world, individual users won't be making personal decisions about how to run core business processes.  Rather, systems will be chosen at the department level because companies can more or less safely assume that whatever the department chooses will integrate smoothly with the corporate backbone. That's certainly a change but bear in mind that departmental buyers will have the same preference as corporate IT groups for working with the smallest possible number of vendors. This means there will still be the familiar tendency for individual vendors to add more functions over time. So industry dynamics may change less than you’d expect.

Read More

idio Does Sophisticated Content Recommendation

Systems in our new Guide to Customer Data Platforms range from B2B data enhancement to campaign managers to audience platforms. This may lead you to wonder whether there’s anything we actually left out.  In fact, there was: although the final choices were admittedly a bit subjective, I tried to ensure the report only included systems that met specific critieria including a persistent database, customer-level data, marketer control, and marketing-related outputs to external systems. In most cases, I could judge whether a system fit before doing a lot of detailed research. But a few systems were so close to the border that I only made the final call after I had evaluated them in depth.

idio was one of those. The company positions itself as a tool to deliver “personalized and relevant multi-channel communications”, which sure sounds like a CDP.  Indeed, it meets almost all the critieria listed above, including the most important one of building and maintaining a persistent customer database. But I ultimately excluded idio because it is tightly focused on identifying the content that customers are most likely to select, a function I felt was too narrow for a proper CDP. The folks at idio didn’t necessarily agree with this judgment, and pointed to planned developments that could indeed change the verdict (more about that later).  But, for now, let’s not worry about CDPs and take idio on its own terms.

The full description on idio's home page reads “idio understands your customer’s interests and intent through the content they consume and uses this to deliver personalized and relevant multi-channel communications” and that pretty much says it all. What idio does is ingest content – typically from a publisher such as ESPN, Virgin Media, Guardian Media, or eConsultancy (all clients) – but also from brands with large content stores such as Diageo, Unilever, and C Spire (also all clients). It uses advanced natural language processing to extract entities and concepts from this content, classifying it with the vendor’s own 23 million item taxonomy.

The system then monitors the content selected by its clients’ customers in emails, Web pages, mobile platforms, and some social platforms and builds an interest profile for each customer.  This in turn lets the system recommend which existing content the customer is most likely to select next. The recommendations are typically fed back to execution systems, such as email generators or Web content managers, which insert links to the recommended content into Web pages, emails, or newsletters.  Reports show selection rates by content, segment, or campaign, and can also show the most common topics published and the most commonly selected. Pricing is based on recommendation volume and starts around $60,000 per year for ten million recommendations.

Describing idio’s basic functions makes it sound similar to other recommendation systems, which doesn’t really do it justice. What sets idio apart are the details and technology.

• Content can include ads, offers and products as well as conventional articles.
• The natural language system classifies content without users tagging each item, a huge labor savings where massive volumes are involved, and can handle most European languages.
• idio's largest client ingests more than 1,000 items per day and stores more than one million items, a scale far beyond the reach of systems designed to choose among a couple hundred offers or products.
• Interest profiles take into account the recency of each selection and give different weights to different types of selections – e.g., more weight to sharing something than just reading it.
• Users can apply rules that limit the set of contents available in a particular situation.
• The system returns recommendations in under 50 milliseconds, which is fast enough to support online advertising selection.
• It stores customer data in a schema-less system that can make any type of input available for segmentation and reporting, although not to help with recommendations.
• It can build a master list of identifiers for each individual, allowing systems to submit any identifier and access a unified customer profile.
• It can return a content abstract, full text, images, or HTML, or simply a pointer to content stored elsewhere.
• It captures responses directly as the content is presented.

Most of these capabilities are exceptional and the combination is almost surely unique. The ultimate goal is to increase engagement by offering content people want, and idio reports it has doubled or even quadrupled selection rates vs. previous choices. All this explains why a small company whose product launched in 2011 has already landed so many large enterprises among its dozen or so clients.

Impressive as it is, I don’t see idio as a CDP because it is primarily limited to interest profiles and  content recommendations. What might yet change my mind is idio’s plan to go beyond recommending content based on likelihood of response, to recommending content based on its impact on reaching future goals such as making a purchase. The vendor promises such goal-driven recommendations in about six months.

Idio is also working on predicting future interests, based on behavior patterns of previous customers.  For example, someone buying a home might start by researching schools, then switch to real estate listings, then to mortgages, then moving companies, and so on. Those predictions could be useful in their own right and also feed predictions of future value, which could support conventional lead scoring applications. Once those features become available, idio may well be of interest to buyers well beyond its current customer base and would probably be flexible enough to serve as as Customer Data Platform.

Read More

Infer Keeps It Simple: B2B Lead Scores and Nothing Else

I’ve nearly finished gathering information from vendors for my new study on Customer Data Platform systems and have started to look for patterns in the results. One thing that has become clear is that the CDP vendors fall into several groups of systems that are similar to each other but quite different from the rest. This makes sense: most of the existing CDP systems were built to solve specific problems , not as general-purpose data platforms. Features will probably converge as vendors extend their products to attract more clients. But right now the groups are quite distinct.

One of these categories is systems for B2B lead scoring. I found three CDPs in this group: Lattice Engines (which I reviewed in April), Mintigo (reviewed in June), and Infer, which I'm reviewing right now.

Like the others, Infer builds a proprietary database of pretty much every company on the Internet by scanning Web sites, blogs, social media, government records, and other sources for company information and relevant events.  It then imports CRM and marketing automation data from its clients' systems, enhances the imported records with information from its big proprietary database, and builds predictive models that score companies and individuals on their likely win rate, conversion rate, deal size, and lifetime revenue.

The models are applied to new records as they enter a client’s system, creating scores that are returned to marketing automation and CRM to use as those systems see fit. The most typical application is deciding which leads should go to sales, be further nurtured by marketing automation,  or discarded entirely. But Infer customers also use the scores to prioritize leads for salespeople within CRM, to measure the quality of leads produced by a marketing program, assess salesperson performance based on the quality of leads they received, and even adjust paid search campaigns based on the quality of leads generated by each source and keyword.

Infer differs from its competitors in many subtle ways: the scope of its data sources, its matching processes to assemble company and individual data, the exact types of scores it produces, its modeling techniques, and reporting.  It also differs in one very obvious way: it returns only scores, while competitors return both scores and enhanced profiles on individual prospects.  Infer gathers the individual detail needed for such profiles, but has decided so far not to make them available. Its reasoning is that scores provide the major value from its system and profiles would detract from them – perhaps because sales people might ignore them scores in favor of profile data. Focusing on scores alone also makes Infer simpler to set up, operate, and understand.

Infer might be right, but it’s hard to imagine they'll will stick with this position once they start selling directly against competitors that offer scores plus profiles.  They will surely lose many deals for that reason alone.  On the other hand, Infer’s initial clients have been companies where free trials versions generate huge lead volumes, including Box, Tableau, NitroPDF, Zendesk, Jive and Yammer. Scores that accurately filter non-productive leads are more important to those companies than individual lead profiles.  Perhaps there are enough such firms for Infer to succeed by selling only to them.

Whether or not Infer expands its outputs, it faces another challenge: convincing buyers that its scores and data are better than its competitors. This might well be true: based on the information I’ve gathered, Infer seems to have a richer set of data sources and more sophisticated identity matching than at least some competitors. But my impressions may be wrong, and most buyers will won’t dig deeply enough to form an opinion.  Instead, their eyes will glaze over when the vendors start to get into the details, and they’ll simply assume that everybody’s data, matching, and modeling are roughly equivalent.

The only real way to measure relative quality is through competitive testing of which scores work better.  Each buyer needs to run her own tests since results may vary from business to business. How many buyers will take the time to do this, and which vendors will agree to cooperate, is a very open question.

That said, I did speak with some current Infer users, who were quite delighted with how easy it had been to deploy the system and with results to date. This is hardly a random sample – these were pioneer users (the system was only launched about a year ago) and hand-picked by the vendor. But their experience does confirm that performance is solid.

Infer pricing is based on the number of records processed and connected systems.  The vendor doesn’t reveal the actual rates but did say it is looking at options to make the system more affordable for smaller clients.

Read More

AgilOne Combines Marketing Database, Analytics and Execution: Yep, That's a Customer Data Platform

Well, this is embarrassing.

Here I am, all excited about discovering a new category of Customer Data Platform systems, which combine marketing database management, predictive modeling, and decision engines. Then I bump into Omer Artun, CEO of AgilOne , which he founded seven years ago to combine marketing database management, predictive modeling, and decision engines. It makes me feel much less clever.

But I guess I can’t hold that against AgilOne. As Artun tells the story, the company was created to provide marketers with a packaged, cloud-based version of the advanced data management, analytics, and execution capabilities that are usually available only to the largest and richest firms. The key is a set of 400 standard metrics, which AgilOne derives by mapping each client’s unique data into a standard structure. This, combined with advanced machine learning techniques, lets AgilOne build ten standard predictive models (engagement, next product, lifetime value, etc.) and three standard cluster models (products, behaviors, and brands) with minimal effort. The system builds on these to deliver packages of standard alerts, reports, guided analytics, individual customer profiles, and campaign lists. It also makes its data and predictions accessible to external systems such as call centers and Web sites via real time API calls, so those systems can use them to guide their own customer treatments.

This quick summary doesn’t do justice to the cleverness or sophistication of AgilOne’s approach. Clever, because the standardization allows it to quickly and cheaply deliver a full stack of capabilities, starting with database building and ending with advanced analytics, recommendations, and execution. Sophisticated, because it tailors the standard structures to each client’s business, so what it delivers isn’t some simple, cookie-cutter output.

Some of the tailoring is unavoidably manual, such as mapping client data sources to the standard data model. But much is highly automated, such as predictive models, clusters, and recommendations. I was particularly intrigued by the standard alerts, which look for significant changes in key performance indicators such as churn, margin, or average order value.  That sort of alerting is exactly what I've long felt marketers really wanted from their analytics tools.  AgilOne takes this a step further by automatically listing the data attributes with the greatest statistical impact on each item. The company refers to these items as goals to prioritize, which is a bit of a stretch – the most powerful variable isn’t necessarily the one marketers should focus on the most. But, as Damon Runyon said*, that’s the way to bet.

The system also recommends actions related to each alert, such as certain types of marketing campaigns. Again, there’s a bit less here than meets the eye, since the recommendations are drawn from a knowledgebase that’s the same for all clients. But that’s still better than nothing, and clients can customize their copy of the knowledgebase if they want.

The other especially noteworthy strength of AgilOne is data preparation. My original concept of the Customer Data Platform included customer data integration, which involves standardizing and matching customer records from different systems. I’ve pulled back from that because almost none of the vendors actually do such processing. Most assume it will be done elsewhere, or not at all, and only associate records with an exact match on a key such as a customer ID.  AgilOne does the hard stuff: quality checks, outlier detection, name parsing, address standardization, geocoding, phonetic matching, persistent ID management, and more. This is also highly automated and uses the company’s own technology. The lack of these capabilities prevents many companies from building a truly integrated customer database at many companies, so it’s extremely valuable for AgilOne to provide it.

If AgilOne has a weakness, it's at the execution end of the process.  Users can set up campaigns that generate lists on demand or on a regular schedule.  But I didn't see multi-step campaign flows or sophisticated decision management, such as arbitration across multiple eligible offers.  Some of that can probably be managed through advanced filters and custom models, which the system does provide.  However, making it truly accessible to non-technical users requires a specialized interface that the system apparently lacks.

While AgilOne just recently appeared on my personal radar, plenty of other people had already noticed: the company says nearly 100 brands are using the system. Sales efforts have been concentrated among mid-size B2C organizations, typically with at least 200,000 customers and $15 to $20 million in revenue. Pricing is published on the company Web site and is based on the features used and number of active customers. Entry price for the complete set of features starts around $9,000 per month.



_________________________________________________________________________________
*“The race is not always to the swift nor the battle to the strong, but that's the way to bet.” Runyon himself credited Chicago journalist Hugh Keough.

Read More

Mintigo InterestBase Harvests Web and Social Data for Marketing and Sales

Every marketer recognizes that the Web and social media could be rich sources of information about customers and prospects. But harvesting that data has been frustratingly difficult.  Doing it yourself  takes multiple tools to gather different kinds of information, and then patching the result together into personal profiles. Most tools do little more than keyword searches, which only capture a fraction of the potential information and only cover keywords that marketers know in advance are important.

More advanced technology does exist. Semantic engines can extract information such as executive changes and product announcements from press releases and social media profiles. Sentiment analysis can (with limited reliability) detect the attitudes that individuals express. Identity aggregators can link email, social media, and other addresses for the same individual. Predictive models can show how different attributes correlate with targeted behaviors such as purchasing a product.

Few marketers have the skill or resources to pull all these tools together for themselves. Vendors are another matter: there’s inherent scale economy to scanning the Web and social media once and applying the results to many different clients. I recently wrote about Lattice Engines,  which has assembled these pieces to create prospect lists. Infer starts with your own customer data, enhances it with information mined from the Web, and generates predictive scores.

Mintigo has also been mining Web and social data to build prospect lists, starting in 2011. This week it announced a new platform, InterestBase, that gives clients an interface to define target groups, analyze group members’ interests, push prospect lists to marketing automation and CRM systems, and enhance individual lead records.

The foundation of InterestBase is a central repository of 30 million names and 3 million companies (and growing), built by scanning Web sites and social media for job postings, product and technology names, group memberships, accounts followed, hashtags, Javascript calls, and other information. The system uses this data to assign individuals and companies such attributes as job title, company size, technologies used, hiring plans, and interest scores for products and topics. Marketers can use titles and other attributes to define their own target groups, called personas.

Lists containing members of a persona can be assigned to marketing campaigns and sent to external marketing automation or CRM systems for execution. Connectors are currently available for Marketo and Salesforce.com, with an Eloqua connector due soon. A campaign list can include the entire persona universe or a quantity specified by the user. Once the campaign is run, responders are loaded back into Mintigo and the system will identify attributes that distinguish them from non-responders.

Clients can also upload their own lists of customers or campaign respondents.  Mintigo will determine which attributes correlate with group membership, display the most important ones in reports, and use the findings in predictive models that score the entire database on likelihood of purchase or response. Clients can also upload other lists for Mintigo to enhance with its own information. This enhanced data can be used in lead scoring or to help guide salespeople.  External systems like Web sites can also accessed the data in real time via API calls.

The features are interesting, but what really matters about Mintigo is the data: fresh, powerful, and unique information about a large share of the business universe. Richer information lets Mintigo clients identify new prospects they’d otherwise miss, distinguish strong prospects from weak ones, and target messages to each prospect’s interests. The result is substantially more effective marketing and sales operations, finally letting marketers use data the Web has so tantalizingly exposed.

In case you're wondering, I do consider Mintigo a Customer Data Platform: it assembles a persistent customer database, uses predictive models to classify the members, and makes the data available to external systems for marketing execution.  

Pricing for InterestBase is based on the number of names in the client’s prospect pool, based on automated analysis of their actual customers. An average client starts around $3,000 per month.

Read More

Provenir Adds Social Listening to Customer Decisions: Another Customer Data Platform

I’m still collecting examples to illustrate my new category of Customer Data Platform (CDP) systems. The latest is Provenir, a company founded in 1992 that has long sold a system to make credit risk and fraud decisions in real time. Over the past year, the company has added “social listening” capabilities and begun offering itself to marketing agencies as a customer interaction manager. It has met with good success and is now offering its “social listening platform” more broadly. *


It’s a slight stretch to call Provenir a CDP, because it doesn’t manage a permanent customer database.  Rather, like most interaction managers, it calls data from external sources during each decision.  But Provenir does have some customer matching capabilities and stores at least some information internally. Moreover, it completely meets the other three CDP criteria: predictive modeling, real-time decisions/recommendations executed through external systems, and a non-technical user interface. It’s also sold as the “glue” connecting data sources, modeling, and execution systems, which is exactly the role played by a CDP.  So, what the heck…welcome to the club!

Provenir is organized around process flows, which cover a particular task such as reacting to a Web site visit. Users define each process by building a flow chart, or, as the cool kids call them today, a graph.** These, um, graphs***, can contain branches, loops, and other advanced structures.  The nodes can also contain other graphs that define a subprocess in more detail. Nodes can perform a wide range of operations including data gathering, calculations, updates, decisions, and messages to external systems. Although setting these up is inevitably rigorous, Provenir makes it as painless as possible by providing help such as letting users draw lines to map fields from one system to another; building rules through score cards, tables and decision trees; and warning if a flow is incomplete.

Provenir relies on external systems to assemble, integrate, and store customer data.  Users can build matching processes with system graphs, although the vendor recommends connecting to other products to load reference data or do advanced "fuzzy" matching.  Provenir can monitor source systems for selected events and issue queries to assemble data as needed. The social listening features can monitor Twitter for keywords and Tweets by specified individuals.  These can trigger process flows that can retweet a message, send a direct Twitter message to the poster, or respond through another channel. The system can also monitor and post messages on Facebook. Other channels will be added over time.

Predictive modeling in Provenir is also done in external systems. The system can import PMML code or call models in SAS, R, or even Excel. Data mapping functions can automatically extract the list of required variables from PMML, do basic transformations and calculations when loading model inputs, and manage parameters, constants, and local variables.

Decisioning is Provenir’s greatest strength. The process flow…I mean graph…is inherently very flexible, and the ability to define rules as tables, trees, score cards, and other formats adds even more power. Users can set up champion/challenger tests as splits within a process flow; results are stored in a database for analysis and reporting. Users can also build simulated data sets, containing specified distributions of particular variables, and use these to forecast results of their flow designs. Such simulation is one mark of a mature decision system.

Provenir has some built-in messaging capabilities, but most decisions are executed externally.  The system has been connected with email, Web content management, call centers, campaign management, text messaging, and other execution platforms.

Pricing for Provenir’s social listening product is based on the size of the customer database. Starting price can be as a low as several thousand dollars per month. The system is usually sold on a Software-as-a-Service (SaaS) basis, but on-premise licenses are also available.


_______________________________________________________________
* For extra credit, compare and contrast Provenir’s primary Web site  with the site for their listening division.

** Defined in Wikipedia as “mathematical structures used to model pairwise relations between objects”.

*** Would it be even cooler to call them grafs or, better still, grafz?

Read More

I've Discovered a New Class of System: the Customer Data Platform. Causata Is An Example.

It has taken me a while to connect the dots, but I’m now pretty sure I see a new type of software emerging. These systems that gather customer data from multiple sources, combine information related to the same individuals, perform predictive analytics on the resulting database, and use the results to guide marketing treatments across multiple channels. This differs quite radically from standard marketing automation systems, which use databases built elsewhere, rarely include integrated predictive modeling, and are focused primarily on moving customers through multi-step campaigns. In fact, the new systems complement rather than compete with marketing automation, which they treat as just one of several execution platforms. The new systems can also feed sales, customer service, online advertising, point of sale, and any other customer-facing systems.

Given how much vendors and analysts love to create new categories, I’m genuinely perplexed that no one has yet named this one. I’ll step in myself, and hereby christen the concept as “Customer Data Platform”.  Aside from having a relatively available three letter abbreviation (see Acronym Finder for other uses of CDP), the merits of this name include:

- “Customer” shows the scope extends to all customer-related functions, not just marketing;
- “Data” shows the primary focus is on data, not execution; and
- “Platform” shows it does more than data management while supporting other systems

But, you may ask, is this really new? Certainly systems for Customer Data Integration (CDI) have been around for decades: these include specialized products like Harte-Hanks Trillium and SAS DataFlux, CDI features within general data management suites like Informatica and Pentaho, and integration within cloud-based business intelligence products like GoodData and Birst. Many of those products have limited capabilities for working with newer data sources like Web sites and social networks, but the real distinction between them and CDPs is that the older systems are mainly designed to assemble data.  Some also provide analytics, but they don't extend to real-time decisions based on predictive models.

Similarly, there have long been specialized systems for real-time interaction management (such as Infor Interaction Advisor and Oracle Real Time Decisions) and for predictive modeling (SAS, IBM SPSS, KXEN). Some interaction managers do create predictive models, and the really big vendors (IBM, SAS, Oracle) have all three key components (CDI, real-time decisions, and predictive models) somewhere in their stables. But systems that closely couple just those features with the goal of feeding data as well as recommendations to execution systems? Those are something new.

By now, you’re probably wondering if I’ll ever get around to actually naming the vendors I have in mind. I’ve recently written about some of them, including Reachforce/SetLogik and Lattice Engines.  I also include RedPoint in the mix, because it has all the key capabilities (database development, predictive models, and real time decisions) even though it also offers conventional campaign management. Others I haven’t yet written about include Mintigo and Gainsight. Of course, each has a different mix of features and its own market position.  Indeed, several have specifically told me they do not compete with the others. Fair enough, but I still see enough similarity to group them together.

All this is a very long-winded introduction to Causata, yet another member of this new class. By now, you can probably guess Causata’s main functions: assemble customer data from multiple sources, consolidate it by customer, place it in an analytics-friendly format, run predictive models against it, and respond in real time to recommendation requests from other systems including Web sites, email, banner ads, and call centers. And you’d be right.

But that’s not the end of the story. With any product, it’s the details that matter. Causata is particularly strong in the data management department, accepting both batch and real-time data feeds and storing data as different types of events (email sent, Web site visit, call center interaction, etc.), each having predefined attributes. The system also has a particularly sophisticated “identity association” service, which looks for simultaneous events involving different identifiers as a way to link them, and can chain identifiers that were linked at different times. When I spoke with Causata about two months ago, the association rules were pretty much the same for all clients, but they promised users would get more control in the future. Users could already choose which types of associations to use in specific queries.

Causata stores the assembled data in HBase, a Hadoop-based database management system that is particularly well suited to large data volumes, many different data types, and ad hoc queries. In addition to the raw data, the system can store derived values such as aggregations (e.g., number of Web page view in past 24 hours) and model scores. Users can run SQL queries to extract data for analysis and predictive modeling in third-party software including QlikView, Tableau, SAS, and R. Prebuilt QlikView reports show the predictive power of different variables for user-specified events. The lack of native analysis and modeling tools creates some friction for users, but also lets them stick with familiar products. So the pros and cons probably cancel each other out.

The system’s decision tools are straightforward. For each situation, users define a “decision engine” that can select among multiple options, such as campaigns, products, or marketing content. These options can have qualification rules. To make a decision, the system can test the options in sequence and pick the first one for which a customer is qualified, or pick the option with the highest predictive model score. Users can also specify a percentage of customers to receive a random option, to gather data for future decisions. An engine can return multiple decisions for situations that require more than one option, such as a Web page with several offers. Causata has some machine learning algorithms to help with the decision process. It plans to expand these to automatically select the best option in a given situation.

Decision engines are called by external systems through a Web services API that can respond in under 50 milliseconds. This is fast enough to manage Web banner ads – something not all interaction managers can achieve. Model scores and other data are updated in real time during an interaction.

Causata can be deployed on-premise by a client or as a cloud-based service. The vendor says a typical implementation starts with three or four data sources and is deployed in about 30 days – very fast for this type of system. In February, Causata introduced prebuilt applications for cross-sell, acquisition, and return programs in financial services, communications, and digital media. These will further speed deployment.

Pricing is based on the number of data sources and touchpoints, with additional charges based on data storage. Cost begins around $150,000 per year.

Read More

Lattice Engines Automates All Steps in Prospect Discovery

There’s nothing new about using public information to identify business opportunities: it’s why lawyers chase ambulances and bankers phone lottery winners. But the Internet has exponentially grown the amount of data available and made it easily accessible. What’s needed to fully exploit this resource is technology that automates the end-to-end process of assembling the information, identifying opportunities, and delivering the results to sales and marketing systems.

Lattice Engines was founded in 2006 to fill this gap. The system scans public databases, company Web pages, and selected social networks to find significant events such as title changes, product launches, job openings, new locations, and investments. It supplements this with data from the clients' own systems including customer profiles, Web site visits, and purchases. It then looks at past data to find patterns which predict selected outcomes, such as making a first purchase, buying an additional product, or renewing. It uses these patterns to identify the best current prospects for each outcome, and makes the lists available to marketing systems or sales people. The sales people also see explanations of why each person was chosen, what they should be offered, and recommended talking points.

Each of these steps takes significant technology. Lattice Engines currently monitors Web sites of five to 10 million U.S. businesses, checking daily for changes.  The system’s semantic engine reads structured texts such as management biographies and press releases, extracting entities and relationships but not trying to understand more subtle meanings such as sentiment. Clients specify blogs to follow, which receive similar treatment. The company also monitors Twitter, Facebook company pages, Quora, and LinkedIn profiles of people within each sales person’s network. Additional data comes from standard sources such as business directories and from special databases requested by clients. Information from all these sources is loaded into a single database available to all Lattice Engine clients.

Lattice Engines also imports data from the clients own systems, although of course this isn’t shared with anyone else. Again, there’s some clever technology needed to recognize individuals and companies across multiple sources. Lattice Engines doesn’t try to link personal and business identities for individuals.

All this information is placed in a timeline so that modeling systems can look at events before and after the target activities. The models themselves are built automatically, once users specify the target activity, product, and time horizon. Users can then build a list of customers or prospects, have the model score it, and send high-ranking names to marketing or sales for further contact. Results can be exported to a marketing automation system or appear within the sales person’s CRM interface. Lattice Engines is directly integrated with cloud-based CRM from Salesforce.com, Microsoft Dynamics, and Oracle, and via file transfer with SAP CRM. Users can export lists to Excel and Marketo, with connectors for Eloqua and other marketing automation systems on the way.

The net result of this is a single system that performs all the tasks needed to exploit the wide range of information available about customers and prospects.  Marketers could theoretically use separate systems for each step in the process, and integrate the results for themselves.  But few really have the skills to do this.  And, in most cases, it would be more expensive than purchasing a single system like Lattice Engines.  It's particularly helpful that Lattice Engines supports both prospecting and customer management -- further reducing the need for multiple products, and further encouraging cooperation between marketing and sales departments. 

Pricing for Lattice Engines starts at $75,000 per year and grows based on the number of data sources and sales users. Client data volume doesn't affect the cost, since Lattice Engines’ own databases are vastly larger than any client data. The company has close to 50 deployments, nearly all at large B2B marketers including Dell, HP, Microsoft, ADP, and Staples.

Read More

Marketing Automation Beer Goggles: What I Think I Learned at Dreamforce

I’m writing this on my way home from Dreamforce, the Salesforce.com user conference that has become the primary industry gathering for marketing automation vendors. With a reported 90,000 attendees (I didn't count them personally), the show is fragmented into many different experiences. My own experience was mostly talking to marketing technology vendors in the exhibit hall, private meetings, and maybe a party or two. I did attend the main keynote and the “marketing cloud” announcement, but neither contained  major product news and the basic story – that social networks change everything – was true but far from novel.

So what did I learn? On reflection, there were two themes that hadn’t expected when I arrived.

The first was data. I generally think of marketing systems as relying primarily on data from the company’s own marketing, sales, and operational systems. But the exhibit hall was filled with vendors offering information – mostly from Web crawling or social media – to supplement the company’s internal resources. Of course, this isn’t new but it seems that external sources are becoming increasingly important. The main reason is so much valuable public information is now available. A lesser factor may be that there’s less internal information, at least for sales and marketing, because so many prospects engage indirectly and anonymously until deep in the buying process.

But there’s more to data than the data itself. The theme includes easier connectivity to external data, via standard connectors in general and the Salesforce.com AppExchange in particular. A closely related trend is real-time, on-demand access to the external data: say, when a salesperson views a lead record or a lead is first added to marketing automation. This requires immediate matching to find the right person in the supplier’s database, and, sure enough, matching was another popular technology on the show floor. I also saw broader use of Hadoop to handle all this new data: as you probably know, Hadoop effectively handles large volumes of unstructured and semi-structured data, so it’s a key enabling technology for data expansion. A final component is continued growth in the reporting, analytics, and predictive modeling systems that make productive use of the newly-available data.

Some products combine all these attributes, others offer a few, and some just one. Obviously a single integrated solution is easiest for the buyer, but as Scott Brinker recently pointed out in an insightful blog post, platforms like Salesforce.com may actually make it practical for marketers to mix and match individual products without the technical pain traditionally associated with integration. It therefore makes sense to view the data-related systems as a cluster of capabilities that will develop as parts of single ecosystem, collectively raising the utility and importance of external data to marketers.

The second theme, considerably less grand, was lead scoring. I suppose this is really just a subset of the analytics component of the data theme, but I saw enough new lead scoring features from enough different vendors to treat it separately. In particular, predictive modeling vendor KXEN announced a free, cloud-based service to automatically score a new Salesforce.com lead’s likelihood of converting into a contact. (If you’re not familiar with Salesforce.com terminology: contacts are linked to an account, while leads are not. The conversion usually indicates the salesperson has deemed the person a valid prospect and is thus a critical stage in most sales processes.)

The KXEN service requires absolutely no set-up; users just install it from the AppExchange. KXEN then reads the data, builds a predictive model based on past results, and returns the scores on current leads. From a technical standpoint, the modeling is nothing new, and indeed the people I met at the KXEN booth seemed to feel the product was barely worth discussing. But I’ve long felt that an automated, predictive-model-based scoring service was a major business opportunity because it would replace the time-consuming, complicated, and surely suboptimal lead scoring models that most companies now build by hand, usually with little basis in real data. Of course, there are plenty of other predictive modeling systems available for marketers, but I’m excited because I don’t think anyone else has made model-based lead scoring as simple as the KXEN offering. Maybe I need to get out more.

Speaking of which, I met SetLogik at a loud party after several glasses of wine, so I may have been wearing the marketing technology equivalent of beer goggles. But if I understood correctly, it tackles the really hard part of revenue attribution by using advanced matching technologies to connect the right leads and contacts to sales (reflected in closed opportunities in Salesforce.com). Once you’ve done that, determining which marketing touches influenced those people is relatively easy.  It’s a unique solution to a huge industry problem. Come to think of it, correct linkages are also critical for building effective lead scoring models, which it turns out that SetLogik also does. (I'll admit it: I Googled them the next day.) So they're part of that theme as well.

As I mentioned earlier, data and lead scoring were themes that emerged for me during the conference. I did have some other themes in mind when I started, which are also worth sharing. I’ll do that another day.

Finally, it’s worth noting that the conference itself was tremendously well run. It sometimes felt that one-third of those 90,000 people were Salesforce.com employees hired to stand around and answer questions. Where they found so many cheerful people outside of the Midwest I’ll never know. Congratulations and thanks to the Salesforce.com team that made it happen.

Read More

Rapid Insight Provides Low-Cost Options for Desktop Data Transformation and Predictive Modeling

Summary: Rapid Insight offers low-cost desktop tools for data transformation and automated regression modeling. They're a good choice for companies that need something simple yet powerful.

Predictive modeling is widely used by consumer marketers to select names for mailing lists and to decide which products to offer existing customers. These models are typically built by statisticians with tools like SAS and SPSS. In other cases, marketers can build them for themselves with automated tools like KXEN that are tightly integrated with the marketing automation system.

But most marketers still don’t have a marketing automation system or even an integrated marketing database. Nor do they have the skills to use a product like SAS. This group needs stand-alone tools to do two key things: assemble data from multiple sources, and build and execute the models themselves. (Okay, three things.)

Rapid Insight offers exactly those two (or three) capabilities in a reasonably priced package.

Veera is the data assembly tool. It lets users connect to most standard data sources and then define a processing flow to filter, merge, aggregate, transform and otherwise manhandle data into a form that makes it useful. Veera also provides some basic analytics including descriptive statistics (mean, median, value frequencies, etc.), cross tabs and graphing. The flow is set up as a sequence of icons with a drag-and-drop interface, which means users don’t have to learn a scripting language. Rather than go into more details, I'll just point you to the vendor's on-demand demo.


The predictive modeling tool, prosaically named Analytics, builds logistic and least squares regression models. (Logistic models predict yes/no outcomes such as whether someone will respond to a promotion; least squares models predict continuous numeric outcomes such as lifetime value.) The Analytics interface is more sequential than Veera: users get a set of tabs that lead them through the steps of loading data, selecting variables, building the model itself, assessing the results, and scoring an audience. At each step along the way, users can make their own decisions or allow the system to choose for them.

I’ve seen quite a few automated modeling systems over the years, and was impressed at how well Analytics provides users with information to understand what's happening and take control when desired. This should let the system satisfy knowledgeable statisticians looking for a productivity enhancer, as well as novices who want to rely on the system's choices. Analytics also has a good online demo.


Veera and Analytics both run in client/server or desktop configurations. They load data into system memory (RAM), which means very large projects could be problematic. The vendor says a half-million rows with a couple hundred variables is a reasonable universe to model.

The two products are sold separately. This makes sense: many companies could use a generic data assembly tool like Veera for purposes other than modeling. For example, marketers might use it to construct a multi-source marketing database for promotions or analytics.

Pricing is $3,000 for the first Veera user and $5,000 for Analytics, with discounts for additional licenses. This is quite reasonable compared with other automated modeling systems, although other products often provide more than just regression models. Annual maintenance for each product is $1,750 per license. Rapid Insights has been selling its products since 2005 and has more than 150 clients with over 200 licenses.

Read More

For Behavior Detection, Simple Triggers May Do the Trick

I was in the middle of writing last week’s post, on marketing systems that react to customers’ Web behavior, when I got a phone call from a friend at a marketing services agency who excitedly described his firm’s success with exactly such programs. Mostly this confirmed my belief that these programs are increasingly important. But it also prompted me to rethink the role of predictive modeling in these projects.

To back up just a bit, behavioral targeting is a hot topic right now in the world of Web marketing. It usually refers to systems that use customer behavior to predict which offers a visitor will find most attractive. By displaying the right offer for each person, rather than showing the same thing to everyone, average response rates can be increased significantly.

This type of behavioral targeting relies heavily on automated models that find correlations between the a relatively small amount of data and subsequent choices. Vendors like Certona and [X+1]tell me they can usually make valuable distinctions among visitors after as few as a half-dozen clicks.

At the risk of stating the obvious, this works because the system is able to track the results of making different offers. But this simple condition is not always met. The type of behavior tracking I wrote about last week—seeing which pages a visitor selected, what information they downloaded, how long they spent in different areas of the site, how often they returned, and so on—often relates to large, considered purchases. The sales cycle for these extends over many interactions as the customer educates herself, gets others involved for their opinions and approvals, speaks with sales people, and moves slowly towards a decision. A single Web visit rarely results in an offer that is rejected or accepted on the spot. Without a set of outcomes—that is, a list of offers that were accepted or rejected—predictive modeling systems don’t have anything to predict.

If your goal is to find a way to do predictive modeling, there are a couple of ways around this. One is to tie together the string of interactions and link them with the customer’s ultimate purchase decision. This can be used to estimate the value of a lead in a lead scoring system. Another solution is to make intermediate offers during each interaction, of “products” such as white papers and sales person contacts. These could be made through display ads on the Web site or something more direct like an email or phone call. The result is to give the modeling system something to predict. You have to be careful, of course, to check the impact of these offers on the customer’s ultimate purchase behavior: a phone call or email might annoy people (not to mention reminding them that you are watching). Information such as comparisons with competitors may be popular but could lead them to delay their decision or even end up purchasing something else.

Of course, predictive modeling is not an end in itself, unless you happen to sell predictive modeling software. The business issue is how to make the best use of the information about detailed Web (and other) behaviors. This information can signal something important about a customer even if it doesn’t include response to an explicit offer.

As I wrote last week, one approach to exploiting this information is to let salespeople review it and decide how to react. This is expensive but make sense where a small number of customers to monitor have been identified in advance. Where manual review is not feasible, behavior detection software including SAS Interaction Management, Unica Affinium Detect, Fair Isaac OfferPoint, Harte-Hanks Allink Agent, Eventricity and ASA Customer Opportunity Advisor can scan huge volumes of information for significant patterns. They can then either react automatically or alert a sales person to take a closer look.

The behavior detection systems monitor complex patterns over multiple interactions. These are usually defined in advance through sophisticated manual and statistical analysis. But trigger events can also be as basic as an abandoned shopping cart or search for information on pricing. These can be identified intuitively, defined in simple rules and captured with standard technology. What’s important is not that sophisticated analytics can uncover subtle relationships, but that access to detailed data exposes behavior which was previously hidden. This is what my friend on the phone found so exciting—it was like finding gold nuggets lying on ground: all you had to do was look.

That said, even simple behavior-based triggers need some technical support. A good marketer can easily think of triggers to consider: in fact, a good marketer can easily think of many more triggers than it’s practical to exploit. So a testing process, and system to support the process, is needed to determine which triggers are actually worth deploying. This involves setting up the triggers, reacting when they fire, and measuring the short- and long-term results. The process can never be fully automated because the trigger definitions themselves will come from humans who perceive new opportunities. But it should be as automated as possible so the company can test new ideas as conditions change over time.

Fortunately, the technical requirements for this sort of testing and execution are largely the same as the requirements for other types of marketing execution. This means that any good customer management system should already meet them. (Another way to look at it: if your customer management system can’t support this, you probably need a new one anyway.)

So my point, for once, is not that some cool new technology can make you rich. It’s that you can do cool new things with your existing technology that can make you rich. All you have to do is look.

Read More

Bah, Humbug: Let's Not Forget the True Meaning of On-Demand

I was skeptical the other day about the significance of on-demand business intelligence. I still am. But I’ve also been thinking about the related notion of on-demand predictive modeling. True on-demand modeling – which to me means the client sends a pile of data and gets back a scored customer or prospect list – faces the same obstacle as on-demand BI: the need for careful data preparation. Any modeler will tell you that fully automated systems make errors that would be obvious to a knowledgeable human. Call it the Sorcerer’s Apprentice effect.

Indeed, if you Google “on demand predictive model”, you will find just a handful of vendors, including CopperKey, Genalytics and Angoss. None of these provides the generic “data in, scores out” service I have in mind. There are, however, some intriguing similarities among them. Both CopperKey and Genalytics match the input data against national consumer and business databases. Both Angoss and CopperKey offer scoring plug-ins to Salesforce.com. Both Genalytics and Angoss will also build custom models using human experts.

I’ll infer from this that the state of the art simply does not support unsupervised development of generic predictive models. Either you need human supervision, or you need standardized inputs (e.g., Salesforce.com data), or you must supplement the data with known variables (e.g. third-party databases).

Still, I wonder if there is an opportunity. I was playing around recently with a very simple, very robust scoring method a statistician showed me more than twenty years ago. (Sum of Z-scores on binary variables, if you care.) This did a reasonably good job of predicting product ownership in my test data. More to the point, the combined modeling-and-scoring process needed just a couple dozen lines of code in QlikView. It might have been a bit harder in other systems, given how powerful QlikView is. But it’s really quite simple regardless.

The only requirements are that the input contains a single record for each customer and that all variables are coded as 1 or 0. Within those constraints, any kind of inputs are usable and any kind of outcome can be predicted. The output is a score that ranks the records by their likelihood of meeting the target condition.

Now, I’m fully aware that better models can be produced with more data preparation and human experts. But there must be many situations where an approximate ranking would be useful if it could be produced in minutes with no prior investment for a couple hundred dollars. That's exactly what this approach makes possible: since the process is fully automated, the incremental cost is basically zero. Pricing would only need to cover overhead, marketing and customer support.

The closest analogy I can think of among existing products are on-demand customer data integration sites. These also take customer lists submitted over the Internet and automatically return enhanced versions – in their case, IDs that link duplicates, postal coding, and sometimes third-party demograhpics and other information. Come to think of it, similar services perform on-line credit checks. Those have proven to be viable businesses, so the fundamental idea is not crazy.

Whether on-demand generic scoring is also viable, I can’t really say. It’s not a business I am likely to pursue. But I think it illustrates that on-demand systems can provide value by letting customers do things with no prior setup. Many software-as-a-service vendors stress other advantages: lower cost of ownership, lack of capital investment, easy remote access, openness to external integration, and so on. These are all important in particular situations. But I’d also like to see vendors explore the niches where “no prior setup and no setup cost” offers the greatest value of all.

Read More