Acquisitions Reshape the Marketing Automation Industry: Growth at the Bottom, Room in the Middle, Fog at the Top

Raab Associates officially released the new edition of our B2B Marketing Automation Vendor Selection Tool (VEST) yesterday. This is our flagship report on the industry, with nearly 200 data points on 23 vendors and separate ratings for micro-business, small to mid-size companies, and enterprise marketing departments. There are quite a few vendor comparisons out there, but none come close to the level of detail in the VEST – and details are what you really need to select a system. I personally suggest that anyone interested in the industry buy a copy for themselves and another for someone they love. See www.raabguide.com/vest for details.

I genuinely enjoy catching up with the vendors while preparing the VEST, but must admit that my favorite part of the process is analyzing the data once it’s assembled. Sadly, the wave of acquisitions that swept the industry in the past year has made this harder: many major vendors are now part of a public company, which severely restricts the information they can share. We’ve probably passed a tipping point where so much information is hidden that I can’t draw a clear picture of industry growth rates or competitive positions.

The table below shows the data available and highlights the holes. I’ve grouped the vendors into three buckets based on the market sectors they serve: micro-business (under $5 million revenue), small to mid-size business ($5 to $500 million), and large enterprises (over $500 million).

You’ll immediately see that the “not reported” information is concentrated among companies serving mid-size and enterprise clients, which is where all the acquisitions to date have taken place. Neolane is an exception but only because they provided the VEST information just before Adobe acquired them in June. I doubt we’ll see new numbers from them in the future. Marketo was mostly missing until they provided key figures in their earnings call this afternoon. Thanks, guys.

I've summarize my thoughts on this data with three oh-so-catchy phrases: growth at the bottom, opportunity in the middle, and fog at the top.

Growth at the Bottom: the green shading in the client growth column highlights companies reporting a year-on-year increase of 60% or more. What jumps out is the concentration at the top of the chart, in the micro-business sector. Four of the five micro-business vendors grew more than 60% and the fifth (Venntive) grew at a far-from-shabby 54%. There’s too much missing data in the other sectors to say for certain that the micro-business vendors are growing the fastest, but it sure looks that way. My interpretation is that the micro-business sector is the least mature and still presents the greatest untapped opportunity – even if buyers are still limited to the small proportion of business owners who are “tech geeks”.

Room in the Middle: Marketo's client count increased just 36% from mid-2012 to mid-2013 (although they’re projecting 54% revenue growth for 2013 vs. 2012).  We can no longer see the growth rates for mid-market heavy weights Pardot and Eloqua, but I’d be surprised if they beat Marketo.  They're certainly not close to the 67% to 90% rates reported by LeadFormix, Act-On, and eTrigue. I suspect Pardot, Eloqua and Marketo will increasingly focus on selling to enterprises, and in Marketo’s case on expanding footprint within existing clients. If so, this might open the way to faster growth by the next tier of mid-market vendors, who are mostly still private.  (LeadFormix is the exception, but seems to be pretty much left alone by its corporate parent). The clear winner in this scenario is Act-On, which has ample venture funding and has indeed been growing very rapidly. They are already the first vendor since Pardot to break the 150-employee barrier (blue shading). Silverpop and HubSpot might also benefit but neither is fully focused on standard B2B marketing automation. Other vendors would need outside funding to squeeze through what will probably be a briefly open window.

Fog at the Top: My visibility into enterprise B2B marketing automation was always clouded because of cross-over by B2C vendors including IBM, SAS, Teradata, and Neolane. It is now completely obscured except for sporadic glimpses of details that vendors choose to reveal. But even if everyone shared all their data with me, the enterprise picture would remain foggy because enterprises are increasingly integrating marketing automation with advertising , sales, service, and Web management. This makes it increasingly meaningless to treat marketing automation as a distinct category. Of course, that integration is exactly why the enterprise vendors purchased all those marketing automation systems in the first place.

If integration really happens at the top then we'll end up with a bizarre symmetry, since the enterprise market will be mirroring the integrated sales / CRM / Web / ecommerce products already bought by micro-businesses.  This would leave stand-alone marketing automation as a niche product for mid-tier companies. It would be a very large niche, but squeezed between broader suites from above and below and, eventually, challenged from within by integrated suites built for mid-market companies. The obvious response from marketing automation vendors is to build those broad suites themselves or to create platforms that are the foundation of such suites. That’s exactly what the larger mid-tier companies are doing, but it’s an expensive proposition. Any small mid-market companies who want to play must grab whatever fleeting opportunity the market offers today for growth, before they are locked out for good.

Read More

Social and Mobile Features Head the List of New Marketing Automation Capabilities

I’m getting ready for the next edition of the B2B Marketing Automation Vendor Selection Tool (VEST). This is based on nearly 200 questions to vendors, mostly about product features. The first step in the process is to update the list of questions. This is based on a review of recent vendor announcements plus my own feeling for what’s important. What emerges is an interesting portrait of industry trends in product development.

You won’t be surprised to learn that most of the changes involve social and mobile marketing, today's two hottest areas in marketing in general. We’ll get back to those in a bit. But first, I’d argue the single most important result is just how few changes there really were. B2B marketing automation is far from mature in terms of market penetration, but the mix of product features is pretty well set. Most of vendor announcements I reviewed were about common features that particular vendors had been lacking or were enhancing.  Social and mobile are the exceptions, but both are still very small contributors to most B2B marketing programs. I saw much more activity around features that were new last year, such as dynamic content and integration with Webinar systems and with Microsoft Dynamics CRM.

So exactly what new social and mobile features are now on my list? The previous report already included basic social capabilities including sharing marketing content to social media, tracking responses generated from social media, and monitoring social media activity. The new VEST expands that list to include:

- track social media influence: individual-level tracking mechanism that can identify the number of times a recipient has shared a promotion to social media and the number of responses generated the shared promotions. This information is part of the contact profile of the individual.

- create social media posts: deliver messages through social media, such as Twitter posts and Facebook updates. These messages can be created and then scheduled for future delivery.

- create social forms: create forms that are delivered within a third-party social media system such as Facebook.

- create social promotions: create social promotions such as contests, polls, ratings, etc.

- social sign-on and data capture: recipients can register using third-party social credentials, such as their Facebook ID. This gives access to information stored within the third-party social media system and allows communication through that system.

- build social profile: capture information about a specified individual by searching public information across multiple social media systems. This information includes social media handles and social activity such as posts, comments, and questions answered. The information is added to the individual profile and activity history.

The broad range of these features represents both a maturation of B2B social marketing and uncertainty about what will ultimately prove useful. We can expect more social features in the near future, although I suspect some will later be abandoned when it turns out they’re not especially effective in a B2B context.

On to mobile.  My previous list of mobile features was limited to text messaging. I’ve expanded that to add:

- mobile formats: generate Web and email versions in formats tailored to delivery on mobile devices such as smartphones and tablets.

- mobile CRM: salespeople can access the system on mobile platforms such as smartphones and tablets.

- mobile reporting: users can access reports on mobile platforms such as smartphones and tablets.

- mobile administration: users can set up campaigns and create content on mobile platforms such as smartphones and tablets.

Only the first of these, mobile formats, is about delivering marketing messages. The others are all about marketers and salespeople accessing the system on their own mobile devices. That’s clearly the current focus on mobile marketing automation, although it’s safe to expect more mobile marketing in the future – such as location-based promotions, which are notably absent so far.

I also added three entries in other categories. These were:

- app marketplace: the vendor has a formal app marketplace that lets third party applications connect to its product without custom integration.

- real time recommendations: rules and/or predictive models can recommend the best treatment for a customer as an interaction takes place within system-managed content such as a Web page.

- real time interactions: rules and/or predictive models can recommend the best treatment for a customer as an interaction takes place within an external platform such as a call center or Web site. This requires features to collect information about the interaction from the external platform, to match this information against the system's own database of contacts profiles and history, to make recommendation using the available information, and to deliver the recommendation to the external platform. .

These features all expand the scope of B2B marketing automation, mostly be connecting it with other systems. In one sense that's the opposite of the previous new entries, which were about adding features to marketing automation itself.  But both approaches aim to place marketing automation at the center of a company’s customer management infrastructure. Since other products, including CRM and Web sites, are also reaching for that position, we’ll see how widely these features get adopted. My sense is they’ll be more successful at small companies, where the labor savings of a unified system are most important because technology resources are most constrained.

None of the features I’ve added are currently available in more than a handful of systems.  Some may not yet be present in any. Few marketers this year will choose a system primarily because these particular features are present.  But we'll find over time which are really important.

Read More

New Workbook: Estimating the Cost of Marketing Automation

We released two more vendor selection workbooks last week, both sponsored by Eloqua and available for free on the RaabGuide Web site. One is about estimating the cost of a marketing automation system and the other is about evaluating vendor services.

The cost workbook was a particular challenge because the subject is so complex. After much thought, I came up with four cost categories:

• direct system costs: the actual price paid for the marketing automation software itself. This is where most buyers focus their analysis, but it’s a tiny fraction of the value at play.  Background research for the workbook suggested that automation costs are from 1% to 5% of an average marketing budget. This means that the direct system cost is pretty much insignificant compared with the marketing budget it will help to manage. To put it another way: even a small improvement in the other 95% to 99% of marketing costs can easily pay for a marketing automation system.

• operations costs: other costs related to running the marketing automation system, such as staff time and costs of related systems. Most of these are marketing operations costs, and there may be others in sales and IT.  You’re already incurring many of them, so the analysis has to identify how much they’ll increase or decrease as a result of marketing automation. This is really hard since it takes a detailed understanding of your current processes and how they'll change.  But the stakes are high: operations costs are about 25% of a typical marketing budget.

• marketing program costs: the expenses for specific marketing programs, such as advertising, trade shows, email, etc. These are the other 75% of the typical marketing budget. Marketing automation can reduce these costs substantially, both through reduced waste and through shifting funds to more effective programs.

• revenue: many marketers shy away from building revenue gains into their marketing automation calculation, but revenue is ultimately the reason for their investment. From an analytical perspective, it’s important not to double-count revenue gains (assuming the same marketing budget) and cost savings (from a lower marketing budget).  It's also important to recognize that revenue isn’t 100% profit. The workbook describes how to do this.  To keep things in perspective: marketing costs are under 10% of revenue at most firms, meaning that direct marketing automation are under 0.5% of revenue.

Analyzing all four items in depth is a big job. The good news is you don’t usually need to assess them all at once. 

• building a business case for marketing automation needs only a rough estimate for direct system costs, since they’re so small compared with the revenue, marketing program costs, and operations.  

• comparing marketing automation systems lets you focus on the system costs and changes in marketing operations costs, since those may vary considerably from one product to another. The impact on program costs and revenues should be about the same unless you're considering systems with widely different capabilities.  But hopefully you identified your needs earlier in the process, so you'll only be comparing similar systems once you reach the final evaluation..

It's useful to understand these cost categories, but the real work is gathering the details for each component.  This is where the workbook comes in: it lists of specific items to consider, so you have a framework to help ensure your analysis is complete. This is important: to take a real-world example, one of my consulting clients recently received quotes from two marketing automation vendors, one of which included email delivery and one of which did not. Recognizing the difference made it easy to prepare a true apples-to-apples comparison, but we could have easily missed it until later the process if we had not used a formal framework.

Read More

Vendor Selection: Writing a Good Requirements Document

My last two posts (not counting this morning’s detour into Marketo-land) described common errors marketers make when selecting marketing automation systems. How did we come to this?

I see two reasons:

• Marketers are like everybody else. Remember all that yammering about how today’s buyers do their own research, don’t talk to sales until late in the process, and get their information from social media rather than experts? Today’s marketers buy that way too. So the carefully structured, professionally managed selection process is a thing of the past.

• Marketers are marketers.  This means they’re facing more change and a less clear future than other types of buyers, and they’re less experienced with purchasing technology. It’s no wonder they can’t define their requirements as well as someone buying a new accounting system.

But all is not lost. Marketers can do a better job of system selection if they try. Specifically, they can do two things: improve the selection process itself and look beyond features to assess the vendors. This post will focus on the selection process and the next will talk about judging vendors.

As I’ve already written more than once, the key to sound selection process is a good set of requirements. These should be packaged into a formal requirements document so you have them all in one place, easily organized and available to share with vendors. But don’t think you’re writing the document for vendors. Instead, imagine you’ll submit it to the Chief of the Prussian General Staff, who just might slice your ear off if you do a less than thorough job.

Here's what he'll be looking for:

• Background: a general description of your business, including the products, company size, and industry characteristics. This gives a vendor an idea of your key issues and what sort of solution would be appropriate. Remember: a solution that’s too sophisticated for your needs can be as ineffective as one that’s too simple.

• Marketing process: describe your current methods for customer acquisition, relationship development, and retention. Include a channel-by-channel breakdown of your major marketing programs, with the volumes, spending and results for each. Your goals are to define the scope of your required solution and to help prioritize different capabilities. 

• Existing systems: describe the current marketing systems, including the technology, how they’re used, and known problems. This provides additional context for judging the scope of change that’s desired and what’s needed to achieve it.

• Project objectives: only now are you ready to state your goals for this project. You’ve waited this long because the objectives only make sense in light of your current situation. The goals you state here should be as specific as possible, so you can later check that proposed solutions  address them.

• Data sources: describe the internal and external systems that will feed your marketing automation platform. A simple marketing automation deployment might integrate only with CRM. But more complex scenarios could include inputs from Web analytics, order processing, point of sale, accounting, and elsewhere. Present this information in a table with record counts and transaction volumes so it can be used to size and price your solution.

• Required functions: this translates your project objectives into specific system requirements. These include data preparation as well as marketing execution. They wouldn’t generally extend to non-functional requirements like vendor background and pricing, although you could include them here if you’re concerned you’ll forget about them otherwise. Even though these are functional requirements, don’t be too specific in how things should work: you want enough flexibility for each vendor to showcase the best way to use their system. This part of the document is where you're most likely to need outside help: it takes an expert to know what functions are implied by each project objective.

• Use case scenarios: here’s the place to get specific. Pick several key processes, such as specific marketing programs, and describe in full detail how you want them set up. This would include segmentation rules, content creation, processing logic, CRM integration, lead scoring, and any other tasks required to run the program. You’ll later ask the vendors to demonstrate how they would perform those tasks.. The key is to define real projects for your business, not vendor-chosen examples that showcase their strengths and bypass their weaknesses.

These same elements should appear in pretty much any requirements document. What will differ is the degree of detail: I’ve written some requirements documents that are three pages long and some that are thirty. The right scale depends on the complexity of your situation. But even a simple requirements document is well worth the trouble, both to clarify your own thinking and to communicate that thinking to potential vendors.

Read More

Marketers Do a Bad Job Selecting Marketing Automation Systems

I presented my Seven Deadly Sins of Marketing Automation Software Selection during last week’s Webinar with Neolane. (To replay the Webinar, click here.)  If you’re wondering how many companies actually commit those sins, the sad answer is: a lot. Here are some statistics.

• About half of buyers consider only one system, I’m told by various vendors. Some may have known exactly what they needed in advance, but most are just buying the first system that seems to do what they need. And it’s a safe bet they haven’t analyzed their requirements well enough to understand those needs correctly.

• 66% of buyers base their selection process on meetings within marketing. This isn’t bad in itself, but many don’t talk to anyone else. You do also have to wonder how other 34% make a decision if they’re NOT talking to anyone in marketing. (This and the following figures come from the CMO Council study “Driving Revenue Through Customer Relevance”, which I analyzed in detail last year).

• 42% of buyers rely on online research. Again, not a bad source in itself, but far from sufficient. The real problem is comparing this figure and the previous 66% to…

• 25% of buyers consult with in-house IT. Think about that: 75% of CMOs are making a major system investment WITHOUT consulting their IT group. This would be fine if most marketers were experts at technology acquisition. But they’re not. Software-as-a-Service  makes it possible for marketers to purchase and deploy a marketing automation system without help from IT, but that doesn’t make it a good idea.

• 19% of buyers do a formal needs assessment and Request for Proposal (RFP). Again, this means the other 81% are buying a system without a formal buying process. Maybe some are just skipping the RFP, which isn't always needed. But I know from my own experience that plenty of marketers don’t do a needs assessment either. That's a big problem: you can't make a sound choice without one. Remember: when you don't know where you're going, any road will take you there.

• 25% do a pilot deployment. A pilot isn’t essential if you’ve run a good selection process. But for the vast majority of marketers who haven't run a good process, a pilot is their last line of defense before buying the wrong system. That so few run one means the most are buying blindfolded and hoping for the best. Let’s just say that this is not a good idea. 

Read More

The Seven Deadly Sins of Marketing Automation System Selection

I’ll be giving a Webinar this Thursday on evaluating marketing automation software, sponsored by Neolane. Part of the content will be a list of Seven Deadly Sins of Marketing System Selection.  I thought that was worth a blog post of its own. So here goes.

1. Ignoring Users. Selection teams often don’t take the the time to understand how future users of the system do their jobs today. The justification may be that everything will change anyway, or that every marketing department has similar needs, or that the users themselves don’t know what they need. The cost of skipping this step is that you don’t learn about existing business processes and user skills. This means you don’t identify what processes need to be changed and what training your users will need.  The immediate result is you can’t factor those items into your vendor evaluation. Longer term, your deployment will take longer since you’ll have to stop to gather this information before you can proceed.

2. Lack of Purpose. It’s frightening how often I ask someone how they expect to use their new marketing automation system and am told they don’t know. Buyers who don’t set business objectives have no way to judge what the system should do or to measure its success after the fact. Ideally you’ll have specific, quantifiable goals in terms of numbers of qualified leads, costs, and revenue created. But even general goals like supporting Webinars or running nurture campaigns are enough to give useful direction. Remember the old saying: “When you don’t know where you’re going, any road will take you there.”

3. No Requirements. Even marketers who know what they want often don’t translate those desires in specific system requirements. This is probably the most common sin of all. Formal, written requirements provide a framework to prioritize your needs, explore them with vendors, and make a complete, consistent assessment of what you learn. Without written requirements as a reference, your project can easily descend into chaos: something that made for great medieval artwork, but in real life is no fun at all.

4. Talk Only to Leaders.  Buyers often limit their consideration to a handful of vendors who are anointed as industry leaders by analysts or simply gain the most attention in social media. The theory seems to be that the most popular products do the best job of meeting a broad spectrum of needs, and are thus most likely to suit the buyer.  It’s an argument that only makes sense to people who don’t know their actual requirements. Think of it this way: would you only consider three best-selling automobiles (Ford F-150 pickup, Chevy Silverado pickup, and Toyota Camry)? Of course not, because you have specific requirements that those products probably don’t meet. Chances are you also have a few marketing automation needs that less popular systems actually perform best. You won’t know unless you look.

5. Let the Vendor Drive. Marketers who don’t know what they want often rely on the vendors to tell them what’s important. At best, the salesperson takes the time to understand your business and demonstrates how her system can best meet your needs. But that’s not the same as defining the best solution. More likely, the salesperson will hand you a list of what her system does best and hope you evaluate everyone else against it. It’s true that some salespeople will walk away from a deal if it’s a poor fit, but now you’re relying on the kindness of strangers – and you remember how that worked out for Blanche DuBois. (Poorly.)

6. Focus on Functions. We all love our bells and whistles, and salespeople love to show them. But functionality isn’t the only thing you need to consider in a vendor.  In fact, given that most systems can meet your basic needs, functions may not be the most important differentiator. You also need to consider how well the vendor will train and support you, whether their underlying technology can meet your present and future needs (there’s those pesky requirements again!), their familiarity with your industry, and how likely they are to remain in business. It's harder to answer these questions than sit through a demo, but they’re critical to your project’s success.

7. Work Without Experts. This is the Original Sin from which all others flow. It takes expertise to define objectives, gather requirements, screen the vendors, and run a smooth process. Marketers, like B2B buyers everyewhere, are increasingly trying to do it all without help – and most of them don’t have the time or skills to succeed. If you’re among the have-nots, see whether your IT department or procurement team have the skills to help. If not, find an external expert who specializes in marketing automation systems (for example, Raab Associates). Chances are, their fee will be less than the value of the time you’d spend doing the work for yourself. More important, you’ll end up with a better decision sooner, greatly increasing the final return on your marketing automation investment.

Read More

B2B Marketing Automation Industry Size and Segments

As I mentioned yesterday, our new B2B Marketing Automation Vendor Selection Tool (VEST) asks vendors to estimate the number of clients in each of four size categories.

This provides an interesting overview of the industry. The segments are defined based on revenue. Installation counts are:

Looking at the raw percentages doesn’t make much sense since businesses in each group are quite different. There’s a strong case to be made that micro-businesses in particular have such different needs that their vendors are not really part of the same industry as the rest of B2B marketing automation. I’ve described those differences in this post and go into them in our Vendor Selection Workbook (different from the VEST, and free on the Raab Guide site.)

But if you do want to consider all these vendors as one industry, the minimum adjustment to make is to account for differences in price. The table below calculates revenues using reasonable assumptions about revenue per client in each segment:

Combined with the previous chart, this shows the micro-business segment represents 61% of clients but just 17% of industry revenues. At the other extreme, large business represents just 6% of clients but 28% of revenue. The small- and mid-size companies are the heart of the industry , with 55% of the revenue from 33% of the clients.

The $257.5 million revenue estimate is reasonable but it excludes revenues from B2B marketing automation vendors not in the VEST report and the B2B revenues of B2C marketing automation firms. So I’d estimate total industry revenue at $325* million for 2011. This represents a 50% growth over my estimate for 2010. That is consistent with the growth rate I reported yesterday.

The figures also shed light on the ever-popular question of penetration rates. The table below shows company counts by revenue range from business list compiler Manta. But not all of these are B2B marketers. Looking at the industry categories, I'd put the estimated market at half the total.


The 26.7% figure for the large company category is clearly too high, but that's easy to explain: big companies have lots of divisions, so many vendors have sold to a little piece of those firms. There’s certainly still plenty of opportunity left. It’s possible the 3% figure for mid-size firms reflects some of this effect as well.

Figures for the first three categories are more intriguing. They're much lower than the usual estimates that 5% to 10% of companies have marketing automation. Either the surveys behind those estimates are incorrect or my market definition is too broad.

It’s probably a bit of each: surveys tend to reach people who have above-average interest in the topic, and my 50% figure is based on categories that could potentially use marketing automation, not the categories that have deployed it so far. A count of the pioneer companies, basically tech and manufacturing industries, would reduce the estimated market to anything from one quarter to one tenth the numbers shown. This would translate to penetration rates of 10% to 30%, which is more in line with current estimates.

But I’d argue that the market is already growing beyond this core group, so the long-term potential is considerably larger. That’s great news – so long as vendors don’t get stuck in the current niche and so long as competitors from the CRM, email, Web software, Web advertising or other industries don’t swoop in and snatch it all away.

______________________________________________________________
*The original version of this post estimated $300 million. On consideration, I raised the estimate to $325 million because
- my revised estimate for 2010 was $225
- the 52% growth rate in the previous post was in number of clients, but growth is faster in the higher-priced segments, so the revenue growth would be higher
- average prices are probably rising a bit in the mid-sized segment and big segments, so revenue would rise faster than client counts
- the client counts were gathered in May and June, so they are not quite mid-year figures

I would have gone higher, but the large-company figures are probably overstated in my estimates because many of the 1,200 installations are small, departmental systems that wouldn't generate anything near $60,000 per year.

Read More

Ranking B2B Marketing Automation Vendors: Part 3

Summary: The first two posts in this series described my scoring for product fit. The third and final post describes scoring for vendor strength. And I'll give a little preview of the charts these scores produce...without product names attached.

Beyond assessing a vendor's current product, buyers also want to understand the current and future market position of the vendor itself. I had much less data to work with relating to vendor strength and there are many fewer conceptual issues. From a buyer’s perspective, the big questions about vendors are whether they’ll remain in business, whether they’ll continue to support and update the product, and whether they understand the needs of customers like me.

As with product fit, I used different weights for different types of buyers. As you'll see below, the bulk of the weight was assigned to concentration within each market. This reflects the fact that buyers really do want vendors who have experience with similar companies. Specific rationales are in the table. I converted the entries to the standard 0-2 scale and originally required the weights to add to 100. This changed when I added negative scoring to sharpen distinctions among vendor groups.


These weights produced a reasonable set of vendor group scores – small vendors scored best for small buyers, mixed and special vendors scored best for mid-size buyers, and big vendors scored best for big buyers. QED.


I should stress that all the score development I've described in these posts was done by looking at the vendor groups, not at individual vendors. (Well, maybe I peeked a little.) The acid test is when the individual vendors scores are plotted -- are different kinds of vendors pretty much where expected, without each category being so tightly clustered together that there's no meaningful differentiation?

The charts below show the results, without revealing specific vendor names. Instead, I've color-coded the points (each representing one vendor) using the same categories as before: green for small business vendors, black for mixed vendors, violet for specialists, and blue for big company vendors.






As you can see, the blue and green dots do dominate the upper right quadrants of their respective charts. The other colors are distributed in intriguing positions that will be very interesting indeed once names are attached. This should happen in early to mid January, once I finish packaging the data into a proper report. Stay tuned, and in the meantime have a Happy New Year.

Read More

Ranking B2B Marketing Automation Vendors: Part 2

Summary: Yesterday's post described the objectives of my product fit scores for B2B marketing automation vendors and how I set up the original weighting for individual elements. But the original set of scores seemed to favor more complex products, even for small business marketers. Here's how I addressed the problem.

Having decided that my weights needed adjusting, I wanted an independent assessment of which features were most appropriate for each type of buyer. I decided I could base this on the features each set of vendors provided. The only necessary assumption is that vendors offer the features that their target buyers need most. That seems like a reasonable premise -- or at least, more reliable than just applying my own opinions.

For this analysis, I first calculated the average score for each feature in each vendor group. Remember that I was working with a matrix of 150+ features for each vendor, each scored from 0 to 2 (0=not provided, 1=partly provided, 2=fully provided). A higher average means that more vendors provide the feature.

I then sorted the feature list based on average scores for the small business vendors. This put the least common small business features at the top and the most common at the bottom. I divided the list into six roughly-equal sized segments, representing feature groups that ranged from rare to very common. The final two segments both contained features shared by all small business vendors. One segment had features that were also shared by all big business vendors; the other had features that big business vendors didn't share. Finally, I calculated an average score for the big business vendors for each of the six groups.

What I found, not surprisingly, was that some features are more common in big-company systems, some are in all types of systems, and a few are concentrated among small-company systems. In each group, the intermediate vendors (mixed and special) had scores between the small and large vendor scores. This is additional confirmation that the groupings reflect a realistic ranking by buyer needs (or, at least, the vendors’ collective judgment of those needs).


The next step was to see whether my judgment matched the vendors’. Using the same feature groups, I calculated the aggregate weights I had already assigned to the those features for each buyer type. Sure enough, the big business features had the highest weights in the big business set, and the small business weights got relatively larger as you moved towards the small business features. The mid-size weights were somewhere in between, exactly where they should have been. Hooray for me!



Self-congratulation aside, we now have firmer ground for adjusting the weights to distinguish systems for different types of buyers. Remember, the small business scores in particular weren’t very different for the different vendor groups, and actually gave higher scores to big business vendors once you removed the adjustment for price. (As you may have guessed, most features in the “more small” group are price-related – proving, as if proof were necessary, that small businesses are very price sensitive.)

From here, the technical solution here is quite obvious: assign negative weights to big business features in the small business weight set. This recognizes that unnecessary features actually reduce the value of a system by making it harder to use. The caveat is that different users need different features. But that's why we have different weight sets in the first place.

(As an aside, it’s worth exploring why only assigning lower weights to the unnecessary features won’t suffice. Start with the fact that even a low weight increases rather than reduces a product score, so products with more features will always have a higher total. This is a fundamental problem with many feature-based scoring systems. In theory, assigning higher weights to other, more relevant factors might overcome this, but only if those features are more common among the simpler systems. In practice, most of the reassigned points will go to basic features which are present in all systems. This means the advanced systems get points for all the simple features plus the advanced features, while simple systems get points for the simple features only. So the advanced systems still win. That's just what happened with my original scores.)

Fortified with this evidence, I revisited my small business scoring and applied negative weights to items I felt were important only to large businesses. I applied similar but less severe adjustments to the mid-size weight set. The mid-size weights were in some ways a harder set of choices, since some big-company features do add value for mid-size firms. Although I worked without looking at the feature groups, the negative scores were indeed concentrated among the features in the large business groups:


I used the adjusted weights to create new product fit scores. These now show much more reasonable relationships across the vendor groups: that is, each vendor group has the highest scores for its primary buyer type and there’s a big difference between small and big business vendors. Hooray for me, again.


One caveat is that negative scores mean that weights in each set no longer add to 100%. This means that scores from different weight sets (i.e., reading down the chart) are no longer directly comparable. There are technical ways to solve this, but it's not worth the trouble for this particular project.

Tomorrow I'll describe the vendor fit scores. Mercifully, they are much simpler.

Read More