B2B Marketing Automation Report Is Ready...My Web Site, Not So Much

The good news is, my new B2B Marketing Automation report (more formally: the Vendor Selection Tool, or VEST) is now available. The bad news is I can't actually sell it online, despite the best efforts of Web masters on two continents. But the good news is I'm more than happy to take credit card orders directly if you send me an email or give me a call. Email is info@raabguide.com.

To recap a bit, the new report is based on a survey of 18 vendors, who answered nearly 200 questions about their products and companies. Most answers were scored from 0, 1 or 2, indicating whether a particular feature was available fully, partly, or not at all. I translated other answers such as starting price or number of employees into similar 0-2 ranges so I could combine everything in a scoring formula. See my posts over the past few weeks for details on that.

The final result was three sets of scores for each vendor. The sets represent fitness for small, mid-size and large businesses, and each set contains a product fit score and vendor fit score. The idea was to simulate the type of scoring that a typical business in each category might do in its own vendor evaluation. Of course, no one's business is truly typical, so the interactive version of the tool also lets you create your own custom scoring weights.

The core of the new report, therefore, contains two sections: scatter diagrams plotting all the vendors in a typical "industry matrix" style and individual vendor profiles.

The industry matrix puts leads at the top right, where God and Gartner evidently intended them to be, and cleverly named other groups everywhere else. The clever part is giving names that are descriptive without being insulting. I settled on:

- "alternatives" (strong product fit but weak vendor fit)
- "anomalies" (weak product but strong vendor fit)
- "long shots" (weak product fit and weak vendor fit)

The vendor profiles give more detail about each vendor, including showing the scores for components within the product fit (7 components) and vendor fit (2 components). This gives some good insight into where the rankings came from.

So far so good. As I hinted before, there's both an interactive version and non-interactive version of the report. This is partly because I don't think everyone will want to pay for the full price for the interactive version and partly because some people have had problems running the interactive version, which uses Adobe Flash within a PDF. The non-interactive version, which I'm tactfully referring to as "basic", has an introductory section with industry explanations, recommendations on a selection process, etc., plus the three industry matrix charts (for small, mid-size and large) and individual profiles on each vendor. The profiles offer some narrative and scores for the components within the larger scores: 7 components within the profit fit (lead generation, campaign management, scoring, etc.) and two within the vendor fit (company strength and sector expertise). These give some insight into where the sales came from. This is priced at $295.

The interactive version has all those elements, which are made interactive by the fact that users can change the weights assigned to the different components within the profit and vendor fit scores. You've seen some of this is the same PDFs I posted over the past few weeks. It's great fun: there are little sliders for the weights and the vendors zoom around on the chart as you move them. A wonderful feeling of power.

The interactive edition also contains three more sections:

- Item Detail, which lets you see the 200-ish individual items used in the scoring, including their definitions and the weights assigned in each of the three scoring schemes.

- Custom Weights, which lets you set your own scoring weights for the individual items. You can start with the existing small, mid-size, or large weights as a base.

- Compare (my personal favorite), which lets you pick any three vendors and see how their scores compare in any of the weighting sets (small, mid-size, large, or custom). You can see bar charts with overviews and then drill into the item-by-item details for each category. This is where you see the specific differences between vendors.

Price for the interactive edition is $795.

I'll be presenting some additional analysis based on what's in the reports over the next few weeks, and of course will make a formal announcement once the e-commerce bugs are worked out. Again, though, you're welcome to send me a note to get your copy at once.

Read More

B2B Marketing Automation Vendor Comparisons: New Report Next Week and The Coolest Sample Yet

I suspect you may be getting tired of reading about the features in my new report comparing B2B marketing automation vendors, and want some actual information. Soon, I promise: the final data is all ready and only some light editing stands between you and a completed report. Well, that and the fact that the e-commerce features of the www.raabguide.com Website need some work. Either way, the report will come out next week -- even if I have to take credit card orders by phone.

But I finished the final interactive component of the report yesterday and I think it's exciting enough to be worth sharing. It lets you do what I think most buyers really want, which is to compare selected vendors side by side on their specific features. You can also compare their scores, which, since you can change the weights applied to different inputs, means you can get your very own, custom comparative ranking. If that's not fun, what is?

But there's more: you get to see the results in colorful graphs. Here's a screenshot:


You can also download an interactive sample. (This is scrambled data and vendor names are replaced by sports teams. Beware that the document uses Adobe Flash; Mac users in particular may need to use Adobe Reader rather than their usual viewer. And, alas, it won't work on your iPad.)

As you can see, the screen lets you pick up three vendors, a weight set (small, mid-size or large), and the type of data to view: a summary or the individual items within each category. For each item, you see the actual input values (2, 1 or 0 depending on whether the vendor complies fully, partly, or not at all) and the scores calculated once the weights are applied. You can change the category weights (by adjusting the figures in the little gray boxes at the right) and watch the scores themselves change as the individual weights are adjusted proportionately. The graphs also adjust immediately as you make changes.

My purpose in all this is to help buyers look beneath the scores themselves to understand where the scores came from. This lets them judge whether they really care about the factors that are driving the relative rankings. Similarly, making it easy to change the weights raises the question of which weights really are appropriate. Thinking about this should lead buyers to a better decision.

The screenshot above illustrates the importance of the weights. Look at Technology: there are pretty big differences between the different "vendors", but the category as a whole has such a low weight that these make little difference in the final rankings. This reflects a judgment on my part that small business buyers don't really care much about technology and that their technology needs are pretty simple.

If you squint hard enough, you'll also notice that the middle vendor has the highest total input value for Technology, but the lowest weighted score. That's pretty common because the weights do vary substantially from one item to the next. In this instance, the main reason is that small business scores apply negative weights to many advanced features, on the theory that they detract from value by adding complexity. You'll recall that I wrote about that in an earlier post.

The downloadable sample only has descriptions under all the other tabs, but everything else is actually ready. I'll make a formal announcement next week about price and availability of the new report.

Read More

B2B Marketing Automation Vendor Comparison -- Here's a Sample

I’ve been having way too much fun working on my new industry report. I decided to make it an interactive document that lets users (viewers? readers? The Chosen?) set their own weights for the different scoring categories and do detailed, side-by-side comparisons of vendors they select. This gives the document vastly more play value than a simple report. Much more important, it reinforces the point that I keep stressing, which is that every evaluation must be based on the buyer’s unique needs. Having three different sets of scores was a step in that direction, and making things interactive goes still further.

Click here to download a sample of the format. (Beware that it's a big file and can take several minutes to load.) Vendor names have been replaced with football teams and the specific details are excluded. But you can see the results of the different scoring schemes and also get a list of the specific items with their weights and definitions. The sample also contains draft versions of the introductory materials, which need some reformatting. (Note to Mac users: this is an Adobe Flash document; you'll have to use Adobe Reader to view it.)

In case it’s not obvious, you can move the little sliders on the “Sector Chart” tab to see how the different vendors move around depending on how you weight different categories of attributes. The final report will show directly how much each category contributes to each vendor’s score. You can also adjust the category weights on the grid within the “Scoring Weights” tab, which shows the detailed items. You have to mouse over the numbers in the grids – not the most convenient method, but the one allowed by the software I’m using (SAP Business Objects’ Xcelsius).

I actually did work up a version of the report that lets users set their own weights for the individual items. Unfortunately, that seems to overtax the software, so I’ll have to leave that out of the final product. I might put it out as a separate product or upgrade to the base report.

Please take a look and let me know what you think. The report itself should be ready for distribution within a few days, and of course I'll announce it here first.

Read More

Another Estimate of B2B Marketing Automation Revenue

Summary: Here's a closer look at revenue per employee and marketing automation revenue in general. I get the same answers as before but now have more detail to back it up.

Some of the comments on last week’s post on the size of the B2B marketing automation industry led me to dig a bit more deeply into the question of revenue per employee. Looking through my files and asking a few questions, here are vendors for whom I have reasonably reliable information:


This gives an average of $171,000 per employee. Given that these are fast-growing companies and the employee counts were based on figures for September or later, the average headcount through the course of the year was lower, meaning the revenue per full-time-equivalent employee would be higher – probably not so far from my $200,000 figure. Indeed, the figure for the three slowest-growing companies (Unica, Aprimo and Alterian) comes to $194,000. That’s pretty darn close to my $200,000 standard. Cool.

These figures also shine more light on the original question of industry size. I don’t know the B2B fraction of Unica, Alterian or Neolane’s revenues, but it’s probably quite low: let's guess 15%. Aprimo has stated they are 40% B2B, and the rest of those vendors are 100% B2B. Doing that math, you get $160 million total:


But what about everyone else? The other big players in enterprise marketing automation are SAS, Teradata and SmartFocus, but they are almost entirely B2C so far I know. So maybe let’s credit them with $10 million.

This leaves all the other B2B marketing automation vendors. The survey for my up-coming report has employee counts, client counts and minimum prices for quite a few: OfficeAutoPilot, True Influence, Pardot, LoopFuse, Net Results, Manticore, Silverpop, Genius, LeadFormix,TreeHouse Interactive, SalesFUSION, and Marketbright. I can use that to prepare two estimates: one based on number of employees x revenue per employee, and another based on number of clients x minimum revenue per client.

- total employees comes to about 470 (I have to make guesses for a couple of small vendors and reduce the Silverpop total to account for its large B2C business). Since these are also fast-growing firms, let’s use a figure of $120,000 per employee, which happens to be the average for Neolane, HubSpot, Marketo and Infusionsoft. That yields $56 million.

- clients x minimum price is calculated separately for each vendor, of course. You’ll have to trust me that the total comes to $37 million. But that’s a very crude figure: it’s certainly low in the sense that many average revenue per client is higher than the minimum price. On the other hand, we have the growth effect again – those client counts were towards the end of the year, so companies weren’t getting a full revenue year from everyone. For sake of argument, let’s assume the two factors cancel each other out.

So we have one estimate of $56 million and another of $37 million. The good news is that they’re in the same ballpark. Let’s split the difference and figure $45 million in revenue for this group.

Finally, there are a number of other B2B marketing automation vendors who weren’t covered in my survey. These include ActiveConversion, Act-On Software, Genoo, LeadLife, eTrigue, Marqui, and others. I do have client counts and pricing for most of them; some rough calculations yield a figure of $10 million.

Add these up, and you get a total B2B marketing automation revenues for 2010 of $225 million:


Maybe I’ll adjust my original $200 million estimate and maybe I won’t bother. Either way, I do feel more confident that it’s close to right.

Read More

How Big Is the B2B Marketing Automation Industry?

Summary: Here are my estimates for the size of the B2B marketing automation industry, broken down by customer segments. Enjoy.

I've been working madly on my new report on B2B marketing automation vendors. One of the things this has forced me to do is come up with an estimate of industry size that I'm willing to defend in print. I based my figures on several approaches: revenues for the few vendors who release their figures; the number of vendor-reported clients multiplied by an estimated revenue per client; and the number of vendor-reported employees multiplied by industry-average revenue per employee.

These methods all yield similar figures -- around a $200 million in revenue for 2010. Bear in mind that the industry nearly doubled last year, so the current run rate is much higher. Also remember that I've excluded:

- the big enterprise marketing automation vendors (Unica, SAS, Teradata), who sell primarily to B2C marketers;

- B2C portions of Aprimo and Neolane; and,

- vendors who work mostly through marketing service providers (Alterian and SmartFocus).

Including those vendors would at least double the total figure. Services are also excluded.

That said, here's an excerpt from the report:

Revenues for B2B marketing automation systems (excluding related services) were $200 million in 2010, according to Raab Associates estimates. The industry can be divided into three segments serving different types of clients:

• Small business (under $20 million revenue). These are unsophisticated marketing departments whose primary interests are outbound email, landing pages, and simple lead nurturing through email autoresponders. Many are very small companies with just one or two marketing automation users. They often do not integrate with a separate sales automation system, either not using one at all or relying on a CRM option offered by the marketing automation vendor itself. The fastest growing industry segment, this group tripled to 12,000 clients and $60 million revenue in 2010. Many small business marketing departments use only email systems (which also provide landing pages and simple nurture campaigns) instead of marketing automation.

• Mid-size business ($20 million to $500 million revenue). This segment covers a broad range of marketing users with widely varied needs. Most require the full range of marketing automation functions, but apply them in relatively simple ways. They have three to fifteen marketing automation users. This segment is the heart of the marketing automation industry, supporting the largest number of competitors and accounting for approximately $100 million in 2010 revenue across 3,000 clients.

• Big business ($500 million revenue and higher). These are large marketing departments that may manage hundreds of campaigns for multiple products in different locations. They need special features for automated content selection, project management, complex lead scores, and tight limits on the rights granted to individual users. This group had about 500 clients generating $40 million revenue in 2010. Although it has been growing less quickly than other segments, adoption will accelerate as the value of B2B marketing automation is more widely recognized, existing B2B systems add more large-company features, and big software vendors enter the field.

Read More

Ranking B2B Marketing Automation Vendors: Part 3

Summary: The first two posts in this series described my scoring for product fit. The third and final post describes scoring for vendor strength. And I'll give a little preview of the charts these scores produce...without product names attached.

Beyond assessing a vendor's current product, buyers also want to understand the current and future market position of the vendor itself. I had much less data to work with relating to vendor strength and there are many fewer conceptual issues. From a buyer’s perspective, the big questions about vendors are whether they’ll remain in business, whether they’ll continue to support and update the product, and whether they understand the needs of customers like me.

As with product fit, I used different weights for different types of buyers. As you'll see below, the bulk of the weight was assigned to concentration within each market. This reflects the fact that buyers really do want vendors who have experience with similar companies. Specific rationales are in the table. I converted the entries to the standard 0-2 scale and originally required the weights to add to 100. This changed when I added negative scoring to sharpen distinctions among vendor groups.


These weights produced a reasonable set of vendor group scores – small vendors scored best for small buyers, mixed and special vendors scored best for mid-size buyers, and big vendors scored best for big buyers. QED.


I should stress that all the score development I've described in these posts was done by looking at the vendor groups, not at individual vendors. (Well, maybe I peeked a little.) The acid test is when the individual vendors scores are plotted -- are different kinds of vendors pretty much where expected, without each category being so tightly clustered together that there's no meaningful differentiation?

The charts below show the results, without revealing specific vendor names. Instead, I've color-coded the points (each representing one vendor) using the same categories as before: green for small business vendors, black for mixed vendors, violet for specialists, and blue for big company vendors.






As you can see, the blue and green dots do dominate the upper right quadrants of their respective charts. The other colors are distributed in intriguing positions that will be very interesting indeed once names are attached. This should happen in early to mid January, once I finish packaging the data into a proper report. Stay tuned, and in the meantime have a Happy New Year.

Read More

Ranking B2B Marketing Automation Vendors: Part 2

Summary: Yesterday's post described the objectives of my product fit scores for B2B marketing automation vendors and how I set up the original weighting for individual elements. But the original set of scores seemed to favor more complex products, even for small business marketers. Here's how I addressed the problem.

Having decided that my weights needed adjusting, I wanted an independent assessment of which features were most appropriate for each type of buyer. I decided I could base this on the features each set of vendors provided. The only necessary assumption is that vendors offer the features that their target buyers need most. That seems like a reasonable premise -- or at least, more reliable than just applying my own opinions.

For this analysis, I first calculated the average score for each feature in each vendor group. Remember that I was working with a matrix of 150+ features for each vendor, each scored from 0 to 2 (0=not provided, 1=partly provided, 2=fully provided). A higher average means that more vendors provide the feature.

I then sorted the feature list based on average scores for the small business vendors. This put the least common small business features at the top and the most common at the bottom. I divided the list into six roughly-equal sized segments, representing feature groups that ranged from rare to very common. The final two segments both contained features shared by all small business vendors. One segment had features that were also shared by all big business vendors; the other had features that big business vendors didn't share. Finally, I calculated an average score for the big business vendors for each of the six groups.

What I found, not surprisingly, was that some features are more common in big-company systems, some are in all types of systems, and a few are concentrated among small-company systems. In each group, the intermediate vendors (mixed and special) had scores between the small and large vendor scores. This is additional confirmation that the groupings reflect a realistic ranking by buyer needs (or, at least, the vendors’ collective judgment of those needs).


The next step was to see whether my judgment matched the vendors’. Using the same feature groups, I calculated the aggregate weights I had already assigned to the those features for each buyer type. Sure enough, the big business features had the highest weights in the big business set, and the small business weights got relatively larger as you moved towards the small business features. The mid-size weights were somewhere in between, exactly where they should have been. Hooray for me!



Self-congratulation aside, we now have firmer ground for adjusting the weights to distinguish systems for different types of buyers. Remember, the small business scores in particular weren’t very different for the different vendor groups, and actually gave higher scores to big business vendors once you removed the adjustment for price. (As you may have guessed, most features in the “more small” group are price-related – proving, as if proof were necessary, that small businesses are very price sensitive.)

From here, the technical solution here is quite obvious: assign negative weights to big business features in the small business weight set. This recognizes that unnecessary features actually reduce the value of a system by making it harder to use. The caveat is that different users need different features. But that's why we have different weight sets in the first place.

(As an aside, it’s worth exploring why only assigning lower weights to the unnecessary features won’t suffice. Start with the fact that even a low weight increases rather than reduces a product score, so products with more features will always have a higher total. This is a fundamental problem with many feature-based scoring systems. In theory, assigning higher weights to other, more relevant factors might overcome this, but only if those features are more common among the simpler systems. In practice, most of the reassigned points will go to basic features which are present in all systems. This means the advanced systems get points for all the simple features plus the advanced features, while simple systems get points for the simple features only. So the advanced systems still win. That's just what happened with my original scores.)

Fortified with this evidence, I revisited my small business scoring and applied negative weights to items I felt were important only to large businesses. I applied similar but less severe adjustments to the mid-size weight set. The mid-size weights were in some ways a harder set of choices, since some big-company features do add value for mid-size firms. Although I worked without looking at the feature groups, the negative scores were indeed concentrated among the features in the large business groups:


I used the adjusted weights to create new product fit scores. These now show much more reasonable relationships across the vendor groups: that is, each vendor group has the highest scores for its primary buyer type and there’s a big difference between small and big business vendors. Hooray for me, again.


One caveat is that negative scores mean that weights in each set no longer add to 100%. This means that scores from different weight sets (i.e., reading down the chart) are no longer directly comparable. There are technical ways to solve this, but it's not worth the trouble for this particular project.

Tomorrow I'll describe the vendor fit scores. Mercifully, they are much simpler.

Read More