Ignore the Analyst clickbait and crack open the spreadsheet

September 10, 2020

It's the "Wave" or the "Quadrant" that draws your attention, but when choosing an ecommerce platform, it's critical to understand what's behind the analyst rankings to be sure they are truly relevant to your business.

This article by Matt Hampshire, Co-Founder of Loop, was originally published on LinkedIn


No sooner had I hit ‘publish’ on my thoughts about hybris and SAP Commerce, my friends at Commercetools triumphantly sent me the latest Forrester B2B Commerce Wave.  I’ve developed a healthy respect for Forrester over the years and had been confident that SAP Commerce would retain its leadership position. I was wrong.

Rather than just accept things on face value though, I did something that unfortunately not enough organizations will actually do… I took a deeper dive into the report to better understand how Forrester determines their rankings, and whether they make sense with what I’ve experienced working in B2B commerce.

Forrester’s scoring system is based upon vendor questionnaires, reference calls and scenario demonstrations.  It’s proprietary so I can’t comment on the underlying scores, and I certainly don’t envy Forrester having to cut through the non-differentiated hyperbole that most of the vendors pitch with.

However, the scores for each of their various criteria (like personalization, sales channel support and AI) are then subjected to a weighting which Forrester DOES provide us with, and it’s a combination of the score and this weighting that determines the Wave rankings.

Forrester states clearly that the Wave is only a guide, that you should apply your own weightings based on your organization’s specific needs, and they will provide you with a spreadsheet to do this.  The problem is, in my experience too many organizations take no notice of this and simply accept things on face value.  The vendors love sending out a free version of the report when it favours them, but if you don’t do the work to understand why Forrester ranks how it does, then how can you be sure their analysis is relevant to your own needs?  To paraphrase former Forrester analyst Brian Walker from a recent episode from his (excellent) podcast series, too many companies just look at the headline Wave graphic (the ‘clickbait’ as he calls it), circle the top 3 or top 5 vendors and only look at those… “but you need to crack open the spreadsheets and apply your own weightings”.

To highlight the importance of doing this, I personally don’t think that the weightings Forrester uses in the report will make sense for a lot of B2B businesses.  In fact, it turns out that the weightings they use for the B2B Wave are exactly the same as they use in their B2C Wave (sample reproduced below).

No alt text provided for this image

There’s certainly no question that B2B organizations need to offer B2C-type experiences for their customers.  Nor is there any doubt that there is a blurring of the lines between the two.  In my view though there are clear differences.

Fundamentally, B2B transactions are more complicated than B2C.  Pricing and payment terms can often vary by customer, customers pay on invoice and there are often rebates to take into account.   This can mean that real-time back-office pricing integration is often much more important for B2B.  B2B products are often complex, meaning configure, price and quote features are a priority.  For B2B, field sales and service teams are a critical enabler (or stumbling block) for commerce and the business case for B2B commerce often hinges on the support of the sales team.  B2C will value email marketing and personalization more highly along with features such as wishlists and contextual shopping, and will need to offer a broader array of payment options at checkout and third party website integrations.

Frankly I think Forrester could have explored the differences between B2B and B2C a little more in the report.  They seem to lean heavily on technology considerations, and I’m particularly confused as to why they call out A/B Testing as a distinct differentiator for two of the platforms.  Whilst I’m sure AB Testing is on the list of features that B2B will ultimately use, it’s going to be a fair way down the list of priorities.

How would I personally weigh the needs of B2C and B2B today? For a start, I’d group the criteria a little differently, and weigh them something like this:

No alt text provided for this image

At least, that’s where I’d begin.  At the end of the day though, it doesn’t really matter what I think.  Nor for that matter, how Forrester does their weighting.  What matters is what’s important for your business.  Maybe you need to prioritize marketplace features,  globalization capabilities, or store-integration?  Perhaps even AB Testing!

All of the vendors on the Wave, regardless of position, are there because they have significant market share, and are worthy of consideration.   So download the spreadsheet and take the time to work out what’s right for your business.

*Forrester applies the other 50% of the weighting to vendor strategy criteria