Some Straight Talk On The Bersin Study
Some Straight Talk On The Bersin Study
We like Josh Bersin of Bersin & Associates: for one thing he's ex-Sybase, always a plus in our book. Sybase 'grads' tend to be among the brightest of the bright in Silicon Valley. He also calls things as he sees them, which is about the highest recommendation anyone can make for an industry analyst.
His LMS Customer Satisfaction 2005 survey is the first serious study to talk to customers about how they feel and to try to measure their responses in a statistically honest fashion.
Now, brandon-hall.com's LMS KnowledgeBase 2005 is also a useful study but completely different. It does side-by-side feature set comparisons of 50+ LMS products and does them well. But what the brandon-hall.com report doesn't tell you is what the LMS customers actually like and don't like about what they end up buying.
The two together make a great book-end combination, giving buyers a tool set for decision making. We would recommend these two studies to any group looking to buy an LMS.
There are lots of other paid opinions out there, some of which are, quite honestly, laughable. I've done interviews with researchers from major IT research organizations and have had to start off by giving them basic training on things as simple as the names of the offering categories in the learning technology field.
I'm not joking. Participating as a vendor in some of the big-firm research exercises is an eye-opener. Often you get junior, non-specialist staff researchers who get a call list somebody somewhere generated and an interview sheet somebody else came up with somehow and then they start working the phones.
The guys on the phones may have no experience in or understanding of the fields they're writing up. They may not know what's important to ask or how to follow up beyond the questions in their scripts.
They're also under tight deadline pressure because by the afternoon they'll be on a new assignment about something entirely different -- multi-player game companies or Taiwan chip houses that specialize in talking toys or something.
Basically, they're kids flying blind. The research reports they help create are in many cases useful mostly as training exercises for them, that is, the kids making the phone calls.
On the other end of the spectrum you get people like Jim Lundy at Gartner Group. He's not a kid. He's been around the block more than once.
Even so, his magic quadrant rankings can be, in my view anyway, as hard to "get" as a Picasso painting that's all angles (full disclosure: we've suggested Jim review us and he has so far declined the invitation).
I think it's fair to say that which companies get reviewed and how they get placed in the Gartner report is mostly a product of Jim's own judgment. And that's fine. He's a professional with some strong points of view. That's what Gartner pays him for.
As a research consumer you just need to keep in mind that what you're getting when you buy the Gartner report is a look at Jim's understanding of the market -- you're not buying a write-up of a statistically valid sample of what the market itself is saying.
The Bersin study is different. It is a statistically valid study of some 660 companies and it's all about what they have to say. It's not about Josh. That's why we like it.
Everyone's a winner: the first day they could, about a third of the LMS vendors in the Bersin study (there were 16, two merged) put out press releases claiming to be the best one way or another. Surprisingly, all of them were telling the truth.
They could all justifiably claim 'best' status for two reasons.
The first is that anyone can claim to be the best at anything. It's called puffery. Words like best and leading are meaningless. Only claims tied to metrics that can be fact-checked mean anything.
You can go home tonight and tell your kids that you really are the best parent in the universe and they cannot sue you for it, even if you're a mean old ogre (calm down -- we know you're not).
The second is that the metrics in the study were sliced and diced in very sophisticated ways, yielding multiple views of the survey outcomes.
Give anything complicated to a good PR person and you will get useful spin. They don't need much to work with.
For example, in the 20+ main survey question categories, the report lists market outperformers, basically the top half in each category, those who beat the average.
What's interesting is that the report lists the market outperformers by vendor category, not by across-the-board ranking, which means that many of the vendors can truthfully claim to be outperformers in one thing or another.
Vendor X and vendor Y can both claim to be categorically the best -- cool huh?
Who's in the study and why are all the vendors in those little boxes? Having lauded Josh above, I'm going to take him to task now. He's a Californian with a somewhat America centered view of the world.
Some of the more important LMS vendors in the world aren't American and weren't surveyed: Samsung, Hitachi, IMC and Acer for example, all of which probably have customer lists orders of magnitude bigger than some of the vendors who were included in the survey.
To put some words in his mouth, I think Josh would argue that America is the world's single largest country software market (which it is) and that most of the survey readers are more interested in products on the U.S. market than in, for example, Japan (the world's second largest single country software market) or Europe (arguably the largest software market of all but unbelievably complicated, segmented and hard to address in a single study).
I would add (and these words are out of my mouth) that this is the first year of the study and that despite my carping, the study is very, very good. Next year's should be even better.
Now to carp: the survey puts all the vendors in one of four groups -- global enterprise, enterprise, mid-market and emerging.
NetDimensions is in the emerging group, which in Josh's words comprises:
" . . . fast-growing companies who compete . . . across all three market segments (Global Enterprise, Enterprise, and Mid-Market) and these categories are not intended to position their solutions as suitable or unsuitable. In the report it is explained in detail - any buyer who purchases the report and would like to discuss with us directly we are happy to explain further."
Fair enough. Though I would argue the report puts us in the wrong category. There are only two global enterprise LMS providers in the world today -- NetDimensions and Saba.
We are the only companies I know of who specialize in truly global clients running 30+ country services with multiple single-byte and multi-byte languages fully integrated on one system.
I could be wrong about this but I don't think so.
Josh disagrees.
But the bottom line is that it is a Bersin & Associates study, not a NetDimensions study, so we are accepting his judgment.
For the record, we really are the best: there, I said it. NetDimensions is a market outperformer across the board.
More importantly, our clients love us. From the study:
“NetDimensions distinguished itself with the highest LMS customer loyalty of any vendor (none of its customers are even interested in switching) and very high ratings in ease of implementation. We expect them to grow rapidly in the U.S. market in 2005 and 2006.”
In the study the emerging market category vendors generally have much higher customer satisfaction scores than do vendors in the other three categories.
This is due to a number of factors, including greater organizational focus on individual client accounts and perhaps most importantly, more up to date technology (all NetDimensions clients are on the current version of the LMS -- we do not leave clients stuck on old versions).
But since the study doesn't give any across-the-board actual number comparisons, we could not claim, for example, that NetDimensions beat every global enterprise, enterprise and mid-market vendor in every single customer satisfaction category, even if it were completely true (to be clear, from the published study you cannot know with certainty any vendor's actual scores).
What we can claim with absolute assurance is that none of the other LMS vendors, even the other vendors in the emerging category, have customers as loyal as ours.
We have 100 percent customer loyalty. None of our customers would consider switching under any circumstance.
That suggests to us that our clients are extraordinarily happy about what we do for them.
There are reasons.
NetDimensions has a very simple approach to product development -- it is 100 percent client centered all of the time.
We have an equally simple approach to client service -- we treat our clients like gold.
Jay Shaw
NetDimensions CEO