Gartner 'playing catch up' as peer review upstarts disrupt status quo

Challenger firm G2 Crowd relies on peer reviews and complete transparency and claims to be racking up customers

IT analysts are in the privileged position of being able to influence the entire industry.

Market data from giants such as IDC and Gartner is dished out in boardrooms and presentations across the globe, and often the numbers can influence key strategic decisions. Anyone who has sat through an analyst presentation will know full well that these firms are among the first to tell the industry it needs to change, or to "disrupt, or be disrupted", to quote the commonly used phrase. And it seems that the analysts themselves are not immune from this disruption, with upstart rivals biting at their heels.

One such newcomer is G2 Crowd, an Illinois-based analyst firm which uses real-time peer reviews to inform its "grid", which ranks software vendors on a quadrant-style grid, similar to that of Gartner's Magic Quadrant. The firm was founded four years ago by former executives at software firm Big Machines, which was later sold to Oracle.

Since then, the company has grown significantly and now has more than 92,000 reviews read by more than 600,000 software buyers each month. G2 Crowd claims its analysis is more up to date than that of traditional analyst rivals, and is based on a large number of user reviews, rather than the view of one analyst, who may not have used the kit.

G2 Crowd's CEO Tim Handorf told CRN that by using peer reviews, the firm's findings are extremely transparent, which customers love. He said he and the other founding members of G2 Crowd realised this during their time at Big Machines.

"Gartner is playing catch-up. Traditional analysts need to up their game and take note. The buyer dynamic has changed."

"We were inspired by our experiences there to start G2 Crowd," he said. "One of those was we had an annual user conference every single year. The unique thing about this conference was it was very collaborative - we would ask for feedback and for [customers] to vote on what they want us to do. They'd give us a great lot of feedback on how we can improve.

"One year though, we decided to invite the prospects in. The first reaction was fear because... [at the conference] we were listening to everything that was wrong, or that we could do better. We invited them in anyway and we found out that those deals ended up closing. The probability of closing was very, very high when we invited [prospects] to the conference.

"We found that when you're very transparent with your prospects and you introduce them to your customers, good things happen for both. You give them what they want - transparency - and that ends up closing deals for you. But the problem was we could only have that conference once a year and it cost a lot of money to do it. So there wasn't a good way to get prospects and customers to interact. That was one of the inspirations we had to start G2 Crowd."

Peer reviews have become extremely valuable online in the consumer space, with many retailers offering dedicated review sections on products' web pages, and completely new companies such as TripAdvisor and CheckATrade being born out of the trend.

According to eConsultancy, 61 per cent of customers read online reviews before making a purchasing decision. And this is echoed in the B2B space, according to SiriusDecisions, which claims 67 per cent of a B2B buyer's journey is done digitally.

Handorf said that this trend was another inspiration for his firm.

"We noticed everyone in their consumer lives were driven by online reviews and peer advice," he said. "It really hadn't happened in the B2B space but we had a hypothesis that it would. Consumer behaviours would start to transfer into their B2B lives."

Analysing the giants

The IT analyst space is crowded, and with some big names too. In 2015, Gartner generated almost $2.2bn in revenue, and as such is a force to be reckoned with, along with the likes of IDC, 451 Research, Forrester and Canalys.

But the traditional model of certain analysts focusing on certain areas of the market, or specific companies, is flawed and ripe for disruption, according to Handorf.

"I was responsible for analyst relations as head of product [at Big Machines]," he said. "My job was to brief the analysts to ensure they knew about Big Machines so that they wrote accurately about us in their reports. They had tremendous influence in buyer behaviour but we were frustrated because we knew we had to be a certain size to get coverage from the analysts and we also knew there was a lag in those reports.They'd typically come out a year or two after the research was done - not real time.

"In addition to that, the report was driven by one person's view [and they] had never actually used the software. We knew a customer knew this space and knew the details, and if you could aggregate a customer's knowledge, that would really help you create a better and more real-time report. So, all those things are what inspired G2 Crowd."

Handorf said another key benefit of his company's model is that the "vast majority" of its data is publicly available, but he added that certain, more in-depth reports, are only available for a fee.

Mitchell Feldman (pictured), CEO of RedPixie, said this model makes sense.

"That has got to be the natural evolution because typically Gartner will speak to one person in the business and that person is not necessarily the voice of the business," he said. "They have an opinion from their own perspective. So baselining all of the knowledge workers is a much more true view of where they are."

Ian Moyse, sales director at Axios Systems, said he has closely followed G2 Crowd but has used "traditional" analysts in the past.

"As with many markets, here is another being disrupted by newcomers with new more today-relevant approaches," he said.

Squaring up

Once G2 Crowd gathers its data from reviewers, it crunches the numbers and displays it on its ‘grid'. A unique, patent-pending algorithm is applied to the data to calculate customer satisfaction and market presence.

When asked about the similarities between G2 Crowd's grid and Gartner's famous Magic Quadrant, Handorf said: "People are used to a two-by-two rating system for software and so we thought it would be appropriate to do that, but in a much better way, with user reviews and real-time APIs from social data."

Despite the growing importance of peer reviews in the buying journey, the system could be more open to abuse from disgruntled users.

Handorf downplayed this and said: "There's a lot of ways we gather reviews and this gives us a wide spectrum of reviewers. Certainly one of those cases will be someone who has had a bad experience and wants to let the world know. That's probably the minority of what we see. The vast majority of people coming to our site are consuming data and they want to give back. They write a review on their software. It's not just people coming to our site to complain. It's people writing about their experiences with it. We reach out to users too internally."

RedPixie's Feldman agreed and said all companies are subject to the same review process, making it fair.

"It's irrelevant because if everyone is giving a bad review, it will be which is the best of the bad ones - that will reveal the result," he said.

Head to head?

The concept of using peer reviews and user insights is not unique to the likes of G2 Crowd. In November last year, Gartner launched Peer Insights - a service for tech buyers which gives them access to moderated reviews. Gartner declined to comment specifically about the emergence of new disrupters when contacted by CRN today, but when launching Peer Insights, it acknowledged the demand for this type of data.

"Certainly one of those cases will be someone who has had a bad experience and wants to let the world know. That's probably the minority of what we see."

"A consistent theme that we remind our tech provider clients of is that buyers look to others - peers and influencers - to help with their buying process," it said in a blog.

"In our research on influencers, peer reviews continues to be one of the more significant buying influences. But, it has dropped somewhat as buyers have come to view many reviews with increased scepticism - questioning whether the reviews are truly from a customer."

Despite the overlap his firm has with IT analyst giants, Handorf said that more often than not, customers want to use both approaches, not just one.

"I think what we're selling is the concept of transparency and openness to software vendors and how they can use this to change the way they market and sell their products. It's considerably different. I don't think when we go into deals people say ‘hey it's either going to be you or Gartner'. It doesn't work out that way and it's sometimes both."

Axios' Moyse said that all industries are ripe for disruption - even those which make money on telling others how to adapt to it.

"B2C users are increasingly familiar and trusting of peer reviews from Tripadvisor, IMDB, Amazon and the like, and this had to come to B2B," he said. "G2 Crowd pioneered this disruption and Gartner is playing catch-up with Peer Insights. Traditional analysts need to up their game and take note. The buyer dynamic has changed."