Cloudera declares victory in the war for big data dominance

What does Cloudera want to be when it grows up?

That was the question one analyst asked Cloudera co-founder and chief strategy officer Mike Olson at the company's analyst day this week, to which Olson responded, "We want to be the big data company" -- not any big data company, but the big data company.



By some measures, Cloudera is well on its way toward the goal, with competitors like Pivotal dropping their Hadoop ambitions -- which led Cloudera CEO Tom Reilly to effectively declare victory. But in fact, the market is still wide open.

The dawn of big data

Hadoop creator and Cloudera chief architect Doug Cutting made the point himself: Hadoop adoption has been doubling each year, which means half the users are newbies. As such, it's not surprising that "lots of Hadoop use is early-stage."

When Gartner asked enterprises where they stood in their big data adoption, the trends clearly point to a fast-growing market:

Within that market, Cloudera claims it has 53 percent of the Hadoop market, compared to Hortonworks (16 percent) and MapR (11 percent). That's nice, but according to Ventana Research, while customers cite big data as critical, "Hadoop is lower in plans than other approaches."

That's because big data is much bigger than Hadoop, although the Hadoop vendors don't seem to know it. In fact, NoSQL databases like MongoDB and Cassandra are much more broadly deployed than Hadoop, and Hadoop itself is under attack from Apache Spark.
That threat, however, is an opportunity, according to Cloudera's Matt Brandwein. Rather than wedding itself to any particular technology (such as MapReduce), Cloudera's model is to embrace the big data technologies its customers most want to run. Hence, Cloudera has become a big Spark proponent.

Making money on free

Regardless of what Cloudera puts into its platform, Wikibon analyst Jeff Kelly highlights that 75 percent of Hadoop users don't pay Cloudera (or anyone else) for commercial support and instead choose to roll their own. Even among those that depend on the Cloudera Data Hub, 32 percent don't pay for support, Cloudera's Justin Kestelyn acknowledged.

As with any open source company, Cloudera's biggest competitor is ... itself.
Clearly, the revenue model remains to be fully fleshed out. At the analyst event, the company even went so far as to say that "we are not making money, but we are losing money more efficiently than our closest competitor [Hortonworks]." That's not enough to sustain Cloudera as the big data company.

Of course, the company has hundreds of millions in the bank, thanks to a gargantuan funding round from Intel last year. But Olson isn't betting on VCs to create Cloudera's future. He's betting on data. As Olson told the assembled analysts, "As long as we stay close to the data, there will be a future for Cloudera. We will figure out the monetization."
Cloudera may have to figure it out soon.

Hortonworks spoiling for a fight

After all, Hortonworks isn't standing around, waiting for Cloudera to dominate the market. I've not been a big fan of the pure-play open source Hadoop vendor (having seen all but one pure-play open source company fail), but judging by the inordinate attention Cloudera paid to Hortonworks during the analyst day, the former still sees the latter as a threat.
The same goes for ODP (Open Data Platform), the Pivotal/Hortonworks love child. While ODP smacks of failed consortia before it, Cloudera refuses to ignore it.

Indeed, as analyst Holger Mueller opined, "The amount Cloudera commits to ODP [shows] that they are ... at least a little worried about it." Though analysts like Merv Adrian rightly pillory ODP as "not [having] actually done anything yet except collect money and put out press releases," Cloudera spent far too much time raging against the ODP and Hortonworks machines (over committer counts, standards wars) to credibly claim nonchalance -- which is too bad, because Cloudera seems to be skating to where the big data puck is heading: applications.
While Cloudera continues to contribute heavily to Hadoop and other big data infrastructure, it's focused on delivering value higher up the stack. As Reilly stresses, "We will move closer and closer to the application layer to deliver value, but [Cloudera is] not going to build apps ourselves."

This leaves Hortonworks to peddle support for infrastructure that will remain open source, a somewhat thankless task with even less generous profit margins, paid by developers and those who muck about in low-level infrastructure. Meanwhile, Cloudera gets the benefit of robust infrastructure while monetizing value closer to the business user.
So far, this strategy pays Cloudera more than $100 million a year. That may be more than others are making, but it's nowhere near enough for Cloudera to declare victory yet.