Big data is a competitive advantage companies can no longer ignore

By Bill Detwiler

Data has always been a key asset for some companies, but over the last three to five years, it has gone from "important" to "critical." Data, and more important, data analysis, have become a true competitive advantage for a variety of industries and businesses, not just the Facebooks, Amazons, and Googles of the world.

According to a 2014 study by Accenture and General Electric, 84% of the companies surveyed believe that big data analytics could "shift the competitive landscape" for their industry within the next year and 89% believe companies that fail to adopt a big data analytics strategy could lose both market share and momentum.
"The core advantage of data is that it tells you something about the world that you didn't know before. As your competitors learn more, you'll need to learn, too," said Hilary Mason, founder at Fast Forward Labs and coauthor of Data Driven: Creating a Data Culture.

Falling costs, easier collection methods, and better analysis

According to Mason, the following three trends are making big data a competitive advantage:

"First, CPUs and data storage have become so cheap that it's viable to collect and analyze data that previously would have been too expensive. Second, we have commodity tools that make it possible to do this without a huge investment in people and infrastructure. And finally, we've made a lot of progress as data scientists in knowing how to find value in data."

Businesses are also collecting significantly more data during their normal operations than they did in the past.

"It used to be that if someone walked into a store the only things the store would know is that their revenue went up and their inventory went down. And if they didn't buy anything, you didn't know anything," said Max Shron, founder of Polynumeral, a data science consulting firm. "Now, with loyalty cards and online shopping, you can collect every click and transaction."

The Internet of Things (IoT) movement is even pushing data collection beyond people's online activity and point-of-sale transactions into their everyday lives. Sensors and wireless communication technology are being embedded into everything from cars and clothes to healthcare devices and household appliances. Companies can use this data to better understand how customers interact with their products in the real world and then quickly adapt to customer wants and needs.

And big data isn't just for big companies. "Even smaller companies are putting resources behind their analytics teams, in the same way they put resources behind engineering and product teams," Mason said. "There are some great tools out there that allow even tiny businesses to use data effectively."

Collect the right kind of data

But merely collecting and warehousing a mountain of data isn't enough. To gain a competitive advantage, companies must use their data to determine behavior. The ultimate goal, according to Shron, is for companies to build a predictive model that's personalized.

Consider targeted ads on Facebook. Facebook knows exactly which ones members click on, which ones they don't, and what they look at after seeing the ad. Likewise, Netflix know which movies their subscribers watch, which they don't, and how long each movie sits in their queue. Polynumeral helped a client design a program that tailored fundraising messages to individual donors.

Collecting the right kind of data is also critical. "It's not how big the dataset is, but how detailed (or fine-grained) it is," Shron said. He suggests that companies collect all the data they can cost-effectively collect. "Don't double your capex budget to do it, but if you can [collect the data] fairly cheaply, then do it." You can always pare down a dataset, but you can't go back and get it. And no matter what type of data or how much you're collecting, Shron said it's critical for organizations to be able to identify customers across all their internal systems with a unique identifier, such as an email address, user ID, or cookie.

In addition to collecting their own data, organizations can also tap into a wide array of public or open datasets. For example, Shron's company worked with the World Bank to study poverty in Bangladesh. Traditionally, data for the study was collected every five years by people going door to door -- an expensive and time-consuming process. Analysts with Polynumeral used freely available satellite data from the National Oceanic and Atmospheric Administration (NOAA) to examine nighttime lighting patterns across the country.

By examining both datasets, they discovered a correlation between nighttime lighting and the economic trends shown in the survey data. They were then able to build a predictive model that used just the lighting information. Although the new satellite-driven model wasn't as accurate as the old hand-collected survey data, it did show overall trends in poverty and (because the data was free) allowed the World Bank to examine these trends on a more regular basis.

Businesses must adapt to this democratization of data. "Companies that rely on licensing a proprietary dataset should expect to be outpaced by competitors using modern data collection techniques, and more frequent updates and greater accuracy," Mason said.

Start with a plan

Whether your company already has a data analytics strategy, is developing one, or is just beginning to think about a plan, Shron and Mason recommend companies (big and small) follow a few basic first steps so the they don't get left behind in the big data gold rush.

"First, ask yourself, 'What data do we have?'" Mason said. "And then, 'What data should we have?' And finally, 'What assumptions do I have about the business that I can now validate with this data?'"

According to Shron, the mistake everyone makes is that they collected a bunch of data, give it to a data scientist, and then ask them to tell them what's in it. An analyst should really come in before the collection starts. You also want to make sure the data analyst understands the business enough know what data should be collected and what can be built from it.

The key is figuring out how to collect the data you want, building the right model, and connecting that model to the business so companies can act.