4bigdataSome marketing questions require robust analytics. For example, understanding what mix of channels are driving sales for a particular product, or in a particular customer set, or what sequence of channels is most effective. These types of questions often require large sets of data, or what is being referred to as Big Data.

Study after study shows that marketers are struggling with mining and analyzing data in order to derive valuable insights, actionable intelligence and manage performance. A recent report by EMC found that only 38% of business intelligence analysts and data scientists strongly agree that their company uses data to learn more about customers.

Big Data isn’t new; it’s just gone main stream. A recent study found that almost half (49%) of US data aggregation leaders defined Big Data as an aggregate of all external and internal web-based data. Others defined it as the mass amounts of internal information stored and managed by an enterprise (16%) or web-based data and content businesses used for their own operations (7%). But 21% of respondents were unsure how to best define Big Data.

IDC defines big data as: “a new generation of technologies and architectures, designed to economically extract value from very large volumes of a wide variety of data, by enabling high-velocity capture, discovery, and/or analysis.”

Big Data can include transactional data, warehoused data, metadata, and other data residing in extremely massive files. Big Data incorporates multiple data sets—customer data, competitive data, online data, offline data, and so forth—for a more holistic approach to business intelligence.

Mobile devices and social media solutions such as Facebook, Foursquare, and Twitter are the newest data sources. Every day, people are creating an incredible 2.5 quintillion bytes of data, with 90% of the world's customer data having been created -- and stored – since 2010. And the vast majority of real-time data created today is unstructured data.

Big Data is the analysis of all the data surrounding content creation and consumption. Smart marketers know they need to incorporate this data their customer models to predict what customers will want, and then use this information to adapt their marketing strategies to give customers the right products when, where, and at the price they want. The difficulties associated with amassing, analyzing and using these large and disparate data sets keep many companies from maximizing their use of Big Data.

A recent study by Connotate found the top challenge with Big Data was the time and manpower required to collect and analyze it. In addition, 44% found the sheer amount of data too overwhelming for businesses to properly leverage.

So while Big Data might seem a bit daunting, these steps will help you navigate using Big Data:

1. Clarify the question

Before you start undertaking any data collection, be sure you have a clear understanding of the question you are trying to answer. Using Big Data starts with knowing what you want to analyze. By knowing what you want to focus on you will be better able to determine what data you need. Some common questions are which customers are the most loyal or which customers are most likely to buy X? Big Data is about looking beyond transactional information, such as a click-through data or website activity.

2. Clarify how you want to use the data

Will you be using the data for your dashboard or to define a customer target set for a specific offer or to make program element decisions (creative, channel, frequency, etc.).

3. Think beyond the initial question

Invariably the answer to one question leads to more questions. If you’re not sure, hold a brainstorming session to explore all the ways the data could be used and potential questions the answers might prompt. Structure your data in a dynamic way to allow for quick manipulation or sharing. Aggregated data structures and data cubes aid with this step. You will want to construct your data cubes so they contain elements and dimensions relevant to your questions.

4. Identify data sources that need to be linked

It’s important to consolidate and link data if you want to run analysis against it. Once you identify the question and how you want to use that data you will have insight into what data you need. More than likely you will need to collect the data from disparate data sources in order to create a clear, concise, and actionable format. It may be necessary to invest in some new tools so you can pull and analyze data from disparate locations, centers, and channels. These tools include massively parallel processing databases, data mining grids, distributed file systems, distributed databases, and scalable storage systems.

5. Organize your data

Create a data inventory so you have a good understanding of all your data points.

6. Create a mock version of your data output

This is a key step to helping you determine the data sets. It will also help you with thinking about how you will convert the results into a business story. A good analyst is able to use the data to tell a story that will illuminate trends and issues, forecast potential outcomes, and identify opportunities for improvement or course adjustments.

Undertaking projects associated with Big Data can seem a big bite to chew, but the effort is more than worth it. The promise of Big Data is more precise information and insights, improved fidelity of information and the ability to respond more accurately and quickly to dynamic situations. Big Data is essential for marketing when it comes to strategy, product and pricing decisions and for performance management.

By Laura Patterson, President, VisionEdgeMarketing