Big companies love big data. It allows detailed analysis of business operations, targeting of potential customers and forecasting of outcomes. It can guide business decisions, provide statistical evidence and enhance managers’ confidence. It may also cause problems when overly simplistic formulas and models are used to represent a complex environment.

Recall or Litigate

In the movie Fight Club, Ed Norton’s character applies “The Formula” in his automotive Recall Coordinator job. According to the narrator, analyzing three or four variables determines whether a recall is the best (read: the cheapest) option for the unnamed company. In practice things are more complex. In August, Progressive Insurance lost a number of customers after choosing to litigate an unclear case and undergoing a social media backlash (see details here and social media lessons here). I doubt Progressive’s formula anticipated the social media reaction when handling this specific case, and some have suggested that businesses add social media indicators to their customer databases. Accommodations could then be made for additional customer variables, such as a social media presence, in business calculations.

Some Contrasts

A challenge with static historical data, whether it is social media activity or purchasing history, is that it may not give a full picture of the environment. Take a group of potential customers – all are in their thirties, have mediocre credit histories and substantial student loans. They may appear identical from a data perspective. But if a subset of the group has recently finished medical school and been awarded lucrative internships, they may be more appealing from a marketing perspective. Many data analysis programs miss this type of difference. In a similar way, companies often narrowly define their internal diversity/inclusiveness initiatives and track statistics based only on ethnic background and a small number of variables. The result is often programs with ill-defined objectives and employees who don’t clearly match any of the classifications.

Beyond the Formula

In most cases, there is a balance between mass classification and customer segments of one. Understanding the key variables that define a group – and how those variables may change over time – is important to perform effective data analysis. Being respectful of individual differences is critical to ensuring your customers, employees and other stakeholders don’t feel like numbers. In many cases moving to some form of self-service data management and using a wide range of data elements are beneficial. Thinking beyond the standard formula is a good place to start.