Technology: How Smart can You Make Your Data?

Casino’s have long been leaders in collecting massive amounts of details about their customers. Casinos like MGM Mirage and Harrah’s Entertainment keep multi-terabyte databases on millions of their customers, tracking all sorts of minutiae to help them personalize customer incentives, and to ultimately help them maximize their own profits.

On-line heavyweights like Google have even larger databases. Google collects user data from AdSense, AdWords, Google Toolbar, Analytics, GMail, Feed Reader, Google Checkout, Youtube, Google Maps, and other on-line assets.

Technology has made it easier and easier to track and aggregate data about customers, and many companies are seeing what advantages the technique can bring to them. Many companies have started initiatives to mine customer data to their advantage. But does the investment in all the number crunching really pay off? Maybe not.
The Wall Street Journal quotes Stanford business professor Robert Sutton as criticizing the overuse of data analysis and data mining. Sutton admits that simple spreadsheets aren’t sophisticated enough to be able to model all the nuances of running a big company, but the opposite extreme of developing complex data models may not be much better at determining how to best run an organization.

Much of the problem is that historical data wasn’t created in a vacuum. There were likely many external forces that significantly influenced a previous set of events. Trying to make predictions from the past is a trap that is easy to fall into, and the results of the policies based on those predictions can be painful when played against a different backdrop.

The Journal quotes an example from Babson College management professor Thomas Davenport of lenders and investors losing massive amounts of money with subprime mortgages. Long term historical data showed that default rates followed a pattern based on the credit score of the borrower. When the pattern changed, lenders who had made bets based on the previous trends lost big.

Techniques like modeling, simulation, data analysis and data mining can all provide useful information, but the results of these exercises need to be interpreted within the context of the assumptions of the underlying data. Without that context, the true meaning of the numerical results can’t really be assessed.