Do Predictive Analytics Work?

Do predictive analytics work?

There’s been a lot of discussion in the channel about forward-looking metrics and predictive analytics, but do predictive analytics work? In our latest Channel Pulse, we asked what percentage of metrics were forward looking?  The answers from our audience of senior channel professionals were all over the place.

Forward Looking Metrics

We expected progressive organizations to have approximately 20% to 50% of their metrics as forward looking.  We found 34% of companies fit this category, while another 47% were lagging behind.  What surprised us, was that a significant group (17%) indicated more than 50% of their metrics were forward looking.

These results made us wonder if we asked the question incorrectly? We went back and here’s how the question was worded:

How much of your channel reporting and analytics are forward looking/predictive?

This leaves us with the quandary of why almost half the audience seems to feel their metrics are forward looking/predictive.  Do organizations have more forward-looking metrics than we thought?

This led to another puzzling result in the Q2 Channel Pulse.  We asked a set of questions about the ability to predict revenue and program outcomes.  The audience did not show a great deal of confidence in their ability to do either.  They also indicated that the further away the outcome, the more accurate they could forecast.

Predictive Analytics Accuracy

This result seems a bit counter intuitive.  Shouldn’t one be able to predict an outcome when you are closer to it versus farther away?  The data seems to indicate the opposite of this.

Forward looking metrics are the foundation of predictive analytics.  Predictive analytics are achieved by taking data derived from metrics and using advanced algorithms to forecast future outcomes.  If the data input is inaccurate (forward-looking metrics) then the prediction will most likely be inaccurate.

Garbage In – Garbage Out

So back to the question: “Do predictive analytics work?”  If more than 50% of organizations have forward-looking/predictive metrics, and more than two thirds indicate the future is “hit or miss” or “unpredictable”, then we have to draw the conclusion that predictive analytics are not working.

If we factor out all the companies having below 50% forward-looking/predictive metrics, we find that almost 50% of the remaining companies can accurately forecast on a quarterly and annual basis.  From this, we conclude that forward-looking/predictive analytics are making an impact in some companies.

At 360insights we have successfully implemented predictive analytics strategies with our customers.  What enabled successful results was the development of configurable, optimized and dynamic forecasting models incorporating multiple variants (or features) of a target KPI (e.g., sales volume, rebate spend, etc.).  We’ve developed a multi-model approach requiring preparation of various data sets which contain historic observations of the target KPIs, and features that influence their outcome.

Daily records of KPIs are aggregated into weekly, monthly or quarterly amounts at a reseller level.  Predictive models at each time scale and reseller level are then optimized to yield highly accurate KPIs.  We then produce a final prediction of the KPIs for the next quarter by taking the average of all predictions yielded at different time scales.

So where does this leave us? Do predictive analytics work?

After reviewing the data, it occurred to me that we might be asking the wrong question.  The data might be telling us more about where we are with adoption and less with the value of the innovation.  There are two methods widely used to describe the maturity of new technology adoption: Gartner’s Hype Curve and Rogers’ Diffusion of Innovations.

Gartner’s Hype Cycle

Gartner’s Hype Cycle provides a conceptual view of technologies as they progress across five phases:

1. Technology Trigger – The technology breakthrough that gets things started. This ranges from an early vision, to use cases, or a proof-of-concept. Often no usable products exist.
2. Peak of Inflated Expectations – Early market buzz produces a few successful attempts and success stories encourage more to try. This is often accompanied by scores of failures.
3. Trough of Disillusionment – Interest decreases when there are more failures than successes.
4. Slope of Enlightenment – The market begins to learn how to repeat success and minimize failure.
5. Plateau of Productivity – Mainstream adoption starts to take off as more and more companies are successful with it.

Gartners Hype Cycle

Looking at the use of predictive analytics in the channel, the number of companies trying to use them has increased but we are still seeing many that are failing to realize value.  Using Gartner’s Hype Cycle would put us in the “Through of Disillusionment”.

Rogers’ Diffusion of Innovations

Another method for looking at the maturity of technology as it spreads in the market is Rogers’ Diffusion of Innovations. This theory was made popular in Geoffrey Moore’s book “Crossing the Chasm” in the early 90’s.

Diffusion of Innovation explains the speed an innovation takes hold in the market over time.  Rogers describes the market segments which adopt innovations at various times as follows:

Innovators – Risk takers who have close contact to entrepreneurs inventing new things.
Early Adopters – Opinion leaders.
Early Majority – Adopt innovations much slower than innovators or early adopters.
Late Majority – Adopt innovations slower than the average participant.
Laggards – The last to adopt an innovation.

Diffusion Innovation

I would conclude that we’re looking at the tail end of the early adopters stage and starting to move towards an early majority if using the Diffusion theory to look at adopters of predictive analytics in the channel.

If we overlay the two theories and compare our analysis, it appears they marry up nicely.  This confirms my assumption that the data from the latest Channel Pulse is telling us about the maturity of predictive analytics, and not their value.

Peak of Inflated Expectations Theory

Were we asking the right question?

Instead of asking do predictive analytics work, we should have asked about the market maturity of predictive analytics.  We’re too early to make a conclusion about the value predictive analytics will produce for the channel.

Our experience has shown us that predictive analytics offer real high value to those who successfully implement them.  Knowing this, we believe channel professionals should be asking where their organization wants to participate on this maturity curve?

If your organization is trying to be an innovator, or early adopter, you’re late to the party.  There is still time to join the early majority, but if you don’t act soon you may find yourself a late adopter or laggard.

Is access to better channel data the key to success?

What holds most companies back is access to the data and data science to ensure success.  Unfortunately, sales and marketing, and particularly the channel often, is a low internal priority to internal technical resources.

Channel data is often siloed in many proprietary systems.  In addition, technical resources and expertise to access this data is at the vendor level and not the consumer level.

Success requires choosing platforms that provide better access to the data and aligning with vendors who can help bring data together, cleanse it and apply the data science necessary to derive value.

To view the detailed findings from the Channel Pulse™ report, visit www.TheChannelPulse.com.

The report is made available free of charge to any interested party.  Those interested in obtaining a copy and/or participating in the Channel Pulse community and any future reports or events can do so at www.TheChannelPulse.com. For more information about 360insights, visit www.360insights.com.