You are here

You are here

Why your predictive analytics models are no longer accurate

public://pictures/dan_simion_0.png
Dan Simion Vice President of AI and Analytics, Capgemini North America
 

Analytics models have always used patterns from the past to predict the future. Naturally, previous behaviors, events, and trends not only shape what we believe will happen, but also the forecasts conducted by machine-learning models.

However, the pandemic has created an unprecedented situation, with conditions that have never before existed. That unfamiliarity has led to unreliable models with irrelevant predictions that aren't applicable during the current crisis.

Certain industries are being affected more than others by this disruption, Capgemini has found. Buying behaviors are shifting across retail and transportation, with 62% of consumers saying they'll start purchasing from brands that show higher levels of product safety, and 75% of car buyers saying their motivation is to gain greater control of hygiene.

Many consumers are also moving online to avoid physical contact, expecting a positive digital customer experience driven by automation and efficiency. New patterns are emerging within organizational datasets—even from a small sample size that contains information collected only within the past few months.

These new patterns are crucial to businesses looking to recapture accurate data and analytics about their customers through real-time forecasting. Instead of taking the traditional approach of relying on historical data—typically three years' worth that's no longer relevant—the new real-time forecasting approach leverages specific data that is only relevant based on the new conditions. For the current pandemic, that would be any data collected since March.

With such a recent data sample, an organization can model based on the current variables, and then evaluate its model's prediction accuracy against what actually takes place. If a company were to analyze behaviors and patterns with data collected from before the pandemic, it would paint an inaccurate and unreliable picture.

To regain control of your predictive analytics and modeling, here are four actionable best practices that your company can take with real-time forecasting.

New reality needs new data

Forecasting models are the opposite of wine; they do not get better with age. Base them on new data driven by the new reality to be more accurate and react more quickly, given the very fast-paced changes we're seeing.

Traditional forecasting methods use time-series approaches that pull data at certain pre-determined moments, but with real-time forecasting, an organization can predict outcomes based on the most recent data as it's coming in, on a continuous basis.

Continuous testing creates resiliency

Retraining machine-learning models at a high frequency makes them more resilient to potential future disruptions. This continuous testing and updating limits the amount of data drift, which occurs when companies run models with data that was relevant several months or years ago. Those models pull in outdated and irrelevant information.

Identify your top-performing model and continue to fine-tune it frequently. Data science teams must react quickly to re-train models to ensure that the data coming in is as close to real-time as possible.

Throw out your three-year plans

In the past, companies created plans several years into the future and would leverage forecasting models to shape their predictions. Long-term forecasting has always been less accurate, but now those models and plans are, in many cases, useless. The new reality calls for more short-term forecasting.

Short-term predictions yield more accuracy and allow companies to make smarter, safer decisions. With the future so unclear, for the moment, focus on short-term planning.

The pandemic has produced many challenges for companies across industry verticals. Major impacts on operations, supply chains, budgets, sales, marketing, and many other business areas have created an essential need for an accurate glimpse into what's coming next. Data science and IT teams that can build fresh models, provide detailed reports, and showcase reliable predictive outcomes will be vital contributors as many organizations push toward recovery.

It has never been more important for companies to see that they have a way to improve the accuracy of their modeling and help their businesses to effectively rebound from the disruption. Real-time forecasting can be that solution to get them back on track.

Keep learning