A few months ago, alongside a data team, I reviewed a customer churn prediction model that had been "finished" for nearly a year. It worked beautifully in the data scientist's notebook, with an accuracy that made everyone proud. The problem is that it lived there: in a notebook, on a laptop, run by hand whenever someone remembered. It had never touched a real customer. That scene, which repeats itself across so many organizations in the region, captures the challenge of 2018: we already know how to train models; what almost no one knows is how to put them to work.
In short: Most machine learning models never reach production because we treat a living system as if it were a one-time deliverable. Operationalizing them requires solving data, deployment, monitoring, and retraining as a continuous cycle. A model that isn't governed degrades in silence.
Training a model is the glamorous part and, in reality, the shortest part of the work. The lab is a friendly environment: the data is clean, frozen in a file, and the scientist runs the code whenever they like. Production is exactly the opposite. There the data arrives dirty and in motion, other systems depend on the response, and no one is watching the screen at three in the morning when something fails.
I have seen several patterns that kill promising models before they generate a single cent of value:
The common denominator is treating the model as a project with a closing date, when it's really a product that has barely begun to live the day it's deployed.
In the lab you work with a snapshot: a dataset extracted on some random Tuesday. In production you work with a moving picture. Sources change schema, a provider stops sending a field, someone renames a column in the source system and, suddenly, the model receives garbage without realizing it.
Operationalizing begins with building a reliable, repeatable data pipeline, not a script run by hand. That implies:
If the data foundation is fragile, everything we build on top of it will be too.
Deploying isn't copying a file to a server. It's exposing the model so that other systems or people can reliably consume its predictions. In practice there are two main approaches, and it's worth choosing consciously:
In both cases, the model stops being a research object and becomes a software component with responsibilities: it has to be available, respond on time, log what it does, and be able to roll back to a previous version if the new one fails. That engineering discipline is precisely what's missing when a project stays in the notebook. In our artificial intelligence practice we insist that deployment be designed from day one, not at the end.
Here is the deepest difference between traditional software and a model. A program that sorts a list will sort it just as well five years from now. A model won't. The world it learned changes: customers behave differently, a competitor appears, the economy shifts, an internal process is modified. We call this drift, and it happens without warning. The model keeps responding with total confidence while its predictions grow steadily worse.
That's why a model in production needs monitoring on two levels:
The second level is the one almost always neglected, because it requires comparing predictions from weeks ago with what actually happened. Without that feedback loop, you discover the model broke when an unhappy customer points it out, and by then it's far too late.
If the model degrades, the logical consequence is that it has to be renewed. Retraining shouldn't be an occasional act of heroism but a planned process: how often, with what data, who approves the change, and how you validate that the new model is genuinely better than the one in production.
And here enters a word that in 2018 still sounds strange applied to analytics: governance. Governing a model means being able to answer, at any moment, uncomfortable questions:
This isn't bureaucracy: it's what separates an experiment from an asset the organization can depend on. The maturity to sustain that cycle can't be improvised, and it's exactly what we measure in an AI readiness assessment: does the company have the data, the processes, and the roles to operate models, not just to build them?
No tool solves this on its own. Taking models to production is, above all, a problem of collaboration between three worlds that historically don't talk to each other: those who understand the data, those who operate the systems, and those who own the business. When those three converse from the start, the model is born designed to live in production. When they don't, it's born condemned to the notebook.
My practical recommendation for this era: start small, with a single model that truly matters, and take it end to end all the way to production. A modest model that makes real decisions every day is far more valuable than ten brilliant models sitting in an archive.
What's the difference between training a model and putting it into production?
Training is teaching it from historical data in a controlled environment. Putting it into production is integrating it into the operation so it makes or supports real decisions reliably, continuously, and monitored. The first is an event; the second is a lifecycle.
Why does a model degrade if the code doesn't change?
Because the model learned from a world that changes. When customer behavior or environmental conditions drift away from the data it was trained on, its predictions lose accuracy even though the code is identical. That's called drift.
How often should a model be retrained?
There's no single rule: it depends on how fast the phenomenon it predicts changes. The right approach isn't to set an arbitrary calendar, but to monitor performance and retrain when the indicators show it's deteriorating. Monitoring tells you when.
Do I need a huge team to operate models?
No. You need discipline more than size: reliable data pipelines, versioned deployment, monitoring, and clear roles of responsibility. A small organization that closes that cycle well outperforms a large one that ships loose notebooks.
If your organization has promising models that never quite leave the lab, the problem is almost never the algorithm: it's the lack of a clear path to production and of a mechanism to govern them once there. That path can be designed, and it's worth doing before accumulating more orphaned experiments.
At SUMāTO we accompany that transition from model to product, with an assessment that identifies where your data, deployment, and monitoring cycle is stuck. If you want to put your models to work instead of archiving them, let's talk about a first assessment for your case.