MLOps: From Model
to Production System
MLOps is the practice of putting machine learning models into operation reliably and keeping them there. Phase 06 of the data science lifecycle: deployment, monitoring, retraining.
From Prototype to Production
Deployment is a continuous process. We ensure reliable, maintainable operations.
Infrastructure
Production deployment, scalable and highly available.
Monitoring
Continuous monitoring with automated alerts.
Continuous Improvement
Processes for model updates as requirements change.
Production Monitoring Dashboard
MLOps: Definition, How It Differs from DevOps and Three Maturity Levels
MLOps (machine learning operations) is the practice of turning a trained model into a system that supports decisions in everyday work: versioned, automatically tested, integrated into the business application, continuously monitored and retrained when needed. MLOps carries the practices of software delivery (DevOps) over to models whose behaviour depends not only on code but on data.
A model only creates value when it actively supports decisions in production. And unlike an application, it behaves differently tomorrow as soon as the data it sees change: new customers or a replaced machine. That is why responsibility for a model begins on the day it goes live, instead of ending there.
MLOps and DevOps compared
| Question | DevOps (software) | MLOps (models) |
|---|---|---|
| What is versioned | the code | code, data and model state |
| What is tested | the function of the code | the function of the code, the validity of the input data and the quality of the model |
| What happens after delivery | operation and bug fixing | operation, monitoring for drift and controlled retraining |
| When a release is done | when the tests pass | when the tests pass and the new model beats the previous state |
Three maturity levels, in the language of the Mittelstand
Manual
A data scientist trains the model on their own computer, hands over a file, someone builds it in. Every retraining is manual work. Whether today's state matches the one from three months ago, nobody knows for sure.
Partly automated
Training, test and deployment run as a repeatable route that a person triggers. Every model state is versioned and reproducible, monitoring reports anomalies, retraining remains a deliberate decision.
Fully automated
The system trains, checks and replaces models itself as soon as defined thresholds are breached. Sensible with many models and rapidly changing data; requires test coverage and an operations team that have to be built up first.
ML Deployment: Six Steps from Code Push to Production
ML deployment, putting a model into operation, follows the route shown in the diagram above; specialists call it an MLOps pipeline. What counts for you is that every step is traceable and repeatable.
- Code push: A change to code, configuration or training data lands in the repository and triggers the route. Nothing reaches production past this entrance, not even a quick fix on a Friday afternoon.
- Test and validation: Automatic tests check the code, data checks the input data, and the newly trained model competes against the current production state. If it loses, the route ends here.
- Container build: Model, runtime environment and dependencies are bundled into a container that runs the same on every server. That leaves no room for "it worked on my machine".
- Staging: The container runs in a copy of the production environment with real interfaces and data formats. This is where errors surface that no automatic test sees: a wrong time zone, a renamed field, a column from the ERP that is suddenly empty.
- Canary: A small share of the traffic or a single site gets the new model, the rest stays with the old one. If the results deviate, a rollback takes it back before day-to-day business notices anything.
- Production: The model takes over full operation, versioned and with a documented state; from now on monitoring takes over, and every further change runs through step 1 again.
Four deployment forms, one use case each
Batch scoring
The model computes once a night across all records, for instance a demand forecast per article. Usually the cheapest form, because no service has to run around the clock.
Interface (API)
A service answers on request. In the predictive maintenance module, the hint of which asset is likely to fail in how many days and which maintenance window is advisable goes automatically into the maintenance system; the sensor data come read-only from the control system.
Embedded in the business system
The recommendation appears where the decision is made: in the ERP or on the planning board. In the fashion module for markdown control, the model first delivers proposals that category management checks; only after one or two validated seasons does the degree of automation rise.
In the browser or on the device
Small models run directly with the user, and the data do not leave the device. Our self-experiment with a drum model shows this form: 397.4 kilobytes of model weights, the prediction runs entirely in the browser, no upload.
The self-experiment: a web application with a model in the browser
Model Monitoring: Data Drift, Concept Drift and the Path to Retraining
A model rarely fails with an error message. It quietly gets worse. Model monitoring measures how fast the quality sinks before the loss reaches decisions.
Two kinds of drift
Data drift means that the input data look different from those in training. After a machine replacement the new sensor delivers different vibration values although the asset is healthy. The model has never seen such values and reports a risk that is none.
Concept drift means that the data look the same but that the relationship behind them has changed. At the change of season in fashion, the same pair of jeans reacts to the same discount differently from the spring; a pricing model that learned the relationship from last season recommends too early or too much.
Model drift is the umbrella term for the consequence of both kinds: predictive quality sinks. The cause decides the response: with data drift a correction of the data route is often enough, with concept drift only a new training with current data helps.
Which signals are monitored
- The distribution of the input data per feature, compared with the training state.
- The distribution of the predictions: if the model suddenly reports ten times as many assets as critical as in the previous week, something is usually wrong with the data.
- Technical metrics: response time, throughput, error rate, availability.
- Predictive quality as soon as actual values are available: only once the asset has failed, or not, does it show whether the warning was right. This comparison with the actual values is the most important metric and the slowest.
The dashboard in the diagram shows the structure: response time 42 milliseconds on average, 12,400 predictions, availability 99.97 percent, last retraining three days ago. These values are an illustration, not a commitment for your system; which values count for you is set by your operating agreement.
When retraining happens, and what it costs
A retraining is triggered when a previously defined threshold is breached: the distribution of a feature moves beyond a defined measure, or the measured quality falls below the limit the model passed in validation. The threshold is fixed before operation, not afterwards. That keeps retraining a considered decision instead of a reaction to the last bad month.
Computing time is rarely the cost item here; the drum model from our self-experiment trained 70 passes (epochs) in 12.3 seconds on a CPU. The expensive part is the work around the training: collecting actual values, checking the new model against the old one, approving and documenting the change. Anyone budgeting for retraining budgets for people and waiting time, not for CPU cores.
MLOps in the Mittelstand: Cloud, On-Premises or Both
The question of where to operate is decided by three criteria, in this order. May the data leave the premises, given data protection and customer contracts? What does the load profile look like: a nightly run, office hours or around the clock? And which operations team already exists?
For the third question we did the maths, on the most expensive case of all: the GPU operation of language models. In our cost analysis from 110 dated, public price sources, the entry-level server in-house costs 829 euros depreciation per month, the staff share for operation 3,924 euros, electricity 121 euros. The person costs more than four times the machine. A model of a few hundred kilobytes is not an infrastructure problem, as the self-experiment shows; the tasks of the people remain the same: monitoring, retraining, approving.
In our calculation the rented EU cloud was cheaper than own hardware in every base scenario. On-premises is nonetheless the right choice when four conditions come together: processing has to stay physically in-house, the load is high and constant, an operations team already exists and the quality of freely available models is sufficient for the use case. Even in this best case for own hardware, a difference of around 4,700 euros a month remained compared with the EU cloud GPU. That difference is the price of having the hardware in your own building: a paid requirement, not a savings programme.
The third answer combines both: training where the data are, and the finished model as a container to wherever it is needed; the container from step 3 makes this change of location possible without touching the model.
On tools, briefly: for maturity levels 1 and 2, open-source tools for versioning, experiment tracking, containers and delivery are enough; an MLOps platform with a licence only becomes a topic when many models and several teams share the same route.
The full calculation with break-even curves and sensitivity: Sovereign AI, honestly calculated: what on-premises really costs.Operations Handover: Who Looks after the Model after the Project
After a year, a model nobody looks after is a risk with a dashboard. That is why the handover is a project step of its own, with three deliverables.
Documentation
What the model does, which data it needs, which metrics it passed, which assumptions apply. Reproducible down to data state, code state and the seed of the random number generator.
Runbook
What your team does when a warning comes: check steps, contacts, rollback, the decision whether to retrain. A runbook is an instruction for the emergency, not a manual.
Training
The people who will work with the predictions tomorrow understand what the model can do and where it stays silent. A warning nobody can place is ignored.
The goal of the handover is that your own team takes over, with the documentation and the runbook in hand.
What deployment, monitoring and operations comprise as a service is on the page MLOps consulting: deployment, monitoring and operations as a service.Our Approach
Productionisation
Preparing for production use with interfaces and quality checks.
Automated Release
Traceable deployment process with tests and rollback options.
Monitoring & Alerting
Early detection of changes in model performance and data quality.
Operations Handover
Knowledge transfer, documentation and process establishment for your team.
Typical Deliverables
Frequently Asked Questions about MLOps
What is MLOps?
MLOps stands for machine learning operations: the practice of putting a trained model into operation reliably and keeping it there. It comprises versioning of code, data and model, automatic tests, deployment as a service or batch run, monitoring for drift and controlled retraining. Without MLOps a model remains a prototype with an expiry date.
What is the difference between MLOps and DevOps?
DevOps delivers software whose behaviour depends on code alone. MLOps delivers models whose behaviour additionally depends on data. That is why MLOps also versions data and model states, tests the validity of the input data alongside the code and monitors after delivery whether the data or the learned relationship have changed.
What does ML deployment mean?
ML deployment (machine learning deployment) is putting a model into operation: the checked model state is bundled as a container, tested in a staging environment against real interfaces, first assigned to part of the operation and then fully released. Deployment forms are the nightly batch run, an interface, embedding in the business system or operation directly in the browser.
What is model drift and how do you detect it?
Model drift means that the predictive quality of a model sinks in operation. The cause is data drift (the input data change, for instance after a machine replacement) or concept drift (the relationship changes, for instance at the change of season). Drift is detected by comparing the data distributions with the training state and, as soon as actual values are available, by the measured quality.
How often does a machine learning model have to be retrained?
As often as the data demand it, not by calendar. Retraining happens when a previously defined threshold is breached: the data distribution moves, or the measured quality falls below the limit from validation. A markdown model can only be evaluated after the end of the season; the module reckons with one or two seasons. What is expensive is checking and approval, not computing time.
Do we need an MLOps platform or are open-source tools enough?
For maturity levels 1 and 2, open-source tools are enough: versioning, experiment tracking, containers and a delivery route, as your IT knows them for software. A licensed MLOps platform pays off when many models and several teams share the same route. What matters is the fit with your existing IT, not the length of the feature list.
Can an AI model be operated on-premises?
Yes. Model, retraining and monitoring can be operated entirely in your own data centre. In our cost analysis the rented EU cloud was cheaper in every base scenario; own operation is a paid requirement. It makes sense when data may not leave the premises, the load is high and constant and an operations team exists. The largest item is staff, not electricity.
Evidence on Deployment, Monitoring and Cost
Sovereign AI, honestly calculated: what on-premises really costs Predictive maintenance in operation Markdown optimisation module for fashion The self-experiment: a web application with a model in the browserLet's talk about your project
Every project is unique. Tell us about your challenge.
Book a free initial call