The data science lifecycle follows CRISP-DM, the standard process for data science projects. myBytes works in its six phases and walks through an example of what happens in each phase.
From Your Raw Data
to Business Impact
A model is only as robust as the process that produces it. That is why we skip none of the six phases.
The Complete CRISP-DM Lifecycle
Each phase builds on the previous one: iterative, transparent and aligned with your business goals.
Point to a phase in the ring to see what it is about and which wasted investment it prevents. As you scroll, the explanation follows the ring by itself.
Understanding
This phase turns a business problem into a measurable question: what should change, how will we know, and what is the result worth? It also sets the boundary of what the model cannot do.
Without this phase, models get built for questions nobody asked. A success criterion in euros stops months flowing into a technically sound model with no business value.
& Exploration
This phase reviews which data exist, whether they are accessible and reliable, and whether they can carry the question from phase 1 at all. Every source gets a verdict: usable, needs work or missing.
AI projects usually fail on data, not on the model. Knowing the gaps before modelling means deciding early whether to continue, rather than halfway through the budget.
& Engineering
Raw data become a clean, documented dataset: linked, cleansed and enriched with the features the model will later use. This phase often takes the most time.
Errors in preparation only surface during evaluation and cost twice as much there. A traceable pipeline makes every later result repeatable and verifiable.
& Modeling
Several modelling approaches compete against a simple reference, documented as experiments. The goal is not the most complex model but the one that answers the question measurably better.
The reference prevents expensive technology without measurable benefit: if a model does not beat the simple rule, it goes no further. Documented experiments spare you repeating the work every time a question comes back.
& Validation
The best model is tested against held-out data and against the success criterion from phase 1, in euros and not only in model metrics. Subgroups and edge cases are checked separately.
This is where the decision to deploy rests on numbers rather than hope. A model that does not hold up in testing does not go into operation, which avoids the most expensive mistake of all: a failed rollout.
& Monitoring
The model is embedded in the systems and workflows where the decision is made, and it is monitored continuously: data drift, accuracy, usage. Retraining and handover to your team are part of it.
A model without monitoring quietly loses value as soon as the data change. Running the model and handing over the knowledge keep the investment working, so that you do not start again from scratch after a year.
CRISP-DM: Definition, Origin and How It Differs
The Cross-Industry Standard Process for Data Mining describes in six phases how a business question becomes a model in operation.
CRISP-DM stands for Cross-Industry Standard Process for Data Mining. The process model breaks a data science project down into six phases: Business Understanding, Data Understanding, Data Preparation, Modeling, Evaluation and Deployment. Each phase has a guiding question, defined tasks and a deliverable that the next phase needs. Since the 2000s it has been regarded as the de facto standard for data mining and data science projects.
CRISP-DM was developed from 1996 by Daimler-Benz, NCR and the tool vendor ISL, which later became part of SPSS, with the insurer OHRA as the fourth partner; the CRISP-DM 1.0 guide was published in 2000. The model is deliberately industry-neutral and tool-independent. That is exactly why it fits projects nobody had in mind in 1996: a returns model in fashion retail, a failure forecast in manufacturing and a turnover model in logistics all pass through the same six phases.
The newer process models build on CRISP-DM rather than replacing it. ASUM-DM from IBM adds operations and project management, TDSP from Microsoft adds roles, tasks and a project structure but has not been maintained since 2023, and DASC-PM from Nordakademie extends it with data provisioning and scientific grounding. Anyone who knows one of these models will recognise CRISP-DM in it.
CRISP-DM itself does not define roles; in practice the phases still fall to recognisable owners. Phases 1 and 2 need the business side: the business question, the success criteria and the knowledge of which data exist in-house and what a field in the merchandise system actually means come from the people who work with the data. Phases 3 and 6 are data engineering: pipelines, versioning, operations. Phases 4 and 5 are the work of the data scientists. No phase belongs to one role alone, and a project that staffs only the middle phases has neither a robust question nor an operation.
CRISP-DM is iterative, not linear. The arrows also run backwards. If the data assessment in phase 2 shows that the business goal from phase 1 cannot be answered with the available data, the project goes back to phase 1. If a model misses the success criteria in evaluation, it goes back to modelling or to data preparation. The outer loop closes after deployment: new data from operations raise new business questions.
The Six Phases of the Data Science Lifecycle in Detail
Click on a phase for the full description.
Business Understanding
Defining business goals and translating them into analytical questions.
Guiding question: Which decision should be better at the end, and how will you measure that? Deliverable: a target picture with success criteria and an assessment of whether the data can carry the question.
Data Acquisition & Exploration (Data Understanding)
Identifying and accessing relevant data sources with quality assessment.
Guiding question: Which data do you already have in-house, and is it enough for the question from phase 1? Deliverable: a data inventory with a quality rating per source.
Data Processing & Engineering (Data Preparation)
Cleaning, transforming and deriving meaningful features.
Guiding question: How do raw data become clean, reproducible datasets? Deliverable: versioned data pipelines with quality checks. This phase takes up the largest share of project work.
Exploratory Analysis & Modeling
Systematic model development and optimisation with traceable experiments.
Guiding question: Which model answers the question better than a simple reference? Deliverable: documented experiments with a baseline and the best candidates.
Evaluation & Validation
Rigorous assessment of model performance against your business goals.
Guiding question: Does the model meet the success criteria from phase 1 under realistic conditions as well? Deliverable: a validation report with metrics, driving factors and a release recommendation.
Deployment & Monitoring
Production deployment with monitoring and continuous improvement.
Guiding question: How does the model get into operation, and who notices when it degrades? Deliverable: a production system with monitoring and an operations handover to your team.
CRISP-DM examples: eight use cases, six phases, each worked through
Eight use cases from our series, from manufacturing to process data, taken through the data science process phase by phase. All figures are model calculations for fictitious companies, not client results.
CRISP-DM example fashion: cutting returns, worked through in six phases
A mid-sized fashion company with 1,800 articles and 18 markets sells jeans online. The return rate for jeans is 38 percent, and 34 percent of those returns are down to "does not fit".
Phase 1 · Business Understanding
The business question is not "we want AI" but: What share of fit-related returns can be avoided if the customer is recommended the right size at checkout? The success criterion is the return rate per category, measured before and after the launch. Phase 1 also sets the limit: new customers without a purchase history can initially only be served with category averages by such a model.
Phase 2 · Data Acquisition & Exploration
The data are in-house: purchase history, return reasons, the ordered and the kept size per customer and cut, plus the size charts per article. Phase 2 checks whether the size information in the returns data is complete. If it is missing, there is no model, only a statistic.
Phase 3 · Data Processing & Engineering
From the raw data, each customer gets a profile of kept and returned sizes per cut, and each article a calibrated size run. The model calculation assumes that a correct size recommendation would have been enough for 68 percent of fit-related returns. Whether that assumption holds is decided here, on the quality of the size fields.
Phase 4 · Exploratory Analysis & Modeling
The model is collaborative filtering with body-shape matching: customers with similar purchase and return behaviour are recommended the size that similar customers kept for this cut. The benchmark is the size chart without a recommendation, as a simple reference model before any more elaborate approach. If the model does not beat that reference, the effort is not worth it.
Phase 5 · Evaluation & Validation
In the model calculation the model picks the right size in 82 percent of cases. Fit-related returns fall by 18 percent, and the jeans return rate falls from 38 to 31 percent. A published vendor case reaches up to 20 percent fewer total returns; the model calculation deliberately stays below that. The evaluation also checks whether the effect appears in all categories or only for jeans.
Phase 6 · Deployment & Monitoring
The model runs as a widget on the product page: "Based on your previous purchases we recommend size 31/32." A second output goes to the product team: articles whose size chart deviates systematically. Monitoring keeps watching the return rate per category. The model calculation puts 61,824 avoided returns at 26 euros each, which is 1.61 million euros a year.
| Phase | Guiding question in the example | Deliverable in the example |
|---|---|---|
| 1 Business Understanding | What share of fit-related returns is avoidable? | Success criterion: return rate per category |
| 2 Data Understanding | Are size and return reason recorded per order? | Data inventory from purchase history, returns, size charts |
| 3 Data Preparation | How do customer and cut become comparable? | Size profile per customer, calibrated size run per article |
| 4 Modeling | Which size do similar customers keep? | Collaborative filtering with body-shape matching |
| 5 Evaluation | Does the model pick the size more often than the chart? | 82 percent hit rate, return rate 38 to 31 percent (model calculation) |
| 6 Deployment | How does the recommendation reach the customer? | Widget on the product page, report to the product team, monitoring |
The next step in the same series shows the second iteration of the loop: a text analysis of 210,000 free-text comments breaks "does not fit" down into five causes and, together with the size recommendation, lowers the overall return rate of the model calculation across all categories and 18 markets from 28 to around 23 percent.
Reducing return rates: the use case behind the example Analysing return reasons: what lies behind "does not fit"CRISP-DM example manufacturing: predicting machine failures, worked through in six phases
A metalworking sample company with 48 machines, six of them bottleneck machines, runs three shifts. Unplanned downtime adds up to 900 hours a year, and every hour costs 750 euros. All figures are a model calculation from our manufacturing series.
Phase 1 · Business Understanding
The business question is: What share of unplanned downtime can be avoided if maintenance knows about a failure a few days in advance? The success criterion is unplanned hours per bottleneck machine, measured before and after the launch. Phase 1 also sets the limit: wear that does not announce itself in the sensor data stays invisible to the model.
Phase 2 · Data Acquisition & Exploration
The data sit in the MES and the control system: vibration, temperature, current draw, cycle times and error codes per machine, plus the maintenance history with the actual failures. Phase 2 checks whether the failures are cleanly dated and whether the sensors record without gaps. If the failure timestamps are missing, there is no remaining useful life, only a consumption curve.
Phase 3 · Data Processing & Engineering
The second-by-second readings become window metrics per shift: trend, spread and outliers of the vibration, load changes, temperature drift. The maintenance history is cut into life segments between two failures. The model calculation sets a median of eleven days between the first signs in the data and the failure. Whether that warning time holds is decided here, on the quality of the error codes.
Phase 4 · Exploratory Analysis & Modeling
The model is a survival model that estimates the remaining useful life per machine, complemented by an LSTM on the sensor trajectories. The benchmark is the fixed calendar-based maintenance schedule, as a simple reference model before any more elaborate approach. If the AI model does not beat that schedule on warning time, the effort is not worth it.
Phase 5 · Evaluation & Validation
With a median warning time of eleven days, maintenance can schedule the intervention for low-load periods. In the model calculation, unplanned downtime hours therefore fall by 32 percent, from 900 to around 610 a year. Published industry figures range from 30 to 50 percent; the model calculation deliberately stays at the lower end. The evaluation also checks whether the warning holds on all six bottleneck machines or only on one.
Phase 6 · Deployment & Monitoring
The model runs as a maintenance alert in the maintenance system: "Machine 14 has a failure risk in nine days, recommended maintenance window: Sunday night shift." The pilot starts on one line and runs for four weeks. Monitoring keeps watching the unplanned hours per machine. The model calculation puts 288 avoided hours at 750 euros each, which is 216,000 euros a year.
| Phase | Guiding question in the example | Deliverable in the example |
|---|---|---|
| 1 Business Understanding | What share of unplanned downtime is avoidable? | Success criterion: unplanned hours per bottleneck machine |
| 2 Data Understanding | Are failures dated and sensor readings complete? | Data inventory from MES, control system, maintenance history |
| 3 Data Preparation | How do second-by-second readings become warning signals? | Window metrics per shift, life segments per machine |
| 4 Modeling | How much remaining useful life does the machine have? | Survival model plus LSTM, reference: calendar-based maintenance schedule |
| 5 Evaluation | Is the warning time long enough to act on? | Median eleven days, 32 percent fewer unplanned hours |
| 6 Deployment | How does the warning reach maintenance? | Maintenance alert in the system, pilot on one line, 216,000 euros a year |
The second use case in the same series applies the loop to quality: a scrap forecast from batch data and process parameters lowers the scrap rate of the model calculation from 4.5 to 3.4 percent, a total of 419,000 euros a year from material and rework.
Predictive maintenance: the use case behind the example Reducing scrap: the second iteration in manufacturingCRISP-DM example logistics: dwell times at the loading bay, worked through in six phases
A mid-sized carrier with 150 trucks and 43 regular customers drives around 18 million kilometres a year. At the customers' loading bays 14,700 dwell hours accumulate, and every minute costs 75 cents. All figures are a model calculation from our logistics series.
Phase 1 · Business Understanding
The business question is: How long does a truck stand at which loading bay, and which customers are slowing the fleet down? The success criterion is avoidable dwell time per customer, measured in hours and euros. Phase 1 draws the line: waiting times the customer causes can be predicted by the model but not abolished. For that, dispatch needs the figure in the customer conversation.
Phase 2 · Data Acquisition & Exploration
The data come from telematics and the order history: arrival and departure per stop, GPS traces, customer, weekday, time of day, load type, around 18,000 stops over twelve months. Phase 2 checks whether the timestamps at the loading bay are clean or whether the break in the car park beforehand is counted in. Without that separation the model measures the wrong thing.
Phase 3 · Data Processing & Engineering
Each stop becomes a record of customer attributes, time window and load, plus rolling averages of the most recent stops at the same customer. That single feature, the rolling dwell time per customer, explains around 40 percent of the variance in the model calculation. So phase 3 already decides whether the rest of the effort is worth it.
Phase 4 · Exploratory Analysis & Modeling
The model is an LSTM with attention that predicts the dwell time per stop. The benchmark is the average of the most recent stops at the customer, as a simple reference model. Only when the AI model clearly beats that reference does it justify integration into the transport management system.
Phase 5 · Evaluation & Validation
In the model calculation the model predicts the dwell time to within plus or minus twelve minutes, 25 percent more accurate than the reference at 16 minutes. Of 661,500 euros in dwell costs a year, 264,600 euros are arithmetically avoidable; the model calculation realistically counts half of that, 132,300 euros. The error also has to be stable across all 43 regular customers, not only for the large ones.
Phase 6 · Deployment & Monitoring
The model runs first as a dashboard for the twenty largest customers, then through an interface in the transport management system with live notes to dispatch: expected dwell time per stop, customers with a rising trend. Monitoring compares predicted and actual dwell time every week and flags when the error at a customer grows.
| Phase | Guiding question in the example | Deliverable in the example |
|---|---|---|
| 1 Business Understanding | Which dwell time is avoidable, and at whose site? | Success criterion: avoidable dwell hours per customer |
| 2 Data Understanding | Are arrival and departure at the loading bay cleanly recorded? | Data inventory from telematics and order history, 18,000 stops |
| 3 Data Preparation | Which features carry the prediction? | Rolling dwell time per customer explains around 40 percent of the variance |
| 4 Modeling | How long will this truck stand at this loading bay? | LSTM with attention, reference: customer average |
| 5 Evaluation | Is the model more accurate than the dispatchers' experience? | Plus or minus twelve minutes, 25 percent more accurate, 132,300 euros realisable |
| 6 Deployment | How does the figure reach dispatch? | Dashboard for top customers, then interface into the transport management system |
The second iteration in the same series applies the loop to diesel: an expectation model per truck and route finds excess consumption that does not match the load and saves 161,688 euros a year in the model calculation, from 1.5 percent of diesel and avoided consequential damage.
Dwell times at the loading bay: the use case behind the example Diesel consumption anomalies: the second iteration in logisticsCRISP-DM example real estate: predicting time on market, worked through in six phases
A mid-sized brokerage with 45 employees, 32 of them in sales, closes 450 transactions a year. A property sits on the market for 87 days on average, with a range from seven to 341 days. All figures are a model calculation from our real estate series.
Phase 1 · Business Understanding
The business question is not "What is the property worth?" but: How long will it sit on the market, and what does that mean for price and acquisition? The success criteria are additional closings per agent and the share of exclusive mandates won. Phase 1 sets the limit: the model predicts time on market, while the decision on the asking price stays with the owner.
Phase 2 · Data Acquisition & Exploration
The data sit in the CRM: 1,350 sales from three years with location, condition, property type, price deviation from market value, photos and season, plus portal statistics. Phase 2 already reveals the structure: in the university quarter a property sits for 48 days, in the western outskirts for 134, and a listing in spring sells 32 percent faster than one in December.
Phase 3 · Data Processing & Engineering
The raw data yield 18 features per property, among them the micro-location, the deviation of the asking price from market value and the month of listing. Price changes during marketing are kept as separate events; otherwise the model learns from the correction instead of from the property. If the market value basis does not hold, neither does the model.
Phase 4 · Exploratory Analysis & Modeling
The model is gradient boosting with quantile regression, so that every forecast carries a confidence interval. The benchmark is the experienced agent's gut feeling. If the AI model does not measurably beat that experience, it remains a toy.
Phase 5 · Evaluation & Validation
In the model calculation the mean error is 14 days, plus or minus 24 days at 80 percent confidence. That yields 16 additional closings a year, because marketing becomes plannable and each agent closes half a deal more, and 22 additional exclusive mandates, because the owner sees a reliable time on market in the acquisition meeting. Together that is 323,000 euros in commission a year. The evaluation checks whether the error holds in all ten districts.
Phase 6 · Deployment & Monitoring
The model runs as an owner report in the acquisition meeting and as a weekly note in the CRM: properties whose time on market exceeds the forecast. Monitoring compares forecast and actual duration every quarter and flags when a district drifts away from the model.
| Phase | Guiding question in the example | Deliverable in the example |
|---|---|---|
| 1 Business Understanding | How long will the property sit, and what follows from that? | Success criterion: closings per agent, share of exclusive mandates |
| 2 Data Understanding | Are three years of CRM history enough? | 1,350 sales with location, condition, price deviation, season |
| 3 Data Preparation | Which features explain the duration? | 18 features per property, price changes as separate events |
| 4 Modeling | How long, and how certain? | Gradient boosting with quantile regression, reference: agent's estimate |
| 5 Evaluation | Is the error small enough for the price conversation? | 14 days mean error, 38 additional closings, 323,000 euros |
| 6 Deployment | How does the forecast reach owners and agents? | Owner report, weekly note in the CRM |
The second iteration in the same series uses the duration model for pricing strategy: a price-duration curve per micro-location finds the asking price that maximises euros per day, avoids 27 of 54 price reductions a year in the model calculation and brings in an additional 89,200 euros.
Predicting time on market: the use case behind the example Optimising the asking price: the second iteration in real estateCRISP-DM example publishing: spotting subscription cancellations early, worked through in six phases
A regional media house with 25,000 digital subscriptions and 55,000 print subscriptions loses 4.5 percent of its digital subscribers a month, 13,500 cancellations a year at 10.50 euros of monthly revenue per subscription. All figures are a model calculation from our publishing series.
Phase 1 · Business Understanding
The business question is: Which cancellation can be seen before it is written, and what share of those can be retained? The success criterion is the churn rate in late churn, from the fourth subscription month onwards. Phase 1 draws the line: early churn in the first three months is an onboarding problem, not a case for a risk list.
Phase 2 · Data Acquisition & Exploration
The data sit in four systems: usage data from the portal, the subscription system, web analytics and the newsletter. Phase 2 links them through the subscriber ID and checks whether the cancellations of the last 24 months are cleanly dated. Only with that history can a model be tested against reality instead of against a pretty demo.
Phase 3 · Data Processing & Engineering
Each subscriber gets an engagement trajectory built from recency, frequency and volume: when was the last visit, how often in recent weeks, how many articles. Added to that are section mix, device and newsletter status. The cancellations become target values with 90 days of lead time. Gaps in the usage data show up here at the latest, not only in the risk list.
Phase 4 · Exploratory Analysis & Modeling
The model is gradient boosting on the engagement trajectories and delivers a cancellation probability for the next 90 days per subscriber, explained by the main drivers. The benchmark is the simple rule "no visit for 30 days", which any customer service team could apply today. If the AI model does not beat that rule, the effort is not worth it.
Phase 5 · Evaluation & Validation
In the model calculation 60 percent of cancellations fall into late churn, 8,100 a year. If 15 percent of those are retained, 1,215 subscriptions stay, on average eight months longer, 102,060 euros a year. Published figures from publishers range from 10 to 17 percent; the model calculation stays in the middle. The evaluation also checks whether the risk list hits the right subscribers or only the occasional readers.
Phase 6 · Deployment & Monitoring
Every week the model delivers a risk list to the CRM, with a reason per subscriber, so that customer service knows what to call about. The source systems stay connected read-only. Monitoring keeps watching the churn rate in late churn and flags when the list loses hit rate.
| Phase | Guiding question in the example | Deliverable in the example |
|---|---|---|
| 1 Business Understanding | Which cancellation is visible early and can be retained? | Success criterion: churn rate in late churn from month four |
| 2 Data Understanding | Can usage, subscription and newsletter be linked per subscriber? | Data inventory from four systems, 24 months of cancellation history |
| 3 Data Preparation | How does reading become a trajectory? | Engagement trajectory from recency, frequency, volume, target with 90 days of lead time |
| 4 Modeling | How likely is this subscriber to cancel? | Gradient boosting with drivers, reference: rule "no visit for 30 days" |
| 5 Evaluation | Does the list hit the right subscribers? | 1,215 retained subscriptions, 102,060 euros a year |
| 6 Deployment | How does the warning reach customer service? | Weekly risk list in the CRM with a reason per subscriber |
The second iteration in the same series turns the loop towards winning new readers: a dynamic paywall sets the barrier per reader by visit frequency and reading depth and brings 3,240 additional subscriptions a year in the model calculation, 238,140 euros.
Churn early warning: the use case behind the example Dynamic paywall: the second iteration in publishingCRISP-DM example food: forecasting seasonal volume, worked through in six phases
A mid-sized chocolate manufacturer with 140 items on four lines sells 12,000 tonnes a year, with peaks at Christmas, Easter and Valentine's Day. Write-offs on seasonal goods cost 570,000 euros, shelf gaps 1.14 million euros in revenue. All figures are a model calculation from our chocolate series.
Phase 1 · Business Understanding
The business question is: How much seasonal volume is produced before the peak so that neither write-offs nor shelf gaps arise? The success criterion is the forecast error over twelve weeks per product group, plus write-offs, shelf gaps and rush costs in euros. Phase 1 draws the line: promotions the retailers schedule at short notice cannot be anticipated by any forecast.
Phase 2 · Data Acquisition & Exploration
The data come from three sources: sell-in from the ERP, sell-out from the retail chains and the promotion calendar. Phase 2 checks whether sell-out is available weekly per chain and item and whether the seasonal figures of previous years are cleanly assigned to calendar weeks. If Easter moves, the whole curve moves.
Phase 3 · Data Processing & Engineering
The weekly values become a time series per item and chain with seasonal features: weeks to the holiday, promotion weeks, last year's trajectory. The hierarchy of product group, item and chain is prepared so that the forecasts can be reconciled later. An incomplete promotion calendar shows up here at the latest, not only in the forecast error.
Phase 4 · Exploratory Analysis & Modeling
The model is gradient boosting with quantiles, hierarchically reconciled, with a forecast over twelve weeks and an uncertainty band per item. The benchmark is planning's last-year method, as a reference before any more elaborate approach. If the AI model does not clearly beat last year, planning stays with last year.
Phase 5 · Evaluation & Validation
In the backtest the forecast error is 9.4 percent against 21.8 percent for the last-year method. From that the model calculation counts 30 percent fewer write-offs, a quarter fewer shelf gaps at contribution margin and 30 percent lower rush costs, a total of 331,500 euros a year. The error is reported separately for pralines, seasonal figures and bars, because one product group can carry the average.
Phase 6 · Deployment & Monitoring
The forecast appears every Monday with its uncertainty band and goes to production planning, which derives the production quantities per line from it. Monitoring compares forecast and sell-out every week and flags when a chain or an item runs out of the band, early enough for the next planning round.
| Phase | Guiding question in the example | Deliverable in the example |
|---|---|---|
| 1 Business Understanding | How much seasonal volume before the peak? | Success criterion: forecast error over twelve weeks, write-offs, shelf gaps |
| 2 Data Understanding | Is sell-out available weekly per chain? | Data inventory from sell-in, sell-out, promotion calendar |
| 3 Data Preparation | How does the season become a feature? | Time series per item and chain, hierarchy for reconciliation |
| 4 Modeling | How much of this item sells in twelve weeks? | Gradient boosting with quantiles, hierarchical, reference: last-year method |
| 5 Evaluation | Is the forecast better than last year? | 9.4 instead of 21.8 percent error, 331,500 euros a year |
| 6 Deployment | How does the forecast get into planning? | Monday forecast with band to production planning |
The second iteration in the same series applies the loop to shelf space: anomaly detection on the rotation per chain spots delistings before they happen and secures 108,000 euros of contribution margin a year in the model calculation.
Seasonal volume: the use case behind the example Shelf space: the second iteration in chocolate manufacturingCRISP-DM example commodities: cocoa coverage by market condition, worked through in six phases
The same chocolate manufacturer buys 3,600 tonnes of cocoa products a year at around 6,000 euros per tonne, 21.6 million euros of purchasing. The question is not where the price is heading but how many months of coverage the company should hold in which market condition. All figures are a model calculation from our chocolate series and the research articles on soft commodities.
Phase 1 · Business Understanding
The business question is explicitly not "Where will cocoa be in six months?", because nobody can say that reliably. It is: How widely do landed costs disperse if purchasing buys as needed, and how much of that dispersion does a coverage rule remove? The success criterion is the dispersion of landed costs per year. Phase 1 sets the limit: the model steers coverage, it does not trade.
Phase 2 · Data Acquisition & Exploration
The data are public: futures prices of the front contract on ICE, the euro-dollar exchange rate and the crop reports of the producing countries. Phase 2 checks the data quality of these sources and names the limit openly: public daily closing prices are enough for a situation assessment, not for a trading system. The model is built for exactly that assessment.
Phase 3 · Data Processing & Engineering
The daily prices become daily returns and rolling volatilities, the crop reports seasonal markers. The time series is cut so that on any given day the model knows only what was known at the time; otherwise it looks into the future in the backtest. That discipline decides whether the lead time later is real or an artefact.
Phase 4 · Exploratory Analysis & Modeling
The model is a regime model with three conditions, calm, elevated and strained, which outputs the probability of the strained condition for each day. The coverage rule hangs on it: three months of minimum coverage in a calm condition, five in an elevated one, hold stock in a strained one. The benchmark is buying as needed without a rule.
Phase 5 · Evaluation & Validation
In the model calculation landed costs disperse by plus or minus 14.2 percent when buying as needed and by plus or minus 8.1 percent with the coverage rule. One percent of avoided dispersion on 21.6 million euros of purchasing is 216,000 euros, deliberately not added to the series total. In the research backtest the regime model registered the 2023 cocoa shock three and a half months before the documented start of the episode, as a brief flicker rather than a sustained signal, with the open limit that there was only one such episode per commodity.
Phase 6 · Deployment & Monitoring
The model runs as a weekly situation report to purchasing: current condition, probability of a change, and from that the coverage in months. Monitoring checks every quarter whether the conditions still fit the market structure and flags false alarms before purchasing loses trust.
| Phase | Guiding question in the example | Deliverable in the example |
|---|---|---|
| 1 Business Understanding | How much dispersion does a coverage rule remove? | Success criterion: dispersion of landed costs per year |
| 2 Data Understanding | Are public prices and crop reports enough? | Futures prices, exchange rate, crop reports, limit named openly |
| 3 Data Preparation | How does the backtest stay honest? | Returns, rolling volatility, only what was known on the day |
| 4 Modeling | Which condition is the market in? | Regime model with three conditions, coverage rule 3 / 5 months / hold |
| 5 Evaluation | Do landed costs disperse less? | Plus or minus 8.1 instead of 14.2 percent, 216,000 euros per percent of dispersion |
| 6 Deployment | How does the condition reach purchasing? | Weekly situation report with coverage in months |
The second iteration in the same series adds geodata: for the EU Deforestation Regulation, growing plots are checked against forest-loss maps; of 1,800 plots, 280 remain for manual review in the model calculation, and the checking costs halve.
Cocoa coverage: the use case behind the example Regime model on soft commodities: the research article on lead timeCRISP-DM example process data: explaining energy peaks, worked through in six phases
The same sample company from the manufacturing series spends 3.2 million euros a year on energy. The controllers of its 48 machines deliver readings every second, and shop floor data collection delivers the orders and downtime reasons. All figures are a model calculation, and the assumptions are named for each phase.
Phase 1 · Business Understanding
The business question is: Which load peaks are normal for the order currently running, and which deviate from normal behaviour because a machine is badly set up or running empty alongside? The success criterion is the share of peaks explained and the kilowatt hours avoided. Phase 1 draws the line: without timestamps that connect controller and order, every peak stays unexplained.
Phase 2 · Data Acquisition & Exploration
The process data come from the controllers: current draw, temperatures, pressures and cycle times per machine. Shop floor data collection delivers order, piece count and downtime reason. They are linked through timestamp, order number and machine ID. Phase 2 finds the typical bottleneck: gaps in the time series and clocks that differ by minutes from machine to machine.
Phase 3 · Data Processing & Engineering
The second-by-second readings are condensed into minute windows and enriched per order with piece count and cycle time. This yields a normal behaviour per machine and order type against which every peak can be measured. The model calculation needs a few months of clean process data for that, whereas new sensors are rarely the first step.
Phase 4 · Exploratory Analysis & Modeling
The model is an expectation model per machine that estimates the expected current draw from order, cycle and temperature. An anomaly is the deviation from that expectation. The benchmark is the classic Tukey outlier rule, 1.5 times the interquartile range. If the AI model does not find more explainable peaks than the rule, the rule is enough.
Phase 5 · Evaluation & Validation
The evaluation checks every reported finding with the plant engineer: idling after the end of a shift, a wrongly parameterised machine, a fan running continuously. The model calculation puts no figure on this: what share of the peaks can be eliminated is shown only by the pilot on the company's own machines. The check per finding is the deliverable of this phase.
Phase 6 · Deployment & Monitoring
The model runs as a finding with an audit trail: every report names machine, time window, expected and measured current draw and the most likely reason. The judgement stays with people. Monitoring watches whether normal behaviour drifts, for instance after a tool change, and then triggers retraining.
| Phase | Guiding question in the example | Deliverable in the example |
|---|---|---|
| 1 Business Understanding | Which load peak is normal, and which is not? | Success criterion: peaks explained, kilowatt hours avoided |
| 2 Data Understanding | Can controller and order be connected? | Process data from the PLC, orders from shop floor data collection, timestamps |
| 3 Data Preparation | What is normal behaviour per machine? | Minute windows per order type, a few months of clean data |
| 4 Modeling | How far does the peak deviate from expectation? | Expectation model per machine, reference: Tukey rule |
| 5 Evaluation | Can the findings be eliminated? | Check per finding with the plant engineer, figure only in the pilot |
| 6 Deployment | How does an anomaly become an action? | Finding with audit trail, retraining on drift |
Process data are the raw material for each of the seven other cases in this section. How they are captured and linked is shown in the guide to process data capture; which methods find anomalies, in the guide to anomaly detection.
Process data capture: from machine reading to AI model (guide in German) Anomaly detection: spotting rare patterns earlyWhere Data Science Projects Fail - and in Which Phase
The published figures on failure point to the first two phases, not to modelling.
The MIT NANDA programme reported in August 2025 that 95 percent of the organisations studied see no measurable effect on results despite their spending on generative AI. A year earlier, Gartner had predicted that at least 30 percent of GenAI projects would be abandoned after the proof of concept by the end of 2025, because of poor data quality, missing risk controls, rising costs or unclear business value. In our view, unclear business value and missing risk controls are questions of phase 1, data quality a question of phase 2; the costs usually only show in operations. Our research article traces the seven methodological causes, from claim inflation in the project proposal to comparison against a weak baseline.
Why 95% of GenAI pilots failThe second finding concerns the data. In our conversations with mid-sized data teams, AI pilots rarely fail on the model and mostly on a data quality nobody checked beforehand. For that we have published an audit that checks a dataset against seven dimensions in ten minutes, with a traffic-light rating per dimension. The audit belongs in phase 2, before effort flows into phase 3.
Seven questions decide whether your AI project failsThe third finding is the simplest: without a written goal there is nothing a model could pass or fail against. The countermeasure costs little in phase 1: a written, pre-registered success endpoint that says which metric is measured on which population over which period against which comparison. Without that endpoint nothing can be validated in phase 5, and the project ends up being judged on impressions instead of numbers.
Our Data Science Approach
Three principles run through every phase of our projects.
Iterative & Agile
Short feedback loops with continuous stakeholder involvement.
Reproducible & Transparent
Documented decisions and traceable results.
Business-First
We measure success by real value delivered, not by model complexity.
Frequently Asked Questions about CRISP-DM
What does CRISP-DM mean and what does the abbreviation stand for?
CRISP-DM stands for Cross-Industry Standard Process for Data Mining. The process model was developed from 1996 by Daimler-Benz, NCR and ISL (later SPSS) and published as a guide in 2000. It describes six phases from the business question to operations and is industry-neutral and tool-independent by design.
What are the six phases of CRISP-DM?
Business Understanding, Data Understanding, Data Preparation, Modeling, Evaluation and Deployment. At myBytes they are called "Business Understanding", "Data Acquisition and Exploration", "Data Processing and Engineering", "Exploratory Analysis and Modeling", "Evaluation and Validation" and "Deployment and Monitoring". The phases run iteratively: stepping back into an earlier phase is intended, not an exception.
Is CRISP-DM still current, or are there newer process models?
Newer models such as ASUM-DM, TDSP and DASC-PM extend CRISP-DM but do not replace it. The 2000 guide treats operations and monitoring only as a task within deployment. That is exactly where we extend phase 6 with monitoring, automated release and operations handover. The six phases themselves endure.
How long does a data science project take under CRISP-DM?
There is no blanket duration, and we deliberately do not name one. The runtime depends on whether the data from phase 2 can carry the question and how much preparation phase 3 demands. Two things can be said reliably: phase 1 starts with a conversation, and a first data check against seven dimensions takes ten minutes.
Which phase takes the most time?
Data preparation in phase 3. It takes up the largest share of project work because missing values, duplicates and inconsistencies from several sources come together here. That is why we automate it with reproducible pipelines and quality checks. What that looks like in detail is on the page for phase 3: data preparation.
What is the difference between CRISP-DM and the data science lifecycle?
CRISP-DM is a concrete, documented process model with six named phases. "Data science lifecycle" is the umbrella term for any cyclical representation of a data science project; many versions are variants of CRISP-DM with a stronger emphasis on operations. Our lifecycle is CRISP-DM under our own phase names, with monitoring as part of the sixth phase.
Is there a CRISP-DM example from practice?
Yes. On this page we work through fashion returns across all six phases, from the business question to the running model, with the guiding question and the result for each phase. The use case behind it shows data, method and business case.
Can CRISP-DM also be used for AI projects with language models?
Yes, and the first two phases become more important there, not less. The published reasons for abandoning GenAI projects, unclear business value and poor data quality, are questions of phase 1 and phase 2. With language models, modelling usually means selection and integration rather than training your own; evaluation and monitoring remain mandatory as before.
The first step towards data-driven results
Every project starts with phase 1, understanding your business. That first step is a conversation, not a contract.
Every project is unique. Tell us about your challenge.
Start your free initial call