Sovereign AI, honestly priced: what on-premises really costs

Anyone deciding right now whether AI workloads belong in the cloud or on their own servers is handed two calculations, and neither of them adds up. The cloud vendor's TCO calculator forgets that even an API needs someone to operate it. The hardware vendor's on-premises quote forgets electricity, maintenance, redundancy and, above all, the people who keep the server running. We completed both sides of the ledger: a 36-month cost model built exclusively from public, dated price sources, with every assumption disclosed and no predetermined winner.

1 · Two self-interested calculations, no neutral yardstick

The sovereign-AI debate has a numbers problem. On one side sit vendor calculators that make on-premises look expensive because they credit the cloud side with nothing but the token list price. On the other side sits an on-prem romanticism that flatters its own hardware by dividing the purchase price by 36 and dropping the rest. Both sides have a business model, not a yardstick.

This article is an attempt at a yardstick. The rules: both cost sides are completed symmetrically, every figure comes from a dated public source, all assumptions are published with their ranges in the companion repository, and the sensitivity of those assumptions is computed rather than hidden. There is deliberately no "winner" - there are conditions under which each procurement path wins.

2 · Method: 110 dated price sources, four workloads, 36 months

The foundation is a price snapshot dated 7 July 2026 with 110 dated entries from public price lists: managed-API prices (OpenAI, Anthropic, Azure OpenAI, Mistral with EU hosting), cloud GPU instances (Azure H100 in the Germany region, OVHcloud H100/L40S as an EU sovereign provider; on-demand and reserved), on-premises hardware in three tiers (1× L40S entry, 2× H100 mid-range, 8× H100 large), Destatis industrial electricity and colocation list prices, plus public salary ranges for MLOps/DevOps. No vendor quotes, no scraping behind logins; anything not publicly verifiable is flagged as an assumption and varied in the sensitivity analysis.

We compute four workload scenarios over 36 months: S (internal RAG/assistant, about 1 million tokens per month, office hours), M (customer app, about 10 million tokens per month, 16/7 with moderate peaks), L (product/platform, about 100 million tokens per month, 24/7 with a peak factor of 3) and B (classic nightly ML batch scoring, no latency requirement at all). Every number in this article comes from the results file results/tco_summary.json in the companion repository; the notebook computes deterministically and exclusively from the snapshot.

3 · The honest cloud bill: idle reserve, egress and operations

The token price is not the bill. On top of the API invoice come logging, monitoring and egress, i.e. outbound data transfer (together modelled at 5 percent of API cost) - and above all operations: even a managed API needs integration, monitoring, error handling and prompt maintenance, modelled at 0.1 FTE. For small workloads this line item dwarfs everything else. In scenario S the frontier API (the most capable API model tier, priced here as OpenAI gpt-5.5) costs about EUR 794 per month - of which all of EUR 8.79 is tokens; the rest is almost entirely personnel. At low volumes, anyone trying to cut API costs is optimising the wrong end of the bill.

Self-hosted cloud GPUs carry two frequently forgotten items: the idle reserve (autoscaling keeps capacity above actual traffic, modelled as a 30 percent surcharge on active hours) and the fact that reserved instances are billed around the clock at peak size, nights included. Add 0.3 FTE for the serving stack. That puts the reserved Azure H100 in the Germany region at a constant EUR 5,669 per month - whether tokens flow or not. The EU provider OVHcloud comes in at EUR 4,363 reserved, and considerably lower on-demand, depending on the load profile.

4 · The honest on-premises bill: people cost more than power

On the on-premises side the traditional talking point is electricity - yet it is the smallest relevant item. The complete bill: straight-line depreciation of the purchase over 36 months, industrial electricity at the Destatis price with a PUE of 1.6 for an in-house server room, i.e. a 60 percent energy overhead for cooling and infrastructure (or colocation at list price), maintenance at 8 percent of the purchase price per year, N+1 redundancy (one additional standby system) wherever the availability requirement forces it - and the biggest underestimated item: 0.5 FTE of MLOps/DevOps for the serving stack, modelled at a fully loaded personnel cost of around EUR 94,000 per year.

That shifts the proportions drastically. For the entry server (1× L40S, in-house), EUR 829 of monthly depreciation faces EUR 3,924 of personnel - the human costs more than four times the machine. Electricity: EUR 121. Anyone talking about on-premises while staying silent about personnel has not done the math.

5 · Base result: on-premises wins in none of the base scenarios

The four scenarios, cheapest variant per procurement path (EUR per month, 36-month model):

The remarkable part: even in scenario L, at 100 million tokens per month, the frontier API (EUR 2,053) stays clearly below the cheapest cloud-GPU variant (EUR 3,695) and at roughly a fifth of the cost of owned hardware (EUR 10,996, including the N+1 redundancy due at that tier). The EU-hosted Mistral API costs EUR 1,117 in scenario L - at the API level, EU data residency comes almost free of charge. And the batch scenario shows the pattern in its purest form: nightly scoring in the cloud pays only for the hours actually computed (EUR 2,531), while your own server depreciates through the day as well (EUR 5,054).

We deliberately included a large system (8× H100) for scenario L as an illustration: EUR 32,941 per month, just under EUR 1.19 million over 36 months - massively over-provisioned for 100 million tokens. That is the price of buying sovereignty headroom you never use.

6 · Break-even: around 574 million tokens per month

At what volume does the picture flip? The break-even curve over token volume (workload profile M, 16/7, peak factor 2) gives the answer:

The frontier API stays cheaper than your own 2× H100 server up to around 574 million tokens per month. Against the reserved cloud GPU (Azure, Germany region) the crossing point sits at around 424 million tokens per month, and correspondingly earlier against the cheaper EU provider. Take the EU-hosted API as the reference instead, and the on-premises break-even moves out to around 2.2 billion tokens per month - the cheap API curve simply crosses the fixed-cost lines much later.

For perspective: 574 million tokens per month is more than five times our scenario L - a 24/7 product platform. The break-even does not sit where most mid-sized companies stand today; it sits where LLM inference has become the core business.

7 · Sensitivity: the three parameters the decision hinges on

Every assumption in the model was varied individually across its documented range. The tornado chart shows what actually moves the break-even - and what does not:

1. API price level. If API prices fall by another 50 percent - in line with their historical direction - the break-even doubles to around 1.15 billion tokens per month. Every on-premises decision is therefore also a bet on the future API price curve.
2. Ops personnel share. Between 0.25 and 1.0 FTE for on-premises operations, the break-even wanders between roughly 403 and 914 million tokens per month. Whether a practised ops team already exists or has to be built first is the single most expensive staffing question in the entire calculation.
3. GPU purchase price. Hardware 30 percent cheaper lowers the break-even to around 489 million tokens per month - noticeable, but weaker than the first two levers.

Just as important is what does not decide: the industrial electricity price shifts the break-even even at ±50 percent only between 564 and 583 million tokens per month - a swing of under 2 percent in each direction. And serving throughput per H100 (varied between 700 and 3,000 tokens per second) has exactly zero effect below GPU saturation, because the volume simply never fills the card. The electricity-price debate and throughput benchmarking are sideshows for this decision.

8 · Steel-manning both sides - and the sovereignty premium

An honest comparison has to price each side at its best, not at its worst.

Best case for the cloud: 2 million tokens per month, strongly fluctuating load (peak factor 4), a small team with no ops capacity, frontier quality required. Result: the API costs EUR 803 per month, the entry-level owned server EUR 5,073 - over 36 months the API is cheaper by a factor of 6.3. Here the decision is not a trade-off; it is arithmetic.

Best case for on-premises: 1.5 billion tokens per month at near-constant 24/7 load, data sovereignty mandatory (processing must physically happen on your own premises), an existing ops team runs it alongside their day job, and open-weights quality is sufficient for the use case. Result: the owned 2× H100 server costs EUR 9,035 per month including N+1 redundancy and beats the EU-hosted frontier API (EUR 19,810) by a factor of 2.2.

But - and this finding is the citable core of the model: even in its constructed best case, on-premises only beats the token APIs. The rented EU cloud GPU (OVHcloud, reserved, EUR 4,363 per month) remains cheaper even then. The difference of around EUR 4,700 per month - roughly EUR 56,000 per year - is the price of the requirement that the hardware stand in your own building rather than in a European provider's EU data centre. This sovereignty premium can be an entirely rational expense; it should just be budgeted as what it is: a paid requirement, not a savings programme.

9 · What we deliberately did not price in

Three things are intentionally not in the model as euro amounts, because any number would be false precision:

• The quality trade-off. Self-hosted systems (cloud GPU and on-premises alike) serve open-weights models; frontier APIs deliver higher quality on some tasks. Whether open weights are good enough is a suitability test against your own use case - not a cost factor one could price across the board. The model names the trade-off and leaves it open.
• The compliance and sovereignty benefit. US providers remain subject to CLOUD Act access even with EU hosting; genuine EU providers and owned hardware step that dependency down, and the AI Act's enforcement stages taking effect from August 2026 raise the documentation pressure. That is a real, qualitative benefit of the sovereign options - "valuing" it in euros would be exactly the pseudo-quantification this article criticises.
• Forecasts. We treat API providers' price-change risk (repricing, model deprecation) as a sensitivity range, not a prediction. A hardware residual value after 36 months (documented assumption: 15 percent, range up to 30) is not applied in the base calculation.

Limitations of the study: list prices rather than negotiated terms (discounts shift both sides), one serving setup per hardware tier rather than finely-tuned configurations, and personnel as FTE shares from public salary ranges. All three simplifications are documented and covered by the sensitivity analysis.

10 · The map of conditions: when each procurement path wins

Instead of a winner, the model delivers conditions:

• Below about 500 million tokens per month and without a hard on-premises mandate: managed API. Where EU data residency is required, the EU-hosted API costs barely any premium (scenario L: EUR 1,117 versus EUR 975 per month against the cheapest global variant).
• High, constant load and EU data residency is enough: rented EU cloud GPU. From mid-three-digit million volumes upwards it is the cheapest sovereign option - no capex, no hardware risk.
• On-premises is right when four conditions coincide: processing must physically happen in your own building, the load is high and constant, ops capacity already exists, and open-weights quality is sufficient for the use case. Then on-premises is a rational, consciously paid decision - carrying a sovereignty premium of around EUR 4,700 per month over the EU cloud GPU even in its best case.
• In every case: the decision hinges on three parameters - API price level, ops personnel share, GPU purchase price. If you know them for your own organisation, the companion repository recomputes the curves for your situation in minutes. The electricity price is almost never the answer.

Reproducibility

Every number in this article comes from results/tco_summary.json in the public companion repository onprem-cloud-tco-benchmark. The notebook computes deterministically and exclusively from the frozen price snapshot data/data_snapshot.json (110 dated entries as of 7 July 2026, every source with URL and retrieval date); it performs no network access. The core claims are verified following our truth-check protocol; the protocol is in the repository as a heatmap.

Sources

• Price snapshot data/data_snapshot.json in the companion repository: 110 dated entries from public price lists (managed APIs: OpenAI, Anthropic, Azure OpenAI, Mistral; cloud GPU: AWS, Azure, OVHcloud; on-premises hardware: list prices for three tiers; colocation list prices), each with source, URL and retrieval date.
• Industrial electricity price: German Federal Statistical Office (Destatis), used under Datenlizenz Deutschland - Namensnennung - 2.0.
• Fully loaded personnel cost: public salary ranges for MLOps/DevOps, modelled as FTE shares (source details in the snapshot).
• EUR/USD conversion: ECB reference rate (dated in the snapshot).