EU/GDPR-friendly AI on Hetzner: a practical setup for B2B SaaS

The problem with most AI infrastructure advice

The default AI stack everyone recommends assumes OpenAI for inference, Pinecone for vectors, and maybe Langsmith for tracing. All of it runs on US cloud infrastructure, all of it processes your users' queries through US-based services, and none of that is straightforwardly compatible with GDPR if those queries contain personal data — which support queries often do.

I've had the same conversation with several EU-based B2B SaaS founders: they want to add an AI feature, they're not sure what GDPR actually requires for AI, and they've heard "just put it on AWS eu-west" as a fix that isn't really a fix. This post is my attempt to lay out what you actually need to think about, and what a realistic setup looks like when EU data residency is a hard requirement.

A caveat upfront: I'm an engineer, not a lawyer. Nothing here is legal advice. What I can tell you is the technical side — where data goes, which services process it, and what controls you can put in place. For your specific situation, a GDPR-specialist lawyer is worth an hour of their time.

What GDPR actually requires for AI features

GDPR doesn't say you can't use AI. It says that when you process personal data, you need a lawful basis, you need to be transparent about what you're doing, and you need to have appropriate technical safeguards. The safeguards part is where AI infrastructure gets complicated.

The key question is: does the data you're sending to your AI feature contain personal data? For a support assistant that processes user queries, the answer is almost certainly yes. A user asking about their account, their billing, their configuration — that query contains at minimum an identifier, and often more. When you send that query to an external LLM API, you're transferring personal data to a data processor. That processor needs to be covered by a DPA (Data Processing Agreement), and if they're outside the EU/EEA, you're into the territory of international data transfers.

OpenAI does offer DPAs and has a zero-data-retention option on the API (when enabled, they claim not to store or train on your inputs). Whether that's sufficient depends on your read of the regulation and your customers' expectations. For some B2B contexts it's enough. For others — healthcare-adjacent, financial services, anything where customers have negotiated specific data handling terms — it isn't.

The cleanest solution from a GDPR standpoint is inference on EU-based infrastructure that you control. That's where Hetzner comes in.

Why Hetzner specifically

Hetzner is a German cloud provider with data centres in Nuremberg, Falkenstein, and Helsinki. Everything stays in the EU, the pricing is about a third of AWS or GCP for comparable compute, and they offer bare-metal servers alongside cloud VMs. For GPU workloads specifically — running inference on open-weight models — the bare-metal GPU servers are relevant.

The trade-off is operational: you get less managed infrastructure. No fully managed Kubernetes, no managed databases with 5-nines SLAs, no auto-scaling that just works out of the box. You manage more. For a small B2B SaaS team that wants EU residency, the trade-off is usually worth it for the parts of the stack that touch personal data, while using managed services for everything else.

Self-hosting inference on Hetzner is practical if your query volume justifies always-on hardware. I've built AI features this way for clients who needed strict EU residency as a contract requirement.

What to self-host on Hetzner vs what to keep managed

You don't need to self-host everything. The question is which components touch personal data — specifically, which components process the actual user queries. That's the inference layer and the vector store. Everything else can stay managed if it's easier.

Inference — self-host. For EU data residency, you want inference running on your own infrastructure. The practical options: Ollama for smaller models and development, vLLM for production serving of larger open-weight models. The hardware question matters here. A Hetzner AX101 (an AMD Epyc bare-metal with 128GB RAM) can run quantised 7B–13B models for a support use case at reasonable latency. For anything larger, you're looking at GPU instances — Hetzner has limited GPU availability, and for production serving of 70B+ models you may need to plan around GPU waitlists or use a different EU provider (OVHcloud has more GPU capacity). For most B2B support use cases, a well-tuned 13B model outperforms what you'd expect, so don't assume you need the largest option.

Embeddings — self-host or use EU-hosted managed. If you're doing RAG (retrieval-augmented generation), you need to embed your documents and embed user queries at search time. Running a local embedding model (bge-m3 or similar) on Hetzner keeps everything in-region and adds minimal latency. Alternatively, Mistral AI offers embedding endpoints from EU-based infrastructure — this is worth knowing if you want a managed option that stays in the EU.

Vector store — self-host on Hetzner. Pgvector on a Postgres instance running in Hetzner is the simplest EU-resident option and it's what I use. You can also run Qdrant or Weaviate yourself if you want a dedicated vector store. The managed vector store options (Pinecone, Weaviate Cloud) are US-based, so they're out if you need strict residency.

Application layer, databases, queues — managed is fine. Your Next.js app, your main Postgres database, your email service — these don't need to be on Hetzner specifically. A Vercel deployment with EU edge functions, or a Fly.io deployment in an EU region, keeps your application in the EU without the operational overhead of managing it yourself. Supabase has an EU hosting option. The key distinction: your application layer probably doesn't process raw user AI queries — it orchestrates the request to the inference layer, which does the actual processing.

Handling GDPR's logging obligations without breaking things

One of the more awkward parts of GDPR for AI features is the tension between good MLOps practice and privacy requirements. Good MLOps means you log everything: inputs, outputs, latencies, errors. You need those logs to debug problems, to improve your models, and to build evals. GDPR means you can't keep personal data longer than necessary, you need to be transparent about it, and you shouldn't use it for purposes the user didn't consent to (like training your models).

These aren't irreconcilable, but they require deliberate decisions:

Separate operational logs from content logs. Latency, error rates, retrieval scores, model version — none of that contains personal data. Log it freely, keep it as long as useful. User query text and response text contain personal data. Keep them separately, with different retention policies and access controls.

Define retention explicitly and enforce it. For query/response content, a 30-day rolling window is often defensible as "necessary for debugging and quality improvement." That's something you can articulate to a customer asking about your data handling. 90 days or indefinite gets harder to justify. Whatever you decide, automate the deletion — a policy that requires manual intervention to enforce is a policy that won't be enforced.

Pseudonymisation helps. Before storing a query/response pair for debugging, replace the user identifier with a hash. You lose the ability to trace a specific user's history (which you probably don't need for debugging), and you reduce the personal data in your logs without losing the debugging utility. When a user requests deletion, you can argue the pseudonymised data is no longer linkable to them — though the legal weight of this depends on whether you can actually re-identify.

Be explicit about training data. If you fine-tune models on user queries, you need explicit consent. If you're not fine-tuning on user queries — you're using a fixed open-weight model and only personalising via retrieval — this is simpler. The RAG approach sidesteps a lot of the training data issues because the model itself doesn't change; only what you retrieve for context does.

The cost and latency reality

Self-hosting inference on Hetzner is cheaper than OpenAI at scale and more expensive than OpenAI at low volume, because you're paying for the server whether or not it's processing queries. The breakeven depends on your query volume and the model you're running. At typical B2B SaaS support volumes, here's the math: a Hetzner bare-metal server capable of serving a 13B model at reasonable latency costs around €200–300/month. OpenAI's GPT-4o-mini runs at €0.15 per million input tokens and €0.60 per million output tokens. For an average support query of 500 input + 200 output tokens, that's roughly €0.00018 per query. You break even at around 1.2 million queries per month (about 40,000 per day). Below that volume, OpenAI's API is more cost-effective. Above it, Hetzner is cheaper.

The latency picture is more nuanced. On consumer-grade laptop hardware without GPU acceleration, inference is slow — 30–90 seconds for a full response on a 13B model. But on Hetzner's bare-metal infrastructure, specifically an AX101 with a well-quantised 13B model, you get 2–5 seconds for a typical support query. That's comparable to, or slightly worse than, OpenAI. For most support use cases, 2–5 seconds is acceptable.

Streaming helps a lot here. If you stream the response token by token — which Ollama and vLLM both support — users see the first token in under a second even when full completion takes 4–5 seconds. The latency is there, but it's invisible. Almost all modern AI UIs use streaming for this reason.

What this doesn't solve

EU data residency handles where your data is processed. It doesn't automatically handle everything GDPR requires. A few things that need separate consideration:

Your users still need to know you're processing their queries with AI. A sentence in your privacy policy and a visible indicator in the UI ("AI-powered, responses may vary") is the minimum. If you're in a high-risk use case — anything involving sensitive categories of personal data — you may need a full DPIA (Data Protection Impact Assessment).

Your DPA with any sub-processors still matters, even EU-based ones. Hetzner has a standard DPA available; make sure you've signed it and reviewed it. If you're using Mistral for embeddings, same thing.

And the hardest part: if your B2B customers have EU-based end users, and those customers' contracts with you specify data handling requirements, those requirements might be stricter than the regulatory minimum. Enterprise sales often involves security reviews where "we self-host on Hetzner" is a much cleaner answer than "we use OpenAI with a DPA."

The practical recommendation

For most B2B SaaS teams adding an AI feature with EU data residency as a requirement, the setup I'd recommend:

Start with a managed EU deployment using Mistral AI (French company, EU-based inference, GDPR DPAs available). It's the fastest path to a compliant, production-ready AI feature without self-hosting overhead. You're trusting Mistral as a data processor, which is defensible for most use cases.

If your customers need infrastructure you control — either because they've asked for it explicitly or because you're in a sector where that matters — move the inference layer to Hetzner. Keep your application layer managed wherever makes sense for your team.

Either way, get the logging architecture right from the start. Retrofitting data retention policies into a system that logs everything is painful. Build in the separation between operational metrics and content logs on day one.

If you're not sure whether your use case needs EU residency, the honest answer is: talk to your customers before you build. The founders who've struggled here are the ones who built on OpenAI, shipped to enterprise customers, and then had to re-architect when procurement asked where the data goes. That conversation is much easier before you've built the thing.

If this is something you're working through right now — figuring out what "GDPR-compliant AI" actually means for your product — I'm happy to talk through the specifics. I offer a free 15-minute scoping call for exactly these conversations: book one here. No pitch, just a clear-eyed look at your situation and what the options are.