Best VPS for PrivateGPT (2026): Specs for Truly Private Doc Q&A

PrivateGPT is the project for people who do not want any document touching an external API. It runs the model locally, indexes your corpus locally, and serves a private chat. That mission means the VPS sizing question becomes the model sizing question.

I tested PrivateGPT in four configurations to write this. CPU only on a Contabo box, CPU on Hetzner CCX, GPU on Hetzner GEX, and the BYO model path where PrivateGPT becomes light again.

What PrivateGPT Actually Needs

Two scenarios:

1. Local model. Llama.cpp running a quantized GGUF model, plus embeddings, plus vector store. 8 GB RAM minimum for a usable 3B model on CPU, 16 GB for a 7B, GPU strongly recommended above that.
2. Remote model. PrivateGPT becomes orchestration only. 4 GB RAM holds it fine.

The thing the README hides: even on a GPU, the vector store retrieval is CPU bound. A weak CPU with a fast GPU still feels slow on retrieval heavy questions.

VPS Comparison for PrivateGPT

Hetzner Cloud CCX23: CPU local mode default

If you want truly local inference without renting a GPU, CCX23 with 16 GB RAM and 4 dedicated vCPU is the realistic floor. A quantized 7B model runs at 1 to 3 tokens per second on this hardware, which is slow but usable for short questions. NVMe makes the embeddings retrieval snappy.

Pros for this setup:

• 16 GB holds a 7B model and the vector store comfortably
• Falkenstein latency is irrelevant since the model is local
• Snapshots let you roll back model swaps

Real downside: even with a 7B model, CPU inference makes long answers feel slow. Plan for GPU if your team uses it heavily.

Get Hetzner: Hetzner Cloud.

Contabo VPS M: Budget CPU local mode

6 vCPU and 16 GB at 8.50 EUR a month is the cheapest path to a workable local PrivateGPT install. The extra vCPU helps CPU inference a bit. NVMe disk holds the model files and vector store without complaints.

The trade offs:

• Provisioning takes hours
• The shared infrastructure on cheaper tiers can show variable inference speed
• Outbound is irrelevant in local mode

Get Contabo: Contabo VPS.

Hetzner GPU GEX44: Real local quality

The moment you want a 13B or larger model running locally with usable response time, you need GPU. The GEX44 with an RTX 6000 at 184 EUR a month is the cheapest serious option in Europe. 48 GB VRAM is plenty for a 13B at full precision or a 70B at 4 bit quant.

This is overkill for most personal users. Pick it if your team uses PrivateGPT seriously and you have a compliance reason that rules out external APIs.

Get Hetzner GPU: Hetzner GPU.

Hetzner Cloud CCX13: BYO model path

If you give up on local inference and point PrivateGPT at an external model API, the local install becomes light again. CCX13 holds the orchestration plus the embeddings plus the vector store comfortably. You lose the privacy story for inference but keep it for retrieval.

Get Hetzner: Hetzner Cloud.

Things Worth Knowing

Three lessons from setup:

1. Quantization matters. A Q4_K_M GGUF runs at maybe half the quality of a Q6_K_M but two thirds the RAM. Test both before deciding.
2. Embeddings model choice impacts disk and quality. The default is fine. A larger embedding model doubles the index size on disk and in RAM.
3. GPU drivers add operational overhead. CUDA versions matter, and the GEX44 image needs the right NVIDIA driver. Plan for that the first day.

What I would actually pick

If you are starting today:

• EU, CPU local, daily use: Hetzner CCX23
• EU, CPU local, lowest spend: Contabo VPS M
• EU, GPU local, real production: Hetzner GEX44
• BYO model, lighter setup: Hetzner CCX13

For the broader self hosting picture, see the SelfHostVPS comparison. PrivateGPT moves at its own pace and I refresh this page when meaningful model integration or RAG changes land.