How to Self-Host Ollama on a VPS (2026 Real-World Setup)

The Ollama install command is famously simple. The complication is that the same command sets up a useful inference server on a 24 GB GPU machine and a frustrating slow one on a 2 GB VPS. The difference is in the model choice and the configuration you do after install.

Here is the setup that actually works in 2026, with real numbers from my Hetzner box.

Pre-Install Decisions

Before running the installer, decide:

1. Which model are you going to run? This drives RAM and disk needs. 7B models need 10 GB RAM, 13B need 16 GB, 70B need 48 GB+ at Q4 quantization.
2. CPU or GPU? CPU works for 7B at 5 to 7 tokens per second, acceptable for batch jobs and painful for chat. GPU is required for anything interactive at 13B+.
3. Who will access it? Personal use can stay local. Team use needs auth and probably a separate frontend like Open WebUI.

Skip the install if your hardware does not match the model. Trying to run a 70B model on 16 GB of RAM will not work, no amount of clever quantization fixes that.

Step 1: Provision the VPS

For this guide I assume a Hetzner CCX23 (4 vCPU, 16 GB RAM, NVMe). Adjust to your situation. SSH in and prep the system:

sudo apt update && sudo apt upgrade -y
sudo apt install -y curl ca-certificates

If you have a GPU VPS (Hetzner GEX, Lambda, etc.), check that the NVIDIA driver is installed:

nvidia-smi

If this fails on a GPU machine, install the driver first via your provider’s instructions. Ollama will not use the GPU if the driver is missing, and the silent fallback to CPU is the most common source of “why is this so slow” questions.

Step 2: Install Ollama

The official installer:

curl -fsSL https://ollama.com/install.sh | sh

This installs the binary to /usr/local/bin/ollama and sets up a systemd service running as the ollama user. By default, Ollama binds to 127.0.0.1:11434 (localhost only), which is what you want for security.

Verify the service:

sudo systemctl status ollama

Should show active (running).

Step 3: Move Model Storage to a Dedicated Path

Models are big. The default location /usr/share/ollama shares root disk space, which usually has the least room. Point Ollama at a path on your data disk:

sudo systemctl stop ollama
sudo mkdir -p /var/ollama/models
sudo chown -R ollama:ollama /var/ollama

Edit the systemd override:

sudo systemctl edit ollama

Add:

[Service]
Environment="OLLAMA_MODELS=/var/ollama/models"
Environment="OLLAMA_KEEP_ALIVE=10m"
Environment="OLLAMA_NUM_PARALLEL=2"

Reload and restart:

sudo systemctl daemon-reload
sudo systemctl start ollama

The OLLAMA_KEEP_ALIVE of 10 minutes balances memory pressure with responsiveness. The default 5 minutes means cold-start latency on every gap in usage. The OLLAMA_NUM_PARALLEL of 2 lets two concurrent requests share the loaded model, useful for team or multi-app setups.

Step 4: Pull Your First Model

For a 16 GB VPS without GPU, start with Llama 3.1 8B at Q4 quantization:

ollama pull llama3.1:8b-instruct-q4_K_M

Download is around 4.6 GB. The first run loads it into RAM, which takes 15 to 30 seconds. Test it:

ollama run llama3.1:8b-instruct-q4_K_M "Explain what self-hosting means in one paragraph."

You should get a coherent response. On a CCX23 CPU-only, expect 5 to 7 tokens per second. On a GPU machine, 50 to 100+ tokens per second depending on the GPU.

Step 5: Expose Ollama to Other Apps (Safely)

For local apps on the same VPS (Open WebUI, Flowise, LangGraph), 127.0.0.1:11434 is what they connect to. Nothing more to do.

For apps on other machines, bind Ollama to a private network and put auth in front. The pattern I use:

1. Tailscale on the VPS and on the consuming machines
2. Bind Ollama to the Tailscale interface

Edit the systemd override again:

sudo systemctl edit ollama

Add (replace 100.x.x.x with your Tailscale IP):

[Service]
Environment="OLLAMA_HOST=100.x.x.x:11434"

Reload and restart. Now other Tailscale-connected machines can hit your Ollama endpoint without exposing it to the public internet.

For public exposure (only if you really need it), put Caddy or Nginx in front with token authentication. Never expose Ollama on 0.0.0.0:11434 directly, it has no built-in auth and you will get crypto miners using your VPS within hours.

Step 6: Monitor Resource Usage

Watch RAM usage as the model loads and runs:

watch -n 2 'free -h && echo "" && systemctl status ollama | grep Memory'

The 8B Q4 model should sit at around 5 to 6 GB of resident memory. If you see Ollama using significantly more, the context window might be set higher than you expect. Check with:

ollama show llama3.1:8b-instruct-q4_K_M

What Comes Next

With Ollama running, the question becomes what to put in front of it:

• Open WebUI for ChatGPT-like multi-user access
• AnythingLLM for document Q&A workspaces
• Flowise for visual workflow building
• Cline for IDE-integrated coding assistance
• SmolAgents or CrewAI for agent frameworks

For VPS comparisons across these tools, see the SelfHostVPS guides. For the realistic GPU vs CPU math on Ollama specifically, the best VPS for Ollama page walks through the token-per-second numbers I measured across hardware tiers.