The hosted AI services are genuinely good products. But they come with constraints that matter more as you move from personal use to business use: data privacy, usage limits, and cost scaling. A VPS-hosted AI gives you a private instance that you control completely, with no per-message costs and no data leaving your server. For power users and small businesses past the experimental stage, the economics and control benefits are significant.
What You Actually Need
The minimum viable setup for running a useful AI model on a VPS is: a VPS with at least 16GB RAM (24GB is better for larger models), an NVIDIA GPU with at least 6GB VRAM, Ubuntu 22.04, and Ollama as the model deployment layer. Ollama handles model downloads, caching, memory management, and API exposure with a single command-line interface. You do not need to understand Docker, CUDA configuration, or API routing to get a working model running.
The total monthly cost from providers like Hetzner, Lambda Labs, or Vast.ai is £8-20 per month depending on GPU tier. This is less than one month of ChatGPT Plus and gives you a completely private instance with no usage limits, no rate limiting, and no context window restrictions beyond what the model itself supports.
The Model Choice Is the Most Important Decision
Smaller models — Llama 3.1 8B, Mistral 7B — run on GPUs with 6-8GB of VRAM and handle most business tasks well: email drafting, document summarisation, research synthesis, code review. Larger models — Llama 3.1 70B, Mixtral 8x22B — require 16-24GB VRAM and are noticeably slower but produce meaningfully better outputs for complex reasoning. Start with smaller models, measure the output quality for your specific use cases, and move up only if needed.
What Nobody Tells You About Setup
The tutorials skip the frustrating parts. The most common failure point is NVIDIA driver installation on a cloud GPU — most VPS providers do not have the correct driver version pre-installed, and the first boot loads a generic open-source driver that does not support GPU acceleration. You need to install the NVIDIA driver from their repo, verify it with nvidia-smi, install the CUDA toolkit, and then install Ollama. Each step has a non-obvious requirement that will stop you unprepared.
The second most common failure is GPU passthrough configuration. Some providers — particularly Hetzner — require specific kernel flags and boot parameters to expose the GPU correctly. These are documented in the provider’s knowledgebase but not in the Ollama tutorials. Budget an hour or two for this step specifically when using a new provider for the first time.
The Privacy Argument Is Real
Running your own AI means your data never leaves your infrastructure. For businesses handling client data, proprietary information, or anything that creates a GDPR or NDA obligation, this is not an abstract concern. Self-hosting removes the data processing relationship that creates the compliance obligation in the first place. Your data is yours, on your server, under your control.
Want a complete step-by-step guide to setting this up? I wrote a guide covering every step — VPS choice, driver installation, Ollama setup, and operational monitoring. Find it here.
The Ollama API: Using Your Private AI From Any Application
Once Ollama is running on your VPS, it exposes a REST API that any application can use. The API is simple: you send a POST request with a JSON body containing the model name and a prompt, and you receive a JSON response with the model’s completion. This means your entire workflow can route through your private AI instance — your text editor, your note-taking app, your browser, your CLI tools — all using the same private model that runs on your server.
The integration options are extensive. VS Code has extensions that connect to Ollama for code completion and pair programming. Obsidian and Logseq have plugins for AI-assisted note-taking and summarisation. Chrome extensions exist that route web queries through an Ollama instance. The practical limit is not the technology — it is your creativity in identifying where AI assistance adds value in your specific workflow. Every tool you use that currently sends data to an external AI API can instead send data to your private Ollama instance.
Keeping It Running: Systemd and Uptime Monitoring
The operational detail that most tutorials skip is making sure your AI stays running. If you set up Ollama by running the start command in a terminal session, it will stop when that session ends. The correct approach is to run Ollama as a systemd service that starts on boot and restarts automatically if it crashes. This requires writing a systemd unit file and enabling the service — a 15-minute task that will save you from waking up to a down AI at a critical moment.
For monitoring, a simple uptime check that pings the Ollama API endpoint and sends an alert if it is not responding will catch most failures before you get a complaint. Combined with the systemd restart configuration, this gives you a private AI that runs reliably without requiring manual intervention. The total operational overhead for a self-hosted Ollama instance, once properly configured, is approximately 15-30 minutes per month of maintenance time — less than most SaaS subscriptions require for configuration and management.
Ready to support your health? Browse supplements on Gumroad — buy now from £8.
Leave a Reply