Ollama and Open WebUI together have become the default stack for teams and individuals who want a genuinely self-hosted AI interface — a ChatGPT-style chat experience running entirely on infrastructure they control, with no inference request ever leaving the network. Getting it running on a laptop for personal use takes minutes. Running it reliably for a team, with real concurrent usage, is a different infrastructure problem entirely — which is exactly where Open WebUI hosting decisions start to matter.
This guide covers what production Ollama hosting actually requires — GPU and VRAM sizing, storage for model files, security considerations, and where a laptop setup stops being adequate. If you’re also considering retrieval-augmented generation on top of this stack, it’s worth pairing this with our guide to vector databases and hosting requirements for AI search applications, since Open WebUI’s built-in RAG features depend on the same infrastructure fundamentals.
What Ollama and Open WebUI Actually Do
- Ollama is the inference layer — it downloads, quantizes, and serves open-weight language models locally, exposing a REST API that other tools (including Open WebUI) talk to.
- Open WebUI is the interface layer — a full-featured web chat UI (multi-user accounts, chat history, document upload for RAG, model switching) that talks to Ollama’s API rather than replacing it.
Together, this is what makes a genuine local AI server possible: unlike hosted assistants that always call out to a third-party API, this stack keeps every prompt, response, and uploaded document entirely within infrastructure you control — the core value proposition for a private AI chatbot deployment.
GPU and VRAM Sizing: The First Real Decision
Model size and quantization level determine VRAM requirements more than almost anything else in this stack:
- Smaller models (7B–8B parameters, quantized) — can run comfortably on consumer-grade GPUs with 8–16GB of VRAM, making them a reasonable starting point for individual use or light team usage.
- Mid-size models (13B–34B parameters) — generally need 24GB+ of VRAM at reasonable quantization levels, moving into prosumer or entry data-center GPU territory.
- Large models (70B+ parameters) — require either a high-VRAM data center GPU or multi-GPU setups with model sharding across cards, since a single consumer GPU’s VRAM capacity is rarely sufficient.
- Quantization tradeoffs — lower-bit quantization (e.g., 4-bit vs 16-bit) dramatically reduces VRAM requirements at some cost to output quality; most self-hosted deployments land on a quantization level that balances quality against available hardware rather than always running full precision.
The practical takeaway: VRAM, not system RAM or CPU core count, is almost always the binding constraint on which models a given Ollama dedicated server can actually run.
Concurrent Users Change the Math Entirely
A model that runs comfortably for one person generating one response at a time behaves very differently under simultaneous requests from a whole team:
- Sequential vs concurrent request handling — Ollama can queue concurrent requests, but each one still competes for the same GPU compute and memory bandwidth; response times degrade as concurrent load increases unless the hardware has real headroom.
- Context window memory overhead — every active conversation with a long context window consumes additional VRAM beyond the base model weights; a deployment supporting many simultaneous long conversations needs meaningfully more headroom than the model’s baseline requirement suggests.
- Sizing for peak, not average, usage — a team of 20 people rarely all prompt simultaneously, but sizing hardware only for average load leads to a frustrating experience exactly when usage is highest — during a shared demo, a busy morning, or heavy collaborative use.
Storage Requirements for Model Files
Model weight files are large — often tens of gigabytes each — and a team experimenting with multiple models accumulates storage needs quickly:
- Fast storage cuts load time meaningfully — loading a large model from disk into GPU memory is I/O-intensive; NVMe storage noticeably reduces cold-start and model-switch latency compared to slower storage, a point covered in depth in our guide to why NVMe storage is essential for modern AI and database workloads.
- Multiple models multiply storage needs — teams that keep several models available for different tasks (a coding model, a general chat model, an embedding model for RAG) need proportionally more disk space than a single-model deployment.
- Document storage for RAG — Open WebUI’s document upload and retrieval features store both the source documents and their vector embeddings, adding to storage requirements beyond the model files themselves.
Deployment: Docker and Container Considerations
Both Ollama and Open WebUI are commonly deployed via Docker, which simplifies updates and isolates dependencies but introduces its own considerations:
- GPU passthrough to containers — running Ollama in Docker requires the NVIDIA Container Toolkit (or equivalent for other GPU vendors) configured correctly so the container can actually access the host’s GPU rather than falling back to slow CPU inference.
- Persistent volumes for models and data — model files, chat history, and uploaded documents need to be mounted on persistent volumes rather than living inside an ephemeral container, or an update/restart wipes everything.
- Orchestration for multi-container setups — teams running Ollama, Open WebUI, and a separate vector database together benefit from the same orchestration decisions covered in our comparison of Kubernetes vs Docker Swarm for dedicated servers.
Security: Don’t Expose the Ollama API Directly
A common and avoidable mistake is exposing Ollama’s raw API port directly to the internet. Ollama’s API has no built-in authentication of its own — Open WebUI is meant to sit in front of it and handle user accounts, authentication, and access control. Practical security steps for an Open WebUI setup:
- Keep Ollama’s API bound to localhost or an internal network — only Open WebUI (or other trusted internal services) should be able to reach it directly; it should never be reachable from the public internet.
- Put a reverse proxy in front of Open WebUI — terminate TLS and handle access control at the proxy layer before traffic reaches the application itself.
- Enable Open WebUI’s authentication — user accounts and access controls should be configured deliberately, not left on permissive defaults, particularly for any deployment reachable outside a trusted local network.
- Treat this like any other production service — the same DDoS and network protection considerations covered in how DDoS attacks affect business websites and how dedicated servers help apply directly to an internet-reachable self-hosted AI deployment.
Why Self-Host Instead of Using a Hosted API?
Self-hosting isn’t automatically the right call for every team — it’s a real infrastructure and maintenance commitment. It’s the right call specifically when:
- Data sensitivity or compliance requirements mean prompts and documents genuinely cannot leave your own infrastructure.
- Sustained, high-volume usage makes fixed hardware cost more predictable than metered per-token API billing — the same fixed-cost-versus-metered reasoning covered in our bare metal servers vs cloud VMs comparison.
- Offline or air-gapped operation is a hard requirement, which no cloud-routed assistant can satisfy regardless of enterprise privacy commitments.
It’s a weaker case for occasional or low-volume use, where a hosted API’s per-request pricing is likely cheaper than dedicating a GPU server that sits mostly idle.
How BeStarHost Supports Self-Hosted LLM Infrastructure
Running Ollama and Open WebUI reliably for real usage comes down to having genuine, unshared compute and storage available — not just enough to load a model once, but enough to serve it under real concurrent load:
- Dedicated servers with guaranteed, unshared CPU and RAM, so inference performance isn’t degraded by another tenant’s workload.
- NVMe storage across server tiers, keeping model loading and switching fast even with multiple large models available.
- Dedicated, unshared bandwidth on a global low-latency network, relevant when a private AI chatbot is accessed by a distributed team.
- 99.9% uptime on Tier 3 / Tier 4 hardware with RAID 0 / RAID 1 configurations.
- IPMI KVM-over-IP for direct remote access when configuring GPU passthrough or troubleshooting a deployment.
- 14 global data center locations across Europe (France, Germany, Netherlands, United Kingdom), Asia (Singapore, Hong Kong, India, South Korea, Taiwan, Philippines, Myanmar, Cambodia), and North America (United States, Canada) — letting you place your self-hosted AI infrastructure close to your team.
- No setup fees and 24/7/365 support if you need help sizing GPU infrastructure for your model and expected concurrent usage.
Explore our dedicated server plans, read more on our About Us page, or contact our team to scope infrastructure for a self-hosted Ollama and Open WebUI deployment.
Frequently Asked Questions
How much VRAM do I need to self-host an LLM with Ollama?
It depends on model size and quantization level. Smaller 7B–8B parameter models quantized to 4-bit can run on 8–16GB of VRAM, while 13B–34B parameter models generally need 24GB or more, and 70B+ parameter models typically require a high-VRAM data center GPU or multiple GPUs with model sharding.
Is it safe to expose Ollama’s API directly to the internet?
No. Ollama’s API has no built-in authentication of its own and should be kept bound to localhost or an internal network. Open WebUI is designed to sit in front of it, handling user authentication and access control, and should be the only externally reachable component, ideally behind a reverse proxy.
Can Open WebUI support multiple users at the same time?
Yes, Open WebUI supports multi-user accounts and chat history, but concurrent usage is ultimately limited by the underlying GPU’s available VRAM and compute. Response times will degrade under simultaneous heavy usage unless the hardware has enough headroom for real concurrent load, not just the baseline requirement of a single active conversation.
Does self-hosting Ollama and Open WebUI actually save money compared to a hosted AI API?
It depends on usage volume. For sustained, high-volume usage, a fixed-cost dedicated GPU server is often more predictable and eventually cheaper than metered per-token API billing. For occasional or low-volume use, a hosted API is usually cheaper than dedicating hardware that sits mostly idle.
What’s the difference between Ollama and Open WebUI?
Ollama is the inference engine that downloads, quantizes, and serves language models, exposing a REST API. Open WebUI is a separate web-based chat interface that talks to Ollama’s API, adding features like multi-user accounts, chat history, and document upload for retrieval-augmented generation. They’re deployed together but serve distinct roles in the stack.
Sizing infrastructure for a self-hosted AI chatbot or team deployment? Talk to BeStarHost about dedicated servers built for Ollama and Open WebUI hosting →