Orchestrating AI: Open WebUI + LiteLLM for the Ultimate Home Lab Assistant

Stop tab-switching between AI providers. By combining Open WebUI with LiteLLM, you can build a unified, self-hosted dashboard that routes your requests to OpenAI, Anthropic, Google, and your local Ollama models from one interface. This guide covers setting up the "Swiss Army Knife" of AI stacks, getting your API keys, and creating custom agent personalities for every task.

I currently have 4 browser tabs open. One for ChatGPT (the coding workhorse), one for Claude (the reasoning king), one for Gemini (the massive context window), and one for my local Llama 3 instance. I'm wasting half my day copying and pasting prompts between UIs just to see which model gives the best answer.

I'll be honest—I was tired of it. I wanted a single local UI for everything. A place where my chat history lives in one database, and I can switch models with a dropdown instead of a Ctrl+Tab. That's when I found the Open WebUI + LiteLLM stack.

What is the "Swiss Army Knife" Stack?

Think of this setup in two parts: the Face and the Brain.

Open WebUI is the Face. It's a beautiful, local-first interface that looks and feels like ChatGPT but runs entirely on your hardware. It handles the users, the chat history, and the documents you want to search through (RAG).

LiteLLM is the Brain. It sits behind the scenes and acts as a universal translator. It takes 100+ different APIs—from Anthropic to Google—and makes them all look like a single OpenAI-compatible endpoint. This is the magic bridge that allows Open WebUI to talk to any model on the planet.

Why bother? Because using official web apps means your data is siloed. With this stack, you own the history, you manage the privacy, and you have a unified dashboard for every AI "employee" in your lab.

Phase 1: Getting Your Keys

Before we build the machine, we need the fuel. Here is how to grab your API keys for the "Big Three" in 2026.

1. Google Gemini (The Generous One)

Google is still the most beginner-friendly. Their free tier is massive and usually doesn't require a credit card to start testing. Head over to aistudio.google.com, sign in, and click "Get API Key." Copy it and keep it safe.

2. OpenAI (The Prepaid One)

OpenAI uses a prepaid model now. You'll need to visit platform.openai.com, create a Project (highly recommended for security), and add at least $5 in credits. Once your balance is above zero, you can generate a secret key under "API Keys."

3. Anthropic Claude (The Precision One)

Anthropic's console at console.anthropic.com requires phone verification to prevent bot abuse. Like OpenAI, they use a credit system. Add your credits, generate your key, and you're ready for the reasoning power of Claude 4.

Phase 2: The Setup (Docker)

The easiest way to run this is via Docker. I've found that keeping LiteLLM on the host or in its own container works best. Here is the configuration I use for my lab.

LiteLLM Configuration

Create a config.yaml file. This is where the routing magic happens. You'll notice I'm using environment variables to keep my keys secret.

model_list:
  - model_name: smart-model
    litellm_params:
      model: anthropic/claude-sonnet-4-6
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: smart-model
    litellm_params:
      model: openai/gpt-5
      api_key: os.environ/OPENAI_API_KEY
  - model_name: smart-model
    litellm_params:
      model: gemini/gemini-2.5-pro
      api_key: os.environ/GEMINI_API_KEY

Pro Tip: The os.environ/ prefix is a LiteLLM-specific feature. It tells the application to pull your API keys from the container's environment variables at runtime, keeping your secrets out of your config file and safely in your Docker environment.

Starting Open WebUI

Now, run the Open WebUI container. We need to tell it to look for LiteLLM on our host machine.

# Start Open WebUI and connect to host LiteLLM
docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  -e OPENAI_API_BASE_URL=http://host.docker.internal:4000/v1 \
  -e OPENAI_API_KEY=sk-1234 \
  --name open-webui \
  ghcr.io/open-webui/open-webui:main

Open your browser to http://localhost:3000 and you're in.

Phase 3: Intelligent Routing

Remember that smart-model alias in the config? This is where it gets powerful. Instead of switching models, you can set up LiteLLM to handle the routing for you.

You can define routing strategies like simple-shuffle for load balancing or use Fallbacks. If OpenAI is having a bad day and returns a 500 error, LiteLLM will automatically retry the request with Anthropic or Google without you ever seeing an error message in the UI.

It's like having a reliable load balancer for your intelligence.

Phase 4: Hiring Your Agents (Personalities)

Now for my favorite part: the Workspace. In Open WebUI, you don't just talk to raw models; you hire "Agents" via Modelfiles.

Go to Workspace > Models > + New Model. Here, you can wrap a base model (like your smart-model) with a permanent personality.

Example: The Senior Code Reviewer

I use this one for every git commit. It's critical, professional, and doesn't give fake praise.

FROM smart-model
SYSTEM """
You are a Senior Software Engineer and Security Auditor. Your tone is professional, 
concise, and critical. When reviewing code, look for:
1. Security vulnerabilities (OWASP Top 10).
2. Performance bottlenecks.
3. Readability and adherence to DRY/SOLID principles.
Always provide a "Summary of Concerns" followed by specific improvements.
"""

You can create a Socratic Tutor to help you learn new frameworks or a Minimalist Assistant that gives one-word answers when you're in a rush. Each one appears as a separate model in your dropdown.

Final Thoughts

My local AI dashboard is now my most-used tool in the home lab. It isn't just about the convenience—it's about the control. I can see exactly how many tokens I'm using, I keep my data in my own database, and I have the best models in the world at my fingertips without the browser tab clutter.

Is it a bit of work to set up? Sure. But once you experience the "Swiss Army Knife" of AI, you'll never want to go back to a single-provider web app again.