LiteLLM
LiteLLM is an open-source proxy that puts one OpenAI-compatible API in front of several AI model providers. It issues scoped keys and logs every request’s model, tokens and cost. Connect it to a Scolah Compass installation to keep AI traffic and spending on your own server and your own provider accounts.
LiteLLM is licensed under MIT. Free to run — not billed. Requests to the models are billed by the providers to your own accounts.
What gets installed
Section titled “What gets installed”- LiteLLM proxy: the API endpoint and its admin UI.
- Database: a PostgreSQL database that stores the proxy’s keys and its spend and usage log.
The proxy comes with these models set up:
| Model | Provider |
|---|---|
claude-sonnet-5, claude-sonnet-4-6, claude-opus-5, claude-haiku-4-5 |
Anthropic |
gpt-4o-mini |
OpenAI |
text-embedding-3-small (embeddings) |
OpenAI |
LiteLLM serves one website. Give it a web address from the installation’s Domains tab: see Domains and HTTPS.
Before you install
Section titled “Before you install”- Get an API key from each provider you want to use: Anthropic for the Claude models, OpenAI for the GPT model and for embeddings. Scolah Compass needs the OpenAI embeddings model for its document search (RAG), so add an OpenAI key if Scolah Compass will use this proxy.
- Choose a Deployment size:
smallormedium.smallruns one proxy worker with 1 GiB of memory and needs about 1 CPU core, 2 GiB of memory and 5 GiB of disk.mediumruns 4 workers with 3 GiB and needs about 4 CPU cores, 5 GiB of memory and 10 GiB of disk. Server fit checks these figures.
Options
Section titled “Options”| Option | What it does |
|---|---|
| Deployment size | small for a lighter footprint. medium runs more workers, with more memory, for more requests. |
| Anthropic API key | In the Secrets box. Your own Anthropic key. Requests to Claude models are billed to your Anthropic account. |
| OpenAI API key | In the Secrets box. Your own OpenAI key, used for GPT models and for the embeddings Scolah Compass needs. |
Both API keys are optional. A model whose key you leave out fails when it’s called, but the proxy still starts. The keys are sealed to your server and are never shown in the portal again.
Under Advanced you can use a PostgreSQL server you already run instead of the bundled database. See External services.
First sign-in
Section titled “First sign-in”The portal doesn’t show any First login credentials for LiteLLM. Its master key and database password are generated for you and stay sealed on the server. Applications you connect get their own scoped key, never the master key.
To sign in to the LiteLLM admin UI, connect a single sign-on provider (see below).
Connections
Section titled “Connections”LiteLLM can offer to other installations:
- LLM proxy (OpenAI-compatible): the AI endpoint for another installation, such as Scolah Compass. The platform issues that installation its own scoped key. When you connect, you pick its Default model: the model it uses unless it asks for another.
- LLM spend and usage log (read-only): read-only access to the proxy’s database, for reporting on AI usage and cost. Key tables are excluded.
LiteLLM can use from other installations:
- Single sign-on for the admin UI: sign in to the admin UI through an identity provider such as Keycloak. This only affects the admin UI; API calls are still authenticated by key. Optional.
See Connections.