Providers

A provider is one upstream endpoint plus its credentials. Providers are independent of the model names clients see — you can add several providers serving the same model and let fallback and priority decide who answers.

Adding a provider

Providers → Add Provider. Fields:

Field	Notes
Name	Unique identifier, referenced by model mappings. Not the vendor name — e.g. openai-prod
Type	One of the 23 built-in adapter types (table below)
API key	Required for most types; not required (and not used) by ollama, vllm, sglang
Base URL	Pre-filled per type. Required for openai-compatible. Trailing slashes are trimmed
Icon URL	Optional logo shown next to the provider in the console; must be a public URL (https:/http:) — private/loopback addresses are rejected by the SSRF guard
Gemini Interactions	gemini type only: use Google's newer Interactions API instead of generateContent (metadata.api_variant). The provider list shows an Interactions badge
API version	Not in the console dialog; set via API/DB only. Not applied by any adapter in v0.2.3
Timeout	Not in the console dialog; set via API/DB only. Not applied in v0.2.3 — upstream calls use a fixed 600 s read timeout (10 s connect)
Custom headers	Extra headers sent upstream (e.g. tenant or organization headers). These win over any other source for the same header name
App Attribution	openrouter type only: application URL (HTTP-Referer), name, marketplace categories and ranking visibility. Unset values fall back to crediting the LLM Proxy project
Endpoint base URLs	Per-endpoint overrides; the URL is used as-is, without appending the endpoint path
Native web search	Set when the upstream provides its own web-search tool; the proxy then passes web_search tools through instead of intercepting them
Priority	Not shown in the console; provider-level priority exists only via API/DB and is ignored by selection — fallback order comes from each model mapping's priority (see Models & Pricing)

Provider API keys are masked in the console — the edit dialog shows only a masked, disabled field. The only way to read a key back is the audited POST /api/config/providers/{name}/api-key/reveal endpoint. Keys are encrypted at rest with Fernet (key derived from ENCRYPTION_KEY); rotating that environment variable makes stored keys undecryptable.

Provider types

Type	Upstream wire protocol	Default base URL
openai	Chat Completions + Responses	https://api.openai.com/v1
anthropic	Anthropic Messages (x-api-key, anthropic-version: 2023-06-01)	https://api.anthropic.com
openai-compatible	Chat Completions (generic; base URL required)	—
gemini	Google generateContent (x-goog-api-key)	https://generativelanguage.googleapis.com/v1beta
ollama	Native Ollama (/api/chat, /api/tags)	http://localhost:11434
vllm	Chat Completions	http://localhost:8000/v1
sglang	Chat Completions	http://localhost:30000/v1
deepseek	Chat Completions + native Anthropic/Responses	https://api.deepseek.com/v1
xai	Chat Completions + native Responses	https://api.x.ai/v1
zai	Chat Completions (GLM layout)	https://api.z.ai/api/paas/v4
zai-coding	Chat Completions + native Anthropic/Responses	https://api.z.ai/api/coding/paas/v4
zhipu	Chat Completions + native Anthropic	https://open.bigmodel.cn/api/paas/v4
zhipu-coding	Chat Completions + native Anthropic/Responses	https://open.bigmodel.cn/api/coding/paas/v4
moonshot	Chat Completions + native Anthropic	https://api.moonshot.ai/v1
kimi-code	Chat Completions + native Anthropic	https://api.kimi.com/coding/v1
minimax	Chat Completions + native Anthropic/Responses	https://api.minimax.io/v1
qwen	Chat Completions + native Anthropic/Responses	https://dashscope.aliyuncs.com/compatible-mode/v1
qwen-intl	Same as qwen (international keys are region-bound)	https://dashscope-intl.aliyuncs.com/compatible-mode/v1
openrouter	Chat Completions + native Responses	https://openrouter.ai/api/v1
chutes	Chat Completions	https://llm.chutes.ai/v1
mistral	Chat Completions	https://api.mistral.ai/v1
nanogpt	Chat Completions	https://nano-gpt.com/api/v1
opencode-zen	Per-model dialect: Chat Completions / Anthropic / Responses / Google / System One (see below)	https://opencode.ai/zen/v1
opencode-go	Same gateway, subscription lane (own model catalog)	https://opencode.ai/zen/go/v1

Native protocol means the proxy can forward a request verbatim to that upstream when the client speaks the same protocol (Anthropic clients hitting a native-Anthropic provider, Responses clients hitting a native-Responses provider), preserving fields the translation layer would otherwise normalize. vllm/sglang opt in per provider via provider metadata native_passthrough: true.

Cost reporting: openrouter and nanogpt return their own cost figures, which the proxy records instead of computing them from token prices.

OpenCode (opencode-zen / opencode-go)

The OpenCode gateway fronts model families whose upstreams speak different wire protocols. Which dialect a request takes is a property of the model, resolved per request from, in order: a learned correction (below), the models.dev catalog (the OpenCode team's own per-model provider.npm data, fetched with a 6h TTL cache — new model families route correctly with no code change), then model-id prefix heuristics (Chat Completions by default). Claude models are served on Anthropic Messages, GPT/Grok on OpenAI Responses, Gemini on generateContent, Jev on System One (/v1/systemone endpoint only — chat requests for Jev models are rejected with a pointer there), and everything else on Chat Completions.

• Types = lanes: opencode-zen is the Console pay-as-you-go lane (https://opencode.ai/zen/v1), opencode-go the subscription lane (https://opencode.ai/zen/go/v1) — the two lanes serve different model catalogs (the same id can even speak a different dialect on each; the adapter reads the lane's own models.dev entry). Both keep every dialect on one flat root under the base URL. The Console lane's dialect-prefixed paths (https://opencode.ai/inference/v1) are reachable by setting base_url to it.
• Headers: Authorization: Bearer everywhere, plus the dialect's auth header where the gateway reads one (x-api-key on the Anthropic route, x-goog-api-key on the Google route). The OpenCode CLI's client-identity headers (User-Agent: opencode/..., x-opencode-client, x-opencode-project, per-request x-opencode-session / x-opencode-request) are sent by default — the free-tier models are rejected with 403 FreeTierError without them. Any of these can be replaced via Custom headers.
• Self-healing: when the gateway rejects a model with a model-flavored 400/404, the request is retried on the remaining dialects (Chat Completions first) and the working dialect is remembered for the process lifetime.
• Model list: the gateway's /v1/models answers with bare ids; display names, context windows and pricing are enriched from the models.dev catalog. Set provider metadata opencode_catalog_url to a mirror to pin the catalog, or to an empty value to disable it (heuristics then carry the routing).

OpenRouter specifics

OpenRouter is a router rather than a single upstream, so the adapter keeps a few of its request shapes intact:

• Responses and Messages passthrough. /v1/responses and /v1/messages clients are forwarded verbatim, so reasoning.mode / reasoning.context, thinking budgets, context_management, provider routing preferences, plugins, models fallbacks and usage accounting all reach OpenRouter. An Anthropic-protocol client avoids the double translation (Messages → Chat Completions → Messages) that used to drop them. Chat Completions clients get the equivalent fields through the request builder. Set provider metadata native_passthrough: false to fall back to the Chat Completions translation (e.g. if the routed model rejects a Responses-only item type).
• Unified reasoning object. The proxy sends reasoning: {effort, mode, context, summary, max_tokens, exclude, enabled} instead of a bare reasoning_effort, because mode: "pro" (and the other Responses-only fields) have no top-level equivalent. OpenRouter only honours mode: "pro" when it can route to OpenAI or Azure, and only for GPT-5.6 and newer.
• App attribution. The App Attribution fields in the provider dialog identify your deployment on OpenRouter's rankings: the application URL (HTTP-Referer), display name (X-OpenRouter-Title), marketplace categories and whether a newly created app is listed publicly. Unset fields fall back to crediting the LLM Proxy project; headers set explicitly under Custom Headers win over both. Values are validated as printable ASCII (they become raw HTTP headers), the URL must be an absolute http(s) URL, and at most two categories are accepted.
• Forwarded client headers. Per-request OpenRouter control headers (X-OpenRouter-Metadata, X-OpenRouter-Cache, X-OpenRouter-Cache-TTL, X-OpenRouter-Cache-Clear) are forwarded from the client, as are x-session-id (sticky routing and prompt-cache affinity on chat; on the embeddings/image/audio endpoints it is the only way to attach a session, used there for usage grouping) and x-anthropic-beta (Claude beta opt-ins such as structured-outputs-2025-11-13, without which OpenRouter strips strict from tool definitions). Client Codex / Claude Code fingerprint headers are not forwarded to a router.
• Per-endpoint passthrough fields. Each endpoint keeps the fields OpenRouter documents beyond the OpenAI schema: chat (provider, models, plugins, session_id, trace, preset, structured_outputs, cache_control, min_p, top_k, top_a, repetition_penalty), embeddings (input_type, session_id, trace, provider, user), image generation (resolution, aspect_ratio, seed, input_references, provider), TTS (input_references, provider) and STT (provider). Chat also forwards the legacy route, transforms, usage and include_reasoning fields for clients that still send them — upstream deprecates them in favour of providers.sort.partition, the context-compression plugin, stream_options.include_usage and reasoning.exclude respectively. Image requests are normalized to the Image API's own schema (output_format, background, output_compression; response_format and style are dropped), and TTS/STT response_format is validated against the formats OpenRouter accepts (mp3/pcm and json/verbose_json) instead of surfacing an upstream 400.
• Response metadata. X-Generation-Id (usable with GET /api/v1/generation?id=) and the X-OpenRouter-Cache-* headers are forwarded to the client; cost, cost_details and is_byok from usage feed the cost record.
• Model picker. The provider's catalog is fetched with output_modalities=all — upstream defaults to text, which would hide the image, video, audio and embedding models this adapter also serves — and keeps the catalog's context_length, modalities, supported_parameters and pricing, so the picker can show which models are non-text and what they cost.
• Audio. Speech (mp3/pcm) and transcription (json/verbose_json) formats are checked at the proxy boundary rather than surfaced as an upstream 400. OpenRouter has no audio translation endpoint, so /v1/audio/translations is rejected locally instead of 404-ing upstream.
• Model variants. :free, :nitro, :floor, :online, :thinking and the *-pro variants are passed through as plain model strings, but model lookup is an exact name match — register each variant you want to expose as its own model, or route to it with a model alias.

Qwen (Model Studio / DashScope) specifics

Two provider types cover the region-bound DashScope domains: qwen (Beijing, dashscope.aliyuncs.com) and qwen-intl (Singapore, dashscope-intl.aliyuncs.com). API keys are region-bound, so an international key only works with qwen-intl and vice versa. Business-space dedicated domains (https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/…) work through the provider Base URL field.

• Native protocols. Anthropic Messages ({host}/apps/anthropic/v1/messages) and OpenAI Responses ({base_url}/responses) are forwarded verbatim, so Claude Code request shapes, cache_control breakpoints and Responses items reach DashScope unchanged. The Anthropic endpoint exposes only /v1/messages — there is no model-list endpoint, so clients that probe /v1/models see a 404 from upstream.
• Chat vendor fields. Unrecognized chat fields pass through to DashScope, so enable_thinking, thinking_budget, enable_search, search_options, tool_stream and enable_code_interpreter work from a client's extra_body.
• Images. Wan and Qwen-Image both run through the DashScope async task API. DashScope's image extension fields are accepted and forwarded, flat on the body: image (URL / base64 / array, turning the call into image-to-image), negative_prompt, seed, prompt_extend, prompt_extend_mode, enable_thinking, watermark, and on edits bbox_list, color_palette, enable_sequential, thinking_mode. There is no mask concept — pass bbox_list instead.
• Embeddings. text-embedding-v1..v4 and qwen3.7-text-embedding ride {base_url}/embeddings. dimensions is validated against DashScope's documented value set, and per-request row caps (20 for qwen3.7-text-embedding, 10 for v3/v4, 25 for v1/v2) are enforced at the proxy.
• No audio. DashScope has no OpenAI-compatible audio surface. Speech synthesis (Qwen-TTS, CosyVoice) uses the native /api/v1/services/aigc/multimodal-generation/generation endpoint, and OpenAI-compatible ASR (qwen3-asr-flash) is /chat/completions with input_audio content parts. /v1/audio/* against a qwen provider is rejected with a 400.
• Web search. DashScope's Responses API has its own web_search / web_extractor / code_interpreter tools. Set provider metadata native_web_search: true to let web_search tools reach it; otherwise the proxy intercepts them and runs its own web search.

Verifying a provider

• GET /api/config/providers/{name}/models queries the upstream model list — this doubles as the effective connectivity check. The console calls it from the provider-model picker in the model editor (the ↻ button in the picker refreshes the list). ollama, vllm, and sglang need no API key for this; everything else does.
• Once a model mapping exists, send a test request from the Chat playground or with curl, then confirm the row in Logs.

Known limitations (v0.2.3)

Documented deliberately so you do not chase them:

• Provider timeout, api_version, and rate_limit are not applied by the request path. Upstream calls use a fixed 600 s read timeout; rate limiting is global/per-key, not per provider.
• Provider priority does not affect selection — set priority on each model → provider mapping instead.
• Provider enabled does not filter traffic; remove the model mappings (or raise another provider's mapping priority) to take a provider out of rotation.
• No vertex/region/project fields for Gemini; no OAuth flows. Upstreams that need client fingerprint headers work via native passthrough only.
• No per-provider health checks; the circuit breaker is the only automatic protection.

Related

• Models & Pricing — map client-facing names onto this provider
• Server Settings — retries, fallback, circuit breaker