FAQ

Can I use my console login as an API key?

No. Console JWTs are accepted only on /api/*. Clients must use an API key (sk-…) on /v1/* and /servers/*. The reverse is also true: API keys are rejected on /api/*.

My key is lost — can I recover it?

No. Keys are bcrypt-hashed and shown once at creation. Create a replacement, switch clients, delete the old key. There is no rotate endpoint, but rotation without downtime is simply: create new → switch → delete old.

Can one model name map to several providers?

Yes — that is the point of provider mappings. Give each mapping a priority; higher priority is preferred, and failures fall back down the chain. See Models & Pricing.

Why does cost_usd show as null?

Null means unknown, not free: no rate is configured for the model/mapping, or the request had nothing billable. Configure pricing (manually or via the models.dev sync) to get real numbers. See Cost Control.

Why did I get HTTP 200 with an error JSON body?

The non-streaming keepalive had already committed a 200 before the failure surfaced. The real error is in the logs; the body contains the error JSON. This only happens for non-streaming requests that outlive the grace period (60 s by default).

A request returned 403 model_not_allowed. Where do I fix it?

Two allowlists exist: the user's and the key's, and the effective access is their intersection. Check Users & Roles and API Keys. An explicitly empty allowlist denies all models.

Does the proxy support vision / tools / reasoning / JSON mode?

Yes for all four, provided the selected upstream supports them — the proxy translates between protocols. Capability flags on the model record drive routing and the catalog; they do not block non-chat endpoints. See Create chat completion and Tools & Reasoning.

How are files and media mapped when I route to another provider?

The proxy renders each part in whatever form the upstream accepts, so the same input_image / input_file / input_audio part may arrive as inline bytes, a remote URI, or an uploaded-file reference. Worth knowing:

• file_id is passed through as-is. There is no /v1/files store, so an id must have been issued by the upstream it is sent to. An OpenAI Files-API id sent to Gemini (or the reverse) fails upstream.
• HTTP(S) image URIs are sent as native URL sources where the upstream has one (Anthropic image URL sources, Gemini file_data.file_uri, OpenAI / OpenRouter image_url); where it does not, the URL is downloaded and inlined before forwarding, so any publicly reachable URL works. SSRF protection applies to the download path.
• Document URLs on Chat Completions are provider-specific. Chat Completions has no single file shape and not every upstream accepts a URL: OpenRouter carries the URL in file.file_data, Zhipu / Z.AI (GLM) in file.file_url, and Mistral in a flat document_url. OpenAI and the generic openai-compatible adapter accept no file URL, so a URL document routed there is downloaded and inlined as base64 (subject to the same SSRF protection) when the upstream accepts the media type; a PDF becomes a file part, while a non-PDF still degrades because OpenAI's file part is PDF-only. DeepSeek's file part carries images and Files-API ids only, so a document routed there degrades. OpenRouter, Zhipu and Mistral accept the document types they document.
• Zhipu / Z.AI (GLM) cannot mix a file part with an image or video part. A message carrying both is split into separate messages (file on one, image_url / video_url on the other) so the upstream does not reject the request; text parts stay with the neighbouring part and ordering is preserved.
• Video URIs are not downloaded. Gemini fetches video itself and reliably accepts only YouTube URLs (or File API / gs:// URIs); an arbitrary https://…/v.mp4 is rejected upstream.
• Anthropic has no file content block. A Responses input_file is mapped onto the block Anthropic actually accepts: application/pdf (base64 or URL) and plain text (.txt / .md / .csv / … decoded into a text source) become document blocks, image payloads become image blocks, and file_id becomes a document/image file source. Media types Anthropic cannot represent (.docx, .xlsx, …) follow the unsupported_block_policy.
• Anthropic has no audio content block. Audio routed there degrades to an [Audio: …] text placeholder (never an audio block the API rejects).
• Audio has no OpenAI Responses content type. The Responses API accepts message content of input_text / input_image / input_file only, so audio routed to the built-in openai provider (which targets /responses) degrades to an [Audio: …] text placeholder instead of being sent as an input_audio part the API rejects. Audio passes through verbatim on the native Responses passthrough path, and Chat Completions upstreams keep their input_audio support.
• Multimodal tool results stay structured where the upstream supports it (Responses function_call_output.output, Gemini 3 functionResponse.parts); elsewhere the media is degraded to a text placeholder rather than dropped.
• A block the target provider cannot represent at all follows the unsupported_block_policy; the default Provider default resolves to drop for every provider today, while degrade keeps a text placeholder ([Image: …], [File: …]) and never silently drops — a block with no specific placeholder gets a generic [<TypeName> block] marker.

Can I export logs to my own system?

There is no export endpoint in v0.2.3; use GET /api/logs with cursor pagination (max 200 per page) or scrape container logs. Langfuse tracing (per user) covers LLM-level observability.

Is there an OpenAPI/Swagger page?

Not at runtime — the FastAPI docs (/docs, /redoc, /openapi.json) stay disabled. A generated OpenAPI 3.1 document covering the /v1 surface is published in this site instead: OpenAPI Specification. It is rendered from docs/public/v1-openapi.json, which uv run llm-proxy-openapi regenerates from the app itself, so the request schemas match what the server validates. Together with the Endpoint Index, that is the reference.

Can I run multiple replicas?

Yes, with PostgreSQL and REDIS_ENABLED=true + REDIS_RATE_LIMIT_ENABLED=true. Without Redis, rate limits, circuit breaker state, and provider statistics are per-process. Migrations run at startup per process — migrate once before scaling. See PostgreSQL & Redis.

Is response keepalive on by default?

Yes. With no stored configuration the runtime uses enabled / 60 s grace / 15 s interval, and the Settings form shows those same values, so the panel reflects what is actually in effect. Turn it off there if a client cannot tolerate the whitespace heartbeats.

Does the proxy inject MCP tools into model requests?

No. MCP servers are re-exposed as MCP endpoints at /servers/<name>/mcp for MCP clients; they are not spliced into /v1/* model requests. See MCP Servers.

What happens to a request when the client disconnects?

The upstream call is cancelled and the request is logged as 499 — visible in the logs instead of silently succeeding. Budgets count what was actually used at that point.

I'm locked out of the console. How do I get back in?

• Login lockout is off by default, so a wrong password normally just returns 401. If you enabled it and are locked out, wait out the 15-minute window — or restart the proxy, since the lockout counters live in process memory.
• Another admin can reset your password from Team (which forces a change at next sign-in).
• If you are the last admin with a forgotten password: there is no CLI recovery command — edit the database directly. Passwords are stored as bcrypt hashes in the users.password_hash column, so replacing the hash is the supported-by-hand route.

Which languages does the console support?

English and 中文, switchable in Settings → General → Preference (browser-local, per user).

Where do feature requests and bugs go?

GitHub issues — see the triage labels in the repository for how issues are handled.