API Reference

Base URL: http://localhost:8080 (your proxy). All LLM endpoints are served in three client protocols — OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages — and the proxy translates between them and every configured upstream.

Authentication is a single API key on every /v1/* route: Authorization: Bearer sk-… (see Authentication).

BASH

curl http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer $LLM_PROXY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "auto", "messages": [{"role": "user", "content": "Hello!"}]}'

Chat

Conversation completion across all three wire protocols, plus response storage and token counting.

• Create chat completion — POST /v1/chat/completions
• Create response — POST /v1/responses (SSE + WebSocket)
• Retrieve response — GET /v1/responses/{response_id}
• Delete response — DELETE /v1/responses/{response_id}
• Cancel response — POST /v1/responses/{id}/cancel
• List input items — GET /v1/responses/{id}/input_items
• Compact response — POST /v1/responses/compact
• Anthropic Messages — POST /v1/messages
• Count tokens — POST /v1/messages/count_tokens

Images

• Create image — POST /v1/images/generations
• Edit image — POST /v1/images/edits

Audio

• Create speech — POST /v1/audio/speech
• Create transcription — POST /v1/audio/transcriptions
• Create translation — POST /v1/audio/translations

Embeddings

• Create embeddings — POST /v1/embeddings

Models

• List models — GET /v1/models

Cross-cutting

• Authentication — API keys vs console JWT
• OpenAPI Specification — generated /v1 contract, rendered from code
• Endpoint Index — every route, including WebSocket, MCP, health, admin
• Streaming — SSE formats per protocol, WebSocket transports
• Errors & Rate Limits — error envelopes, status codes, budgets
• Tools, Reasoning & Web Search — tool calling, thinking, web search interception
• Virtual Models & Routing — auto / fast / best

Conventions

• model is always the name your proxy configured — including the virtual models auto, fast, best when smart routing is on.
• The model field in responses echoes the client-requested alias, never the resolved upstream model.
• Every response carries X-Request-Id; pass it when reporting problems.
• Upstream rate-limit headers (x-ratelimit-*, retry-after) are forwarded.

BASH

curl http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer $LLM_PROXY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "auto", "messages": [{"role": "user", "content": "Hello!"}]}'