LiteLLM, one API shape in front of a hundred models

There are two products sharing a name and both are useful. The SDK normalises more than a hundred providers onto one request and response shape, so replacing Bedrock with Vertex becomes a string change instead of a refactor. The proxy is the one that matters on an engagement: a self-hosted server offering virtual keys with budgets, centralised logging, caching, guardrails and an admin interface, which any OpenAI-compatible client can point at without changing a line of application code. A client running on Azure with a procurement rule against calling a third-party model directly is an ordinary Tuesday, and a gateway you deploy inside their own network is the answer that survives their review. Per-key budgets mean a runaway agent exhausts one key rather than one company's card. Cost tracking and callbacks land in your logs or your tracing tool without you writing the glue, which is an afternoon you get back.

The abstraction leaks exactly where it costs you. One common shape means a lowest common denominator, so provider-specific behaviour like prompt caching semantics, reasoning blocks, structured output modes and citation formats arrives as pass-through keyword arguments, and pass-through keyword arguments are untyped. The failure mode is genuinely nasty: your code works against one provider, silently drops a field against another, and you learn about it from degraded output quality rather than from an exception. Running the proxy means running a service, which is another deployment, another database, another entry on the incident rota, in an environment that may not be yours to operate. The feature surface moves quickly enough that pinning a version is mandatory rather than cautious. And the documentation describes enterprise capabilities without always drawing a clean line around what is paid, so budget time to find out what you actually get before you promise it.

Spend an hour on it if you will touch more than one model provider this quarter, or if a client's network rules make direct provider calls awkward. Skip the proxy entirely if you are one team on one provider, because you would be operating infrastructure to solve a problem you have not got. The SDK on its own is a reasonable middle position and costs you nothing to run.

OpenRouter if you would rather someone else operated the gateway and the client permits sending traffic to a third party.