Back to Tools
Inference servingOpen sourceUpdated 2026

LiteLLM

Intermediate · LLM gateway/proxy

Proxy and SDK for routing requests across many LLM providers with OpenAI-compatible interfaces.

Best for

Teams managing multiple model providers, budgets, keys, and routing rules.

Why use it

Useful when you want one gateway across local, open, and commercial models.

Tradeoffs

A gateway adds another operational layer; keep routing policies simple.

Key features

  • Provider routing
  • OpenAI-compatible proxy
  • Spend controls

Alternatives

OpenRouter, Portkey, LocalAI

Where it fits

LiteLLM belongs in the inference serving layer of an open AI stack. Evaluate it against your model runtime, privacy needs, deployment target, and the amount of operational complexity your team can support.

CategoryInference servingLicenseMITDeploymentLLM gateway/proxyModeSelf-hosted proxy or cloud
LiteLLM GitHub →

Recommendation

Use LiteLLM when model routing and provider abstraction matter.