Add support for configuring providers in priority tiers, where lower-tier (higher priority) providers are used first, and higher-tier providers serve as fallbacks only when all lower-tier providers are exhausted.
Use Case
Many users have multiple API access methods with different cost structures:
- Tier 1: Subscription-based accounts (e.g., Claude Max) - cheaper/included usage
- Tier 2: Usage-based API keys (e.g., pay-per-token) - more expensive
Currently, CLIProxyAPI uses round-robin across all providers equally, which means expensive usage-based credentials get consumed even when cheaper subscription-based options are available.
Proposed Configuration
claude-api-key:
- api-key: "AIzaSy...subscription"
tier: 1 # Use first (subscription-based, cheaper)
- api-key: "AIzaSy...payasyougo"
tier: 2 # Fallback only (usage-based, expensive)
openai-compatibility:
- name: "openrouter"
tier: 2 # Fallback provider
# ...
- name: "local-llm"
tier: 1 # Preferred provider
# ...
Expected behavior:
- Always prefer tier 1 providers (round-robin within tier 1)
- Only use tier 2 when ALL tier 1 providers are:
- Quota exceeded (in cooldown)
- Temporarily unavailable (auth errors, rate limits)
- Automatically return to tier 1 when any tier 1 provider recovers
- Default tier: 1 if not specified (backward compatible)
- Able to add more tiers (tier 3, tier 4, etc.) following the same pattern
Alternatives considered:
- Manually disabling expensive credentials and re-enabling when needed (inconvenient)
- Running separate proxy instances per tier (complex, doesn't solve automatic fallback)
Add support for configuring providers in priority tiers, where lower-tier (higher priority) providers are used first, and higher-tier providers serve as fallbacks only when all lower-tier providers are exhausted.
Use Case
Many users have multiple API access methods with different cost structures:
Currently, CLIProxyAPI uses round-robin across all providers equally, which means expensive usage-based credentials get consumed even when cheaper subscription-based options are available.
Proposed Configuration
Expected behavior:
Alternatives considered: