I have API keys for OpenAI, Anthropic, Google, Groq, Together AI, Replicate, and counting.

Every new AI tool I try wants me to paste them manually. Copy from one dashboard. Paste into a config. Repeat for every provider. Repeat for every app.

My keys are now scattered across dozens of apps. Some store them in localStorage. Some in plaintext config files. I have no visibility into who’s using what, or how much they’re spending.

We have OAuth for user authentication. Why don’t we have the equivalent for API keys?

The Current Landscape

Good news: people are already solving pieces of this problem.

LiteLLM Proxy

LiteLLM runs a local proxy that unifies all your AI providers behind one OpenAI-compatible endpoint:

# litellm-config.yaml
model_list:
  - model_name: gpt-4
    litellm_params:
      model: openai/gpt-4
      api_key: os.environ/OPENAI_API_KEY
  - model_name: claude-3
    litellm_params:
      model: anthropic/claude-3-opus
      api_key: os.environ/ANTHROPIC_API_KEY

One endpoint for all providers. Virtual keys for rate limiting and budgets. But you still manage the master keys yourself.

Cloudflare AI Gateway

Cloudflare’s AI Gateway lets you store API keys in their dashboard:

• Secure storage with encryption
• Easy key rotation
• Usage analytics across providers
• Rate limiting and caching

They’re adding features for dynamic routing and scoped access. Getting closer to what I want.

Eden AI

Eden AI goes further. Single API, multiple providers:

• One Eden AI key to access OpenAI, Anthropic, Google, etc.
• They handle the provider routing
• Unified dashboard for usage and costs
• Automatic failover between providers

Trade-off: you’re trusting them with your traffic.

llmux

I built llmux to solve this for my own projects. It’s a unified LLM inference proxy with multi-provider routing:

routing:
  default_strategy: round-robin
  fallback_chain: [groq, cerebras, together, openrouter]

model_aliases:
  llama-70b:
    groq: llama-3.1-70b-versatile
    together: meta-llama/Llama-3.1-70B-Instruct-Turbo

One API key, round-robin across providers, automatic fallbacks, caching. Works with any OpenAI SDK. Deploy locally, Docker, or Cloudflare Workers.

Still requires managing provider keys yourself, but at least they’re in one place.

Other Players

• IBYOK: Encrypted vault specifically for LLM API keys
• Kilo Code: Organization-wide API credential management
• ngrok AI Gateway: Proxy with failover and load balancing across providers

What’s Still Missing

These tools help, but they’re all different pieces. Here’s what I actually want:

1. User-Controlled Authorization Flow

Like OAuth, but for API keys:

1. App requests: "I need OpenAI GPT-4 access, $10/month limit"
2. I get a prompt: "Example App wants GPT-4 access. Allow?"
3. I approve
4. App gets a scoped token, not my master key

No copy-paste. No trusting apps with my master key. I approve once, revoke anytime.

2. Scoped Tokens, Not Master Keys

Apps should never see my sk-... key. They get a proxy token that:

• Is rate limited
• Is model restricted
• Has a spending cap
• Can be revoked instantly

LiteLLM’s virtual keys do this if you self-host. But it requires running your own infrastructure.

3. Universal Standard

Each solution above is siloed. What if there was a standard that:

• Apps request access using a common protocol
• Any vault/gateway can fulfill the request
• Users choose their preferred vault

Like how OAuth works across providers.

OKAP: A Proposal

I’ve drafted a spec: OKAP - Open Key Access Protocol

Here’s what an OKAP request could look like:

{
  "okap": "1.0",
  "request": {
    "provider": "openai",
    "models": ["gpt-4", "gpt-4o-mini"],
    "capabilities": ["chat", "embeddings"],
    "limits": {
      "monthly_spend": 10.00,
      "requests_per_minute": 60
    },
    "expires": "2025-03-01"
  }
}

A browser extension or native app intercepts this, shows me the request, I approve or deny. The app gets a scoped OKAP token.

What Would Make OKAP Work

For OKAP to become a real standard:

1. Provider buy-in: OpenAI, Anthropic, Google would need to support scoped tokens or recognize a standard proxy protocol
2. User adoption: A trusted vault app/extension that people actually install
3. Developer adoption: Apps willing to integrate the flow

The infrastructure exists. LiteLLM, Cloudflare, and others prove the proxy model works. OKAP would standardize the handshake between apps and user-controlled vaults.

What I’m Doing Today

For now, I use:

• Environment variables for local development
• llmux for multi-provider routing and fallbacks
• Cloudflare AI Gateway for production apps with caching

Not ideal. Keys are still in a config file. No universal authorization flow.

What’s Next?

The pieces exist. LiteLLM does virtual keys. Cloudflare does secure storage. llmux does routing.

What’s missing: a standard way for apps to request access and users to grant it.

If you’re:

• Building an AI app: Would you integrate a BYOK authorization flow?
• Working on key management: Want to collaborate on a spec?
• Using multiple AI APIs: What’s your current workflow?

I’m thinking about writing a draft spec. Let me know if you’d use it.

LinkedIn | GitHub