Capability · AI Gateway
Keep LLM provider keys out of agent pods and stop runaway loops at the door.
Built on Envoy AI Gateway, Zelkor holds your upstream model credentials in gateway Secrets, caps request rates before calls leave the cluster, and checks prompts with NVIDIA NeMo Guardrails.
Why Putting API Keys in Agent Pods Fails
Putting OpenAI, Anthropic, or Bedrock keys directly into agent deployment manifests spreads high-value credentials across dozens of pods. If one pod is compromised or gets stuck in a retry loop, it can leak the raw key or burn through thousands of upstream API calls with no central cutoff.
Central Provider Key Custody
Agent pods receive only a local gateway token and cluster URL. Real credentials for OpenAI, Anthropic, Gemini, Bedrock, Vertex, Azure OpenAI, Cohere, Ollama, and vLLM stay in Kubernetes Secrets mounted only on the gateway.
Global Request Circuit Breaker
A configurable per-minute cap (50 requests per minute by default) stops runaway agent loops at the gateway. Extra model calls are rejected inside the cluster before reaching your provider.
Fail-Closed NVIDIA NeMo Guardrails
When enabled, chat completions pass through NVIDIA NeMo Guardrails for topic and content safety checks before reaching the model or returning to the caller. If a check fails, the call is blocked.
Header-Based Agent Routing
Envoy fronts both model traffic and standard agent run and thread endpoints, routing incoming requests to the right LangGraph, LangChain, or Deep Agents deployment by graph header.
How Teams Use It
workspace:
models:
rateLimit:
enabled: true
requestsPerMinute: 50
policies:
nemo:
enabled: true
intercept:
enabled: trueRemove upstream API keys from every agent pod.