Configuring LLM Providers
CleverThis connects to upstream LLM providers to serve inference requests. Providers can be configured via environment variables (for static, server-level credentials) or via the management API / Dashboard (for workspace-level credentials added at runtime).
Static provider configuration (env vars)
Set the provider's API key as an environment variable. CleverThis automatically registers the provider and makes its models available in all workspaces:
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_AI_API_KEY=...
MISTRAL_API_KEY=...
GROQ_API_KEY=gsk_...
TOGETHER_API_KEY=...
Custom / self-hosted providers
For providers with a custom base URL (e.g. Azure OpenAI, local Ollama, vLLM), use:
CUSTOM_PROVIDER_NAME=azure-openai
CUSTOM_PROVIDER_API_KEY=your-azure-key
CUSTOM_PROVIDER_BASE_URL=https://your-instance.openai.azure.com/
Multiple custom providers can be configured by using indexed variable names:
CUSTOM_PROVIDER_1_NAME=ollama-local
CUSTOM_PROVIDER_1_BASE_URL=http://ollama:11434/v1
CUSTOM_PROVIDER_1_API_KEY=ollama # Ollama ignores this but it must be set
CUSTOM_PROVIDER_2_NAME=vllm-gpu
CUSTOM_PROVIDER_2_BASE_URL=http://vllm-server:8000/v1
CUSTOM_PROVIDER_2_API_KEY=token-from-vllm
Dynamic provider configuration (API)
Workspace administrators can add, update, and remove provider credentials at runtime via the A2A management API without restarting the server. See A2A — Providers.
Provider health
If a provider's API key is invalid or the provider is unreachable, requests to that provider return 502 Bad Gateway. You can check provider health from the admin panel at https://your-domain.com/admin/providers.
Per-provider fallback routing (automatically fall back to a secondary provider if the primary is down) is being added.