Provider Setup¶

AMP uses external AI providers for content generation. At least one LLM provider is required.

LLM Providers¶

Anthropic Claude (Recommended)¶

ANTHROPIC_API_KEY=sk-ant-xxxxx

Supported models: - claude-3-opus (highest quality) - claude-3-sonnet (balanced) - claude-3-haiku (fastest)

OpenAI¶

OPENAI_API_KEY=sk-xxxxx

Supported models: - gpt-4-turbo - gpt-4 - gpt-3.5-turbo

Google Gemini¶

GOOGLE_API_KEY=xxxxx

Supported models: - gemini-pro - gemini-pro-vision

Image Providers¶

OpenAI DALL-E¶

OPENAI_API_KEY=sk-xxxxx  # Same key as LLM

Stability AI¶

STABILITY_API_KEY=sk-xxxxx

Replicate¶

REPLICATE_API_TOKEN=xxxxx

Provider Configuration¶

Default Provider¶

Set default LLM provider:

DEFAULT_LLM_PROVIDER=claude
DEFAULT_IMAGE_PROVIDER=openai

Per-Tenant Configuration¶

Configure via API:

curl -X PUT https://api.amp.dev/v1/tenants/{id} \
  -H "Authorization: Bearer $AMP_API_KEY" \
  -d '{
    "config": {
      "provider_routing": {
        "llm_preference": "claude",
        "llm_fallback": ["openai"],
        "image_preference": "openai"
      }
    }
  }'

Provider Fallback¶

When primary provider fails, AMP automatically tries fallbacks:

graph LR
    A[Request] --> B{Claude}
    B -->|Success| C[Response]
    B -->|Fail| D{OpenAI}
    D -->|Success| C
    D -->|Fail| E{Gemini}
    E -->|Success| C
    E -->|Fail| F[Error]

Fallback Configuration¶

ENABLE_PROVIDER_FALLBACK=true
PROVIDER_TIMEOUT=30s
MAX_PROVIDER_RETRIES=2

Cost Management¶

Tracking¶

AMP tracks provider costs per request:

curl https://api.amp.dev/v1/analytics/costs \
  -H "Authorization: Bearer $AMP_API_KEY"

Limits¶

Set spending limits:

curl -X PUT https://api.amp.dev/v1/tenants/{id} \
  -d '{
    "config": {
      "cost_limits": {
        "monthly_limit_cents": 100000,
        "alert_threshold_percent": 80
      }
    }
  }'

Rate Limits¶

Provider Rate Limits¶

Handling Rate Limits¶

AMP automatically: - Queues requests when approaching limits - Backs off with exponential delay - Falls back to alternate providers

Self-Hosted LLMs¶

Ollama¶

OLLAMA_URL=http://localhost:11434
OLLAMA_MODEL=llama2

vLLM¶

VLLM_URL=http://localhost:8000
VLLM_MODEL=meta-llama/Llama-2-70b

Configuration¶

CUSTOM_LLM_PROVIDER=ollama
CUSTOM_LLM_URL=http://localhost:11434
CUSTOM_LLM_MODEL=llama2

Monitoring¶

Health Checks¶

curl https://api.amp.dev/v1/providers/claude/health \
  -H "Authorization: Bearer $AMP_API_KEY"

Usage Statistics¶

curl https://api.amp.dev/v1/analytics/costs?breakdown=provider \
  -H "Authorization: Bearer $AMP_API_KEY"

Troubleshooting¶

Provider Unavailable¶

Check provider status pages: - Anthropic Status - OpenAI Status - Google Cloud Status

Rate Limit Errors¶

{
  "error": {
    "code": "provider_rate_limited",
    "message": "Claude rate limit exceeded",
    "retry_after": 60
  }
}

Solutions: - Wait for rate limit reset - Enable fallback providers - Upgrade provider plan

Authentication Errors¶

{
  "error": {
    "code": "provider_auth_failed",
    "message": "Invalid API key for provider: claude"
  }
}

Solutions: - Verify API key is correct - Check key hasn't expired - Ensure key has required permissions