API keys

All LLM requests to LLMGW must be protected by API key or User based token.

User based tokens can be generated in admin portal. For now API keys can be configured in config files via plain text or reference to secret store.

Format

API keys should be specified in the following format: secret:*secret name*. Legacy format for Azure Key Vault keys is also supported: azure:*key vault url*/secrets/your_secret or azure:*key vault url*/your_secret

Azure Key Vault Keys

Initial Setup

Create a new secret in Azure Key Vault. Add link to the secret to application’s configuration with “azure:” prefix. For example, if the secret is named your_secret in the key vault your-key-vault, the configuration would look like this:

service_api_keys:
  your_key_name:
    user_key: "azure:https://your-key-vault.vault.azure.net/secrets/your_secret"

Additional properties

Additionally, you can assign a project and user to API keys, which will then be automatically associated as spend entities with any requests authenticated using the token. Support for additional entity types can be added on request.

service_api_keys:
  your_key_name:
    user_key: "azure:https://your-key-vault.vault.azure.net/secrets/your_secret"
    project: "test"

Rotating Azure Key Vault Secrets

When rotating secrets, add new version to the secret in Azure Key Vault. To ensure a smooth transition, the old secret should have expiration date set at least 30 minutes after the new secret’s activation date.

Plaintext Keys

For testing purposes you can define API key with plaintext

service_api_keys:
  local_dev_key:
    user_key: "plaintext:start123"