LLMGW container

This page lists the environment variables supported by the LLMGW container. Variables are grouped by area and include defaults and usage notes where applicable.

App

  • DEBUG Whether the app is in debug mode. Mainly affects logging and telemetry.
    • Default is false.
  • ENVIRONMENT Deployment environment.
    • Supported values: dev, test, prd, local.
  • SESSION_SECRET_KEY The key used to protect FastAPI/Starlette session cookies. Should be at least 256 bits in production.
  • ADMIN_API_KEY Admin API key that can be used for access to the admin portal.
  • ALLOWED_HOSTS A comma-separated list of hosts against which the incoming request’s Host header will be checked.
    • Example: ["localhost", "my-test-domain.net"]
  • ADMIN_PORTAL_BRAND_COLOR Brand color (hex color code for primary brand color).
  • ADMIN_PORTAL_BRAND_ICON Brand icon (base64-encoded PNG for favicon).
  • ADMIN_PORTAL_BRAND_LOGO Brand logo (base64-encoded PNG for sidebar header).

Configuration sources

  • CONFIG_SOURCES A JSON list of configuration source definitions. Always in a format of *prefix*:*service-specific source identifier*. Allowed prefices are dir, s3 and azureblob..
    All sources must use the same backend, e.g., s3: cannot be mixed with azureblob:. For Azure Blob Storage the format is <container>/<blob>; for AWS S3 <bucket>/<file_path>; for local directory a filepath.

    Examples:

    [
  "dir:/etc/llmgw",
  "dir:./config"
]
    [
  "s3:config-bucket/config-prod.yaml",
  "s3:config-bucket/extra-models.yaml"
]

Entities

  • ALLOW_ANONYMOUS_ENTITIES Allow requests without a defined entity (user/project).
    • Default: true (helpful when spend limits are not enforced).

Secrets and secret stores

  • SECRET_STORE Secret provider backend used to resolve the secrets.

    • Supported values: env, azure_kv, aws_secrets_manager.

  • APPLICATION_KEY_VAULT_URL Azure Key Vault URL used when SECRET_STORE=azure_kv.

    • Example: https://llmgw-kv.vault.azure.net/

  • ENV_SERVICE_CACHE_EXPIRATION_SECONDS Cache expiration time (seconds) for local secrets.

    • Default: 600

Storage backends

  • STORAGE Storage backend used to store/load the YAML config.

    • Supported values: dir, azure_blob, aws_s3.

  • AZURE_BLOB_STORAGE_ACCOUNT_URL Azure Blob Storage account URL (required when STORAGE=azure_blob).

Database

  • DB_CONNECTION_STRING The Postgres connection string in libpq (key=value) or URL style.

OTEL

  • OTEL_SDK_DISABLED Disable OpenTelemetry configuration.

    • Default: false (OTEL enabled)

  • OTEL_TRACES_EXPORTER OTEL Traces exporter.

    • Supported values: otlp, console, or none.
    • Default: otlp.

  • OTEL_METRICS_EXPORTER OTEL Metrics exporter.

    • Supported values: otlp, console, or none.
    • Default: otlp.

  • OTEL_LOGS_EXPORTER OTEL Logs exporter.

    • Supported values: otlp or none.
    • Default: otlp.

  • OTEL_EXPORTER_ENDPOINT OTLP exporter endpoint. Required if any of OTEL_*_EXPORTER is otlp.

    • Example: http://<SERVICE_HOST>:4318
    • Azure Container Apps: <SERVICE_HOST> is the ACA service name (e.g., bss-llm-gateway-dev-otel).

  • OTEL_PERIODIC_EXPORT_INTERVAL_MILLIS OTEL periodic export interval (ms).

    • Default: 60000

  • DEBUG_METRICS When main exporter is otlp, enable additional console exporters / debug messages.

    • Boolean flag: 1, true, True.

Admin portal and SSO

  • ADMIN_AUTH_ENABLED Enable admin authentication/SSO. Warning: disabling this protection and removes SSO auth endpoints.

  • ADMIN_SSO_CLIENT_ID OAuth client ID (e.g., Entra ID App ID).

  • ADMIN_SSO_REDIRECT_CALLBACK URL the SSO flow redirects to after browser login.

  • ADMIN_SSO_REDIRECT_SUCCESS URL the backend redirects to after successful SSO completion.

  • ADMIN_SSO_AUTH_METHOD OAuth client auth method: client_secret_basic (client secret) or private_key_jwt (client certificate).

  • ADMIN_SSO_AUTH_CREDENTIAL OAuth credential. Must be either an Entra ID client secret or a base64-encoded private key (for private_key_jwt). See How to generate Azure credentials

  • ADMIN_SSO_PUBLIC_CERT Base64-encoded public certificate. Required when ADMIN_SSO_AUTH_METHOD=private_key_jwt.

  • AZURE_SERVER_METADATA_URL Azure tenant OpenID Connect metadata URL. Example: https://login.microsoftonline.com/{tenant_id}/v2.0/.well-known/openid-configuration

Azure

These are only relevant when deploying on Azure.

  • AZURE_CLIENT_ID Client ID of the user-assigned managed identity used by the container.

  • APPLICATIONINSIGHTS_CONNECTION_STRING Connection string for Azure Application Insights.

Logging

  • LOG_FILE Optional path to a log file. If empty or not set, logs go to stdout only.

  • STD_JSON_LOG Output logs as JSON to stdout.

    • Default: false.

  • FILE_JSON_LOG Output logs as JSON to LOG_FILE (requires LOG_FILE to be set).

    • Default: false.

  • LOG_RESOURCE_USAGE Periodically log CPU and memory usage.

    • Default: false.

  • LOG_USER_DATA Log user request/response payloads. Caution: useful for debugging, but not recommended in production due to potential sensitive data exposure.

    • Default: false.

Generate private key and public certificate for SSO cert-based flow

This can be done in multiple ways. A bash instruction is shown below. Requires openssl and base64 utils.

  1. Generate a private key:
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:3072 -out app-private.pem && \
chmod 600 app-private.pem
  1. Generate a public certificate using that key:
openssl req -new -x509 -key app-private.pem -sha256 -days 730 \
  -subj "/CN=mytestapp" -out app-public.cer
  1. Turn both to b64:
base64 < app-public.cer | tr -d '\n' > app-public.cer.b64 && \
base64 < app-private.pem | tr -d '\n' > app-private.pem.b64
  1. Upload the public cert file to EntraID
  2. Make b64 secret available via ADMIN_SSO_AUTH_CREDENTIAL var.
  3. Make b64 certificate available via ADMIN_SSO_PUBLIC_CERT var.