Request Identity

Request Identity Definition

Request identities in LLMGW can be defined in several ways. Before you can use an identity, the corresponding entity type must be configured here so that LLMGW can recognize it.

LLMGW Headers

You can specify request identities using custom HTTP headers. Headers with the llmgw- prefix are filtered and checked against the allowed entity types.

For example, the following request will set these identities:

  • project = test
  • user = user@llmgw.com
  • groups = marketing
  • groups = CZE

The header llmgw-nonsense will be ignored since it is not among the allowed entity types.

with AzureOpenAI(
    azure_endpoint="https://<llmgw-deployment-url>",
    api_key=<project_api_key>,
) as client:
    completion = client.chat.completions.create(
        model="gpt4",
        extra_headers={
            "llmgw-user": "user@llmgw.com",
            "llmgw-project": "test",
            "llmgw-nonsense": "something",  # This will be ignored
            "llmgw-groups": "marketing, CZE"
        },
        messages=[
            {
                "role": "user",
                "content": "Tell me a joke, please!",
            },
        ],
    )

Project API Key

Project API keys can be set in admin portal and it is dedicated to a single project entity. Support for additional entity types is by default supported in direct API calls or can be added to admin portal on request.

IMPORTANT: Project api keys in config files are deprecated.

Example

In this example, the project key has been configured for the test project, and the llmgw-user header is used.
The following setup will generate these identities:

with AzureOpenAI(
    azure_endpoint="https://<llmgw-deployment-url>",
    api_key=<project_api_key>,
) as client:
    completion = client.chat.completions.create(
        model="gpt4",
        extra_headers={"llmgw-user": "user@llmgw.com"},
        messages=[
            {
                "role": "user",
                "content": "Tell me a joke, please!",
            },
        ],
    )

User-Based Tokens

A user-based token is uniquely assigned to a specific user email and entity.

Tokens can be generated via /admin/endpoints or through the admin portal.

In this example, a user_based_token has been generated for user@llmgw.com and the project demo.
No extra headers are required—the following request will generate these identities:

with AzureOpenAI(
    azure_endpoint="https://<llmgw-deployment-url>",
    api_key=<user_based_token>,
) as client:
    completion = client.chat.completions.create(
        model="gpt4",
        messages=[
            {
                "role": "user",
                "content": "Tell me a joke, please!",
            },
        ],
    )

Entra ID Token

An Entra ID token can be used for both validation and to carry information about the user.

When a request is authenticated using an Entra ID token, you can configure which entities should be extracted, such as:

  • Mapping the user email to a user entity
  • Mapping Entra ID groups to user groups

Example usage in a request

Note: An LLMGW API key is still required.

with AzureOpenAI(
    azure_endpoint="https://<llmgw-deployment-url>",
    api_key=<your_api_key>,
) as client:
    completion = client.chat.completions.create(
        model="gpt4",
        extra_headers={"X-Entra-Access-Token": "<USER_ENTRA_ID_TOKEN>"},
        messages=[
            {
                "role": "user",
                "content": "Tell me a joke, please!",
            },
        ],
    )