Request Identity¶

Request Identity Definition¶

Request identities in LLMGW can be defined in several ways. Before you can use an identity, the corresponding entity type must be configured here so that LLMGW can recognize it.

LLMGW Headers¶

You can specify request identities using custom HTTP headers. Headers with the llmgw- prefix are filtered and checked against the allowed entity types.

For example, the following request will set these identities:

user = user@llmgw.com
groups = marketing
groups = CZE

The header llmgw-nonsense will be ignored since it is not among the allowed entity types.

with AzureOpenAI(
    azure_endpoint="https://<llmgw-deployment-url>",
    api_key=<api_key>,
) as client:
    completion = client.chat.completions.create(
        model="gpt4",
        extra_headers={
            "llmgw-user": "user@llmgw.com",
            "llmgw-nonsense": "something",  # This will be ignored
            "llmgw-groups": "marketing, CZE"
        },
        messages=[
            {
                "role": "user",
                "content": "Tell me a joke, please!",
            },
        ],
    )

Project API Key¶

Project API keys can be set in admin portal and it is dedicated to a single project entity. Support for additional entity types is by default supported in direct API calls or can be added to admin portal on request.

Example¶

In this example, the project key has been configured for the test project, and the llmgw-user header is used.
The following setup will generate these identities:

project = test
user = user@llmgw.com

with AzureOpenAI(
    azure_endpoint="https://<llmgw-deployment-url>",
    api_key=<project_api_key>,
) as client:
    completion = client.chat.completions.create(
        model="gpt4",
        extra_headers={"llmgw-user": "user@llmgw.com"},
        messages=[
            {
                "role": "user",
                "content": "Tell me a joke, please!",
            },
        ],
    )

User-Based Tokens¶

A user-based token is uniquely assigned to a specific user email and entity.

Tokens can be generated via /admin/endpoints or through the admin portal.

In this example, a user_based_token has been generated for user@llmgw.com and the project demo.
No extra headers are required—the following request will generate these identities:

project = demo
user = user@llmgw.com

with AzureOpenAI(
    azure_endpoint="https://<llmgw-deployment-url>",
    api_key=<user_based_token>,
) as client:
    completion = client.chat.completions.create(
        model="gpt4",
        messages=[
            {
                "role": "user",
                "content": "Tell me a joke, please!",
            },
        ],
    )

Identity Precedence¶

A single request can supply the same entity type through more than one method — for example, a request authenticated with a credential that already implies a user while also sending a llmgw-user header. In that case, the identity established by authentication always wins: a llmgw- header is ignored whenever the authenticating credential already provides that entity type. Headers only fill in entity types that authentication does not set.

A project API key sets the project; a user-based token sets the user and project; adding a llmgw-entra-user header to an API key request sets the user and groups from Entra ID (see below). Headers for anything already set this way are ignored:

Authentication	`llmgw-project`	`llmgw-user`	`llmgw-groups`
Project API key	ignored (key wins)	applied	applied
Project API key + `llmgw-entra-user`	ignored (key wins)	ignored (Entra wins)	ignored (Entra wins)
User-based token	ignored (token wins)	ignored (token wins)	applied

Entra User Header¶

The llmgw-entra-user header can be used to fetch additional user information from Entra ID.

When a request contains this header with a user's email address, the following actions are performed via the Microsoft Graph API:

Look up the user in Entra ID
Fetch the user's group memberships (only groups imported into the Admin portal are tracked for the request)

To enable this functionality Graph API must be configured, see the LLMGW environment configuration.

Example usage in a request

Note: An LLMGW API key is still required.

with AzureOpenAI(
    azure_endpoint="https://<llmgw-deployment-url>",
    api_key=<your_api_key>,
) as client:
    completion = client.chat.completions.create(
        model="gpt4",
        extra_headers={"llmgw-entra-user": "user@company.com"},
        messages=[
            {
                "role": "user",
                "content": "Tell me a joke, please!",
            },
        ],
    )