Request Identity¶

Request Identity Definition¶

Request identities in LLMGW can be defined in several ways. Before you can use an identity, the corresponding entity type must be configured here so that LLMGW can recognize it.

LLMGW Headers¶

You can specify request identities using custom HTTP headers. Headers with the llmgw- prefix are filtered and checked against the allowed entity types.

For example, the following request will set these identities:

user = user@llmgw.com
groups = marketing
groups = CZE

The header llmgw-nonsense will be ignored since it is not among the allowed entity types.

with AzureOpenAI(
    azure_endpoint="https://<llmgw-deployment-url>",
    api_key=<api_key>,
) as client:
    completion = client.chat.completions.create(
        model="gpt4",
        extra_headers={
            "llmgw-user": "user@llmgw.com",
            "llmgw-nonsense": "something",  # This will be ignored
            "llmgw-groups": "marketing, CZE"
        },
        messages=[
            {
                "role": "user",
                "content": "Tell me a joke, please!",
            },
        ],
    )

Project API Key¶

Project API keys can be set in admin portal and it is dedicated to a single project entity. Support for additional entity types is by default supported in direct API calls or can be added to admin portal on request.

Example¶

In this example, the project key has been configured for the test project, and the llmgw-user header is used.
The following setup will generate these identities:

project = test
user = user@llmgw.com

with AzureOpenAI(
    azure_endpoint="https://<llmgw-deployment-url>",
    api_key=<project_api_key>,
) as client:
    completion = client.chat.completions.create(
        model="gpt4",
        extra_headers={"llmgw-user": "user@llmgw.com"},
        messages=[
            {
                "role": "user",
                "content": "Tell me a joke, please!",
            },
        ],
    )

User-Based Tokens¶

A user-based token is uniquely assigned to a specific user email and entity.

Tokens can be generated via /admin/endpoints or through the admin portal.

In this example, a user_based_token has been generated for user@llmgw.com and the project demo.
No extra headers are required—the following request will generate these identities:

project = demo
user = user@llmgw.com

with AzureOpenAI(
    azure_endpoint="https://<llmgw-deployment-url>",
    api_key=<user_based_token>,
) as client:
    completion = client.chat.completions.create(
        model="gpt4",
        messages=[
            {
                "role": "user",
                "content": "Tell me a joke, please!",
            },
        ],
    )

User OpenID header¶

User OpenID header llmgw-user-openid-id can be used to carry information about the user. At the moment we support only Entra authentication might be extended in the future.

When a request is authenticated using this header, following info will be extracted using Microsoft Graph API:

User email
User groups (groups must be imported to Admin portal in order to be tracked)

To enable this functionality Graph API must be configured, see the LLMGW environment configuration.

Example usage in a request

Note: An LLMGW API key is still required.

with AzureOpenAI(
    azure_endpoint="https://<llmgw-deployment-url>",
    api_key=<your_api_key>,
) as client:
    completion = client.chat.completions.create(
        model="gpt4",
        extra_headers={"llmgw-user-openid-id": "<USER_ENTRA_ID_OID>"},
        messages=[
            {
                "role": "user",
                "content": "Tell me a joke, please!",
            },
        ],
    )