OpenAI¶

Use the standard OpenAI client for broad API compatibility without Azure-specific configuration.

Installation¶

Install the OpenAI Python package:

pip install openai

Basic Setup¶

Configure your OpenAI client:

from openai import OpenAI

client = OpenAI(
    base_url="https://<llmgw-deployment-url>/openai/v1",
    api_key="https://<llmgw-deployment-url>/<YOUR_LLMGW_API_KEY>",
    default_headers={
        "llmgw-project": "your-project",
        "llmgw-user": "your-user"
    }
)

The default_headers parameter associates metadata with each request. Check with your administrator if these are required.

Chat Completions¶

Generate text responses to user messages:

response = client.chat.completions.create(
    model="gpt-4.1",
    messages=[
        {
            "role": "user",
            "content": "What is the weather like?"
        }
    ]
)

print(response.choices[0].message.content)

Embeddings¶

Create vector representations of text for semantic search:

embedding = client.embeddings.create(
    model="text-embedding-3-large",
    input="Text to embed"
)

vector = embedding.data[0].embedding
print(f"Embedding dimension: {len(vector)}")

Streaming¶

Get responses in real-time as they're generated:

response = client.chat.completions.create(
    model="gpt-4.1",
    messages=[{"role": "user", "content": "Tell me a story"}],
    stream=True
)

for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Accessing Response Metadata¶

Track costs, request IDs, and which model was used:

import json

response = client.chat.with_raw_response.completions.create(
    model="gpt-4.1",
    messages=[{"role": "user", "content": "What is the weather like?"}]
)

# Extract LLMGW headers
llmgw_headers = {
    key: value
    for key, value in response.headers.items()
    if key.startswith('x-llmgw')
}

print(json.dumps(llmgw_headers, indent=2))

For full details and what headers are included, see Response Headers.

Using AWS Bedrock Models¶

The OpenAI client supports calling AWS Bedrock models like Claude:

response = client.chat.completions.create(
    model="anthropic.claude-sonnet-4-5-20250929-v1:0",
    messages=[
        {
            "role": "user",
            "content": "Tell me a story"
        }
    ]
)

print(response.choices[0].message.content)

List Available Models¶

Get all available models through the /models endpoint:

curl "https://<llmgw-deployment-url>/openai/models"

Returns a list compatible with the OpenAI API format.