Skip to content

Langchain (Azure Chat OpenAI)

This guide shows how to use Langchain's Azure ChatOpenAI client with Adastra LLMGW. Langchain provides additional abstractions and tools for building AI applications on top of the base OpenAI functionality.

Client Setup

Install the Langchain OpenAI package, which provides Langchain integrations for OpenAI models:

pip install langchain_openai

Configure your endpoint to use the Azure OpenAI client through Langchain:

LLMGW_API_ENDPOINT = "https://<llmgw-deployment-url>/azure-open-ai/"
LLMGW_API_KEY = "<YOUR_LLMGW_API_KEY>"

Note: LLMGW_API_ENDPOINT is the same as it was for Azure OpenAI.

Create the Langchain Azure ChatOpenAI model:

import json
from langchain_openai import AzureChatOpenAI

model = AzureChatOpenAI(
    azure_deployment="gpt-4.1",
    api_version="2025-01-01-preview",
    azure_endpoint=LLMGW_API_ENDPOINT,
    api_key=LLMGW_API_KEY,
    model_kwargs={
        "extra_headers": {"llmgw-project": "your-project", "llmgw-user": "your-user"},
    },
    include_response_headers=True,
)

The extra_headers in model_kwargs allows you to associate metadata such as the project name and user with each request, which may be required based on your configuration. Check with your administrator for specific header requirements.

Making Requests

Make a request using Langchain's simplified interface:

question = "What is the weather like?"
response = model.invoke(question)
print(response.content)

In this example:

  • The azure_deployment parameter should match deployment_name values in your LLMGW config.yaml. It refers to the model group configured in LLMGW (see config docs).
  • Langchain's invoke() method provides a simplified interface compared to the raw OpenAI client.

Accessing Response Metadata

For more detailed information, such as request cost and model information, you can inspect the response metadata. Since we set include_response_headers=True, LLMGW includes custom headers prefixed with x-llmgw.

headers = response.response_metadata.get('headers', {})
llmgw_headers = {key: value for key, value in headers.items() if key.startswith('x-llmgw')}
print(json.dumps(llmgw_headers, indent=2))

The output may look like this:

{
    'x-llmgw-cost': '4e-05',
    'x-llmgw-request-id': '3cb26481-d869-4923-8093-3feb92f8d9fc',
    'x-llmgw-model-id': 'azure-us-gpt35',
    'x-llmgw-attempts': '2'
}

For full details and what headers are included, see Response Headers.

Streaming Responses

Langchain also supports streaming responses for real-time output. You can use the stream() method to get response chunks as they are generated.

question = "Tell me a story"
for chunk in model.stream(question):
    content = chunk.content
    if content:
        print(content, end='')

This provides the same streaming experience as the raw OpenAI client but through Langchain's interface.

Advanced Langchain Features

With Langchain, you can leverage additional features like:

  • Prompt templates for consistent message formatting
  • Chains for complex multi-step operations
  • Agents for autonomous task execution
  • Memory for conversation persistence

For more information on these advanced features, see the Langchain documentation.