Skip to content

LLM Gateway MODEL ANALYTICS

The Model Analytics dashboard provides comprehensive insights into AI model usage, costs, and performance across your LLM Gateway deployment.

Model Analytics Dashboard - Overview

Dashboard Components

  • Spend per model - Shows cost breakdown by individual AI model, allowing you to identify which specific models are generating the highest expenses (for example azure-uk-gpt4o),

  • Spend per model group - Displays aggregated spending by model groups, providing a higher-level view of cost distribution across different AI model types (for example gpt-4o),

  • Call duration - Tracks response time patterns for LLM Gateway requests over time, helping identify performance trends and potential latency issues.

  • Model call duration - Shows LLM response times over time for different AI models across various regions.

  • Call vs model call duration - For each request, compares the overall LLM Gateway response time with the response time from the LLM provider.

  • Response codes per model (10min interval) - Shows HTTP response status codes for different models over time, tracking success/failure rates in 10-minute intervals.

  • Response code per model - Table showing HTTP response codes and total request counts for each AI model, with all models.

Filtering Options

The dashboard provides two key filtering mechanisms to help you analyze your AI model usage at different levels of granularity:

Models Filter

Filtering by Model

  • Select specific individual models to focus analysis on,
  • Narrows all dashboard metrics to show data only for chosen models,
  • Useful for detailed cost and performance analysis of specific models.

Models Group Filter

Filtering by Model group

  • Filter by model groups (e.g., gpt-4o, anthropic.claude-3-5-sonnet),
  • Provides broader view across different AI model groups,
  • Helps analyze spending patterns at the model group level.

Note: If you select a model and model group that are not from the same origin, you will get no data in the dashboard. Ensure your filter selections are compatible.