Prometheus

Overview

DeepIntShield exposes Prometheus metrics via two methods:

1. Pull-based (Scraping): Traditional /metrics endpoint that Prometheus can scrape
2. Push-based (Push Gateway): Push metrics to a Prometheus Push Gateway for cluster deployments

Note

For multi-node deployments: Use the Push Gateway method to ensure accurate metric aggregation. Traditional scraping may miss nodes behind load balancers.

---

Pull-based Scraping

DeepIntShield automatically exposes a /metrics endpoint when the telemetry plugin is enabled (enabled by default). No additional configuration is needed.

Note

When DeepIntShield’s authentication is enabled (auth_config.is_enabled = true), the /metrics endpoint requires Basic auth credentials. You must include the same admin_username and admin_password from your auth_config in the Prometheus scrape configuration. Without this, Prometheus will receive 401 Unauthorized responses and scraping will silently fail.

Prometheus Configuration

Add DeepIntShield to your Prometheus prometheus.yml:

scrape_configs:
  - job_name: 'deepintshield'
    static_configs:
      - targets: ['deepintshield-host:8080']
    scrape_interval: 15s

If DeepIntShield authentication is enabled, add basic_auth to your scrape config:

scrape_configs:
  - job_name: 'deepintshield'
    static_configs:
      - targets: ['deepintshield-host:8080']
    scrape_interval: 15s
    basic_auth:
      username: '<admin_username>'
      password: '<admin_password>'

Endpoint

GET /metrics

Returns metrics in Prometheus exposition format.

---

Push-based (Push Gateway)

For multi-node cluster deployments, the Prometheus plugin pushes metrics to a Prometheus Push Gateway. This ensures all nodes’ metrics are captured regardless of load balancer routing.

Configuration

Field	Type	Required	Default	Description
push_gateway_url	string	✅ Yes	-	Push Gateway URL (e.g., http://pushgateway:9091)
job_name	string	❌ No	deepintshield	Job label for pushed metrics
instance_id	string	❌ No	hostname	Instance identifier for metric grouping
push_interval	integer	❌ No	15	Push interval in seconds (1-300)
basic_auth	object	❌ No	-	Basic auth credentials

Basic Auth Configuration

Field	Type	Required	Description
username	string	✅ Yes	Basic auth username
password	string	✅ Yes	Basic auth password

---

Setup

UI

1. Navigate to Observability → Prometheus in the DeepIntShield UI
2. The /metrics endpoint is shown at the top for scraping configuration
3. To enable Push Gateway:
   • Enter the Push Gateway URL
   • Configure Job Name and Push Interval as needed
   • Optionally set a custom Instance ID
   • Enable Basic Authentication if required
   • Toggle Enable Push Gateway on
   • Click Save Prometheus Configuration

Config File

{
  "plugins": [
    {
      "name": "prometheus",
      "enabled": true,
      "config": {
        "push_gateway_url": "http://pushgateway:9091",
        "job_name": "deepintshield",
        "push_interval": 15
      }
    }
  ]
}

With Basic Auth

{
  "plugins": [
    {
      "name": "prometheus",
      "enabled": true,
      "config": {
        "push_gateway_url": "http://pushgateway:9091",
        "job_name": "deepintshield",
        "push_interval": 15,
        "instance_id": "deepintshield-node-1",
        "basic_auth": {
          "username": "admin",
          "password": "secret"
        }
      }
    }
  ]
}

---

Available Metrics

The following metrics are available from both the /metrics endpoint and Push Gateway:

HTTP Metrics

Metric	Type	Description
http_requests_total	Counter	Total HTTP requests by path, method, status
http_request_duration_seconds	Histogram	HTTP request latency
http_request_size_bytes	Histogram	Request body size
http_response_size_bytes	Histogram	Response body size

DeepIntShield LLM Metrics

Metric	Type	Description
bifrost_upstream_requests_total	Counter	Total requests to LLM providers
bifrost_upstream_latency_seconds	Histogram	Provider request latency
bifrost_success_requests_total	Counter	Successful provider requests
bifrost_error_requests_total	Counter	Failed provider requests
bifrost_input_tokens_total	Counter	Total input tokens processed
bifrost_output_tokens_total	Counter	Total output tokens generated
bifrost_cost_total	Counter	Total cost in USD
bifrost_cache_hits_total	Counter	Cache hits by type
bifrost_stream_first_token_latency_seconds	Histogram	Time to first token (streaming)
bifrost_stream_inter_token_latency_seconds	Histogram	Inter-token latency (streaming)

Default Labels

All DeepIntShield metrics include these labels:

• provider - LLM provider name
• model - Model identifier
• method - Request type (chat, completion, embedding, etc.)
• virtual_key_id / virtual_key_name - Virtual key identifiers
• selected_key_id / selected_key_name - Actual key used
• number_of_retries - Retry count
• fallback_index - Fallback position
• team_id / team_name - Team identifiers (if governance enabled)
• customer_id / customer_name - Customer identifiers (if governance enabled)

---

Push Gateway Setup

If you don’t have a Push Gateway running, deploy one:

Docker

docker run -d -p 9091:9091 prom/pushgateway

Kubernetes (Helm)

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install pushgateway prometheus-community/prometheus-pushgateway

Configure Prometheus to Scrape Push Gateway

Add to your prometheus.yml:

scrape_configs:
  - job_name: 'pushgateway'
    honor_labels: true
    static_configs:
      - targets: ['pushgateway:9091']

Note

The honor_labels: true setting is important - it preserves the job and instance labels pushed by DeepIntShield instead of overwriting them with the Push Gateway’s labels.

---

Pull vs Push: When to Use Each

Scenario	Recommended Method
Single DeepIntShield instance	Pull (scraping)
Multiple instances, direct access	Pull (scraping)
Multiple instances behind load balancer	Push (Push Gateway)
Kubernetes with service mesh	Pull or Push
Serverless / ephemeral instances	Push (Push Gateway)

Why Push for Clusters?

When multiple DeepIntShield instances run behind a load balancer:

1. Scraping randomness: Each scrape may hit different nodes, missing metrics from others
2. Instance tracking: Push Gateway properly tracks per-instance metrics via instance label
3. Aggregation: Downstream tools (Grafana, Datadog) can aggregate across all instances

---

Troubleshooting

Push Gateway Connection Failed

failed to push metrics to push gateway: connection refused

• Verify the Push Gateway URL is correct and reachable from DeepIntShield
• Check firewall rules between DeepIntShield and Push Gateway
• Ensure Push Gateway is running: curl http://pushgateway:9091/metrics

Metrics Not Appearing

• Verify the telemetry plugin is enabled (required for metrics collection)
• Check DeepIntShield logs for push errors
• Verify Prometheus is scraping the Push Gateway with honor_labels: true

Authentication Failed

• Double-check username and password
• Ensure basic auth is configured on the Push Gateway side
• Check for special characters that may need escaping