Skip to content
DeepintShield DeepintShield

Overview

Overview

Section titled “Overview”

DeepintShield provides a Bedrock-compatible endpoint for the Converse and Invoke APIs via protocol adaptation. The integration handles request transformation, response normalization, and error mapping between AWS Bedrock’s API specification and DeepintShield’s internal processing pipeline.

This integration enables you to utilize DeepintShield’s features like governance, load balancing, semantic caching, multi-provider support, and more, all while preserving your existing Bedrock SDK-based architecture.

Endpoint: /bedrock

Setup

Section titled “Setup”

Install with the Bedrock extra:

Terminal window
pip install "deepintshield[bedrock]"
from deepintshield import DeepintShield


shield = DeepintShield(virtual_key="sk-bf-your-virtual-key")
client = shield.bedrock(region_name="us-west-2")  # pre-wired boto3 bedrock-runtime


response = client.converse(
    modelId="anthropic.claude-3-5-sonnet-20240620-v1:0",
    messages=[{"role": "user", "content": [{"text": "Hello!"}]}],
)


print(response)

Provider/Model Usage Examples

Section titled “Provider/Model Usage Examples”

Because Bedrock itself is a multi-provider platform, you can use any Bedrock-supported model ID and still route through DeepintShield. DeepintShield will handle governance, observability, and other cross-cutting concerns.

import boto3
import json


client = boto3.client(
    service_name="bedrock-runtime",
    endpoint_url="http://localhost:8080/bedrock",
    region_name="us-west-2",
    aws_access_key_id="deepintshield-dummy-key",
    aws_secret_access_key="deepintshield-dummy-secret"
)


# Anthropic via Bedrock (Converse API)
anthropic_response = client.converse(
    modelId="anthropic.claude-3-sonnet-20240229",
    messages=[{"role": "user", "content": [{"text": "Hello from Claude!"}]}]
)


# Mistral via Bedrock (Converse API)
mistral_response = client.converse(
    modelId="mistral.mistral-large-2407",
    messages=[{"role": "user", "content": [{"text": "Hello from Mistral!"}]}]
)


# Mistral via Bedrock (Invoke API)
mistral_invoke_response = client.invoke_model(
    modelId="mistral.mistral-large-2407",
    contentType="application/json",
    accept="application/json",
    body=json.dumps({
        "prompt": "Say hello from Mistral using Invoke API.",
        "max_tokens": 50,
        "temperature": 0.7
    }),
)

Adding Custom Headers

Section titled “Adding Custom Headers”

Pass custom headers required by DeepintShield plugins (like governance, telemetry, etc.) using boto3’s event system:

import boto3


def add_bifrost_headers(request, **kwargs):
    """Add custom DeepintShield headers to the request before signing."""
    request.headers.add_header("x-bf-vk", "vk_12345")  # Virtual key for governance
    request.headers.add_header("x-bf-env", "production")  # Environment tag


client = boto3.client(
    service_name="bedrock-runtime",
    endpoint_url="http://localhost:8080/bedrock",
    region_name="us-west-2",
    aws_access_key_id="deepintshield-dummy-key",
    aws_secret_access_key="deepintshield-dummy-secret"
)


# Register the header injection for all Bedrock API calls
client.meta.events.register_first(
    "before-sign.bedrock-runtime.*",
    add_bifrost_headers,
)


# Now make requests with custom headers
response = client.converse(
    modelId="anthropic.claude-3-5-sonnet-20240620-v1:0",
    messages=[{"role": "user", "content": [{"text": "Hello with custom headers!"}]}]
)

Note: Use register_first to ensure headers are added before request signing. The event name format is before-sign.<service-name>.<operation-name>. You need to register for each API operation you plan to use (Converse, ConverseStream, InvokeModel, etc.).

Streaming Examples

Section titled “Streaming Examples”

Converse Stream

Section titled “Converse Stream”

Use converse_stream for chat-based streaming with a unified interface across models.

import boto3


client = boto3.client(
    service_name="bedrock-runtime",
    endpoint_url="http://localhost:8080/bedrock",
    region_name="us-west-2",
    aws_access_key_id="deepintshield-dummy-key",
    aws_secret_access_key="deepintshield-dummy-secret"
)


response = client.converse_stream(
    modelId="anthropic.claude-3-5-sonnet-20240620-v1:0",
    messages=[{"role": "user", "content": [{"text": "Tell me a story about a brave knight."}]}],
    inferenceConfig={"maxTokens": 512, "temperature": 0.5}
)


print("Response:")
for chunk in response["stream"]:
    if "contentBlockDelta" in chunk:
        text = chunk["contentBlockDelta"]["delta"]["text"]
        print(text, end="", flush=True)

Invoke Stream

Section titled “Invoke Stream”

Use invoke_model_with_response_stream for model-specific streaming payloads.

import boto3
import json


client = boto3.client(
    service_name="bedrock-runtime",
    endpoint_url="http://localhost:8080/bedrock",
    region_name="us-west-2",
    aws_access_key_id="deepintshield-dummy-key",
    aws_secret_access_key="deepintshield-dummy-secret"
)


# Example for Claude 3 (Messages API format)
body = json.dumps({
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 1024,
    "messages": [
        {"role": "user", "content": "Write a haiku about coding."}
    ]
})


response = client.invoke_model_with_response_stream(
    modelId="anthropic.claude-3-haiku-20240307-v1:0",
    body=body,
    contentType="application/json",
    accept="application/json"
)


print("Response:")
for event in response.get("body"):
    if "chunk" in event:
        chunk = event["chunk"]
        if "bytes" in chunk:
            # The chunk bytes contain the model-specific JSON response
            result = json.loads(chunk["bytes"].decode("utf-8"))


            # Extract content based on model (e.g., Claude)
            if "delta" in result and "text" in result["delta"]:
                print(result["delta"]["text"], end="", flush=True)
            elif "completion" in result:
                print(result["completion"], end="", flush=True)

Using Direct Keys

Section titled “Using Direct Keys”

Pass AWS credentials or Bedrock API keys directly in requests to bypass DeepintShield’s load balancing. This requires the Allow Direct API keys option to be enabled in DeepintShield configuration.

Learn more: See Key Management for enabling direct API key usage.

When direct keys are enabled, you can pass your AWS credentials directly to the boto3 client instead of using dummy credentials.

import boto3


# When direct keys are enabled, pass real AWS credentials to boto3
client = boto3.client(
    service_name="bedrock-runtime",
    endpoint_url="http://localhost:8080/bedrock",
    region_name="us-west-2",
    aws_access_key_id="your-aws-access-key",  # Real credentials when direct keys enabled
    aws_secret_access_key="your-aws-secret-key"  # Real credentials when direct keys enabled
)


response = client.converse(
    modelId="anthropic.claude-3-5-sonnet-20240620-v1:0",
    messages=[{"role": "user", "content": [{"text": "Hello!"}]}]
)

Note: When using DeepintShield’s configured keys (not direct keys), you must provide dummy AWS credentials (aws_access_key_id and aws_secret_access_key) to the boto3 client. This is because boto3 requires credentials to sign requests, even though DeepintShield will use its own configured keys. The dummy values can be any string (e.g., "deepintshield-dummy-key" and "deepintshield-dummy-secret").

Supported Features

Section titled “Supported Features”

The Bedrock integration currently supports:

  • Converse API (/bedrock/model/{modelId}/converse) for text/chat-style workloads
  • Invoke API (/bedrock/model/{modelId}/invoke) for model-specific text completion workloads
  • Streaming via converse_stream and invoke_model_with_response_stream
  • Tools via toolConfig, toolUse, and toolResult inside Converse requests
  • Image and multimodal responses where supported by the underlying Bedrock model
  • All DeepintShield core features that apply to these flows (governance, load balancing, semantic cache, observability, etc.)

Next Steps

Section titled “Next Steps”