Skip to content
DeepintShield DeepintShield

LiteLLM SDK

Since LiteLLM already provides multi-provider abstraction, DeepintShield adds enterprise features like governance, semantic caching, MCP tools, observability, etc, on top of your existing setup.

Endpoint: /litellm

Setup

Section titled “Setup”

Install with the LiteLLM extra:

Terminal window
pip install "deepintshield[litellm]"
from deepintshield import DeepintShield


shield = DeepintShield(virtual_key="sk-bf-your-virtual-key")
litellm = shield.litellm()


response = litellm.completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
)


print(response.choices[0].message.content)

Provider/Model Usage Examples

Section titled “Provider/Model Usage Examples”

Your existing LiteLLM provider switching works unchanged through DeepintShield:

from litellm import completion


# All your existing LiteLLM patterns work the same
base_url = "http://localhost:8080/litellm"


# OpenAI models
openai_response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello GPT!"}],
    base_url=base_url
)


# Anthropic models
anthropic_response = completion(
    model="claude-3-sonnet-20240229",
    messages=[{"role": "user", "content": "Hello Claude!"}],
    base_url=base_url
)


# Google models
google_response = completion(
    model="gemini/gemini-1.5-flash",
    messages=[{"role": "user", "content": "Hello Gemini!"}],
    base_url=base_url
)


# Azure models
azure_response = completion(
    model="azure/gpt-4o",
    messages=[{"role": "user", "content": "Hello Azure!"}],
    base_url=base_url
)

Adding Custom Headers

Section titled “Adding Custom Headers”

Add DeepintShield-specific headers for governance and tracking:

from litellm import completion


# Add custom headers for DeepintShield features
response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
    base_url="http://localhost:8080/litellm",
    extra_headers={
        "x-bf-vk": "your-virtual-key",          # Virtual key for governance
    }
)


print(response.choices[0].message.content)

Using Direct Keys

Section titled “Using Direct Keys”

Pass API keys directly to bypass DeepintShield’s key management. You can pass any provider’s API key since DeepintShield only looks for Authorization or x-api-key headers. This requires the Allow Direct API keys option to be enabled in DeepintShield configuration.

Learn more: See Key Management for enabling direct API key usage.

from litellm import completion


# Using OpenAI key directly
openai_response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello GPT!"}],
    base_url="http://localhost:8080/litellm",
    extra_headers={
        "Authorization": "Bearer sk-your-openai-key"
    }
)


# Using Anthropic key for Claude models
anthropic_response = completion(
    model="claude-3-sonnet-20240229",
    messages=[{"role": "user", "content": "Hello Claude!"}],
    base_url="http://localhost:8080/litellm",
    extra_headers={
        "x-api-key": "sk-ant-your-anthropic-key"
    }
)


# Using Azure with direct Azure key
import os


deployment = os.getenv("AZURE_OPENAI_DEPLOYMENT", "my-azure-deployment")
model = f"azure/{deployment}"


azure_response = completion(
    model=model,
    messages=[{"role": "user", "content": "Hello from LiteLLM (Azure demo)!"}],
    base_url="http://localhost:8080/litellm",
    api_key=os.getenv("AZURE_API_KEY", "your-azure-api-key"),
    deployment_id=os.getenv("AZURE_OPENAI_DEPLOYMENT", "gpt-4o-aug"),
    max_tokens=100,
    extra_headers={
        "x-bf-azure-endpoint": "https://your-resource.openai.azure.com",
    }
)

Supported Features

Section titled “Supported Features”

The LiteLLM integration supports all features that are available in both the LiteLLM SDK and DeepintShield core functionality. Your existing LiteLLM code works seamlessly with DeepintShield’s enterprise features. 😄

Next Steps

Section titled “Next Steps”