Usage

This section provides comprehensive documentation on how to use the Trustwise SDK, including basic setup, configuration, and detailed examples for evaluating different metrics.

Note

For understanding the concepts of Trustwise metrics, please refer to the Trustwise Metrics Documentation.

Basic Setup (TrustwiseClient)

TrustwiseClient is the recommended client for all new development. It provides access to all platform APIs — agents, guardrails, policies, evaluations, metrics, and more.

import os
from trustwise.sdk.config import TrustwiseConfig
from trustwise.sdk.platform import TrustwiseClient

# Configure using environment variables
config = TrustwiseConfig(api_key=os.environ["TW_API_KEY"])

# Or configure directly
# config = TrustwiseConfig(
#     api_key="your-api-key",
#     base_url="https://api.trustwise.ai"
# )

client = TrustwiseClient(config)

# Evaluate a metric
result = client.metrics.faithfulness(
    query="What is the capital of France?",
    response="The capital of France is Paris.",
    context=[{"chunk_text": "Paris is the capital of France.", "chunk_id": "doc:idx:1"}],
)

# Manage agents
agents = client.agents.list(limit=5)

# Manage guardrails
guardrails = client.guardrails.list()

For full details on all resources, see Platform API Client.

Legacy Setup (TrustwiseSDK)

Deprecated since version ``TrustwiseSDK``: is deprecated. See Migrating from TrustwiseSDK to TrustwiseClient for migration instructions.

import os
from trustwise.sdk import TrustwiseSDK
from trustwise.sdk.config import TrustwiseConfig

# Configure using environment variables
os.environ["TW_API_KEY"] = "your-api-key"
os.environ["TW_BASE_URL"] = "https://api.trustwise.ai"
config = TrustwiseConfig()

# Or configure directly
# config = TrustwiseConfig(
#     api_key="your-api-key",
#     base_url="https://api.trustwise.ai"
# )

# Initialize the SDK
trustwise = TrustwiseSDK(config)

Evaluating Metrics

Below are example requests and responses for evaluating different metrics using the SDK. Each metric exposes an evaluate() endpoint. We will use faithfulness as an example.

Faithfulness Metric

Copy and run this example to evaluate faithfulness:

# Evaluate faithfulness
trustwise.metrics.faithfulness.evaluate(
    query="What is the capital of France?",
    response="The capital of France is Paris.",
    context=[
        {"chunk_text": "Paris is the capital of France.", "chunk_id": "doc:idx:1"}
    ]
)

Output:

score=99.971924 facts=[Fact(statement='The capital of France is Paris.', label='Safe', prob=0.9997192, sentence_span=[0, 30])]

Async Usage

The SDK provides a fully asynchronous interface for high-throughput or concurrent evaluation. The async API mirrors the synchronous API, but requires await and an event loop.

import asyncio
from trustwise.sdk import TrustwiseSDKAsync
from trustwise.sdk.config import TrustwiseConfig

async def main():
    config = TrustwiseConfig()
    trustwise = TrustwiseSDKAsync(config)
    await trustwise.metrics.faithfulness.evaluate(
        query="What is the capital of France?",
        response="The capital of France is Paris.",
        context=[{"chunk_text": "Paris is the capital of France.", "chunk_id": "doc:idx:1"}]
    )

asyncio.run(main())

Note

All metrics are available in both synchronous and asynchronous forms. For async, use TrustwiseSDKAsync and await the evaluate() methods.

Working with JSON Output

# Convert to JSON format for serialization
faithfulness_result.to_json(indent=2)

Output:

{
  "score": 99.971924,
  "facts": [
    {
      "statement": "The capital of France is Paris.",
      "label": "Safe",
      "prob": 0.9997192,
      "sentence_span": [0, 30]
    }
  ]
}

Working with Python Dict Output

# Convert to Python dict for programmatic access
faithfulness_result.to_dict()

Output:

{'score': 99.971924, 'facts': [{'statement': 'The capital of France is Paris.', 'label': 'Safe', 'prob': 0.9997192, 'sentence_span': [0, 30]}]}

Working with Result Properties

# Access individual properties
print(f"Faithfulness score: {faithfulness_result.score}")
for fact in faithfulness_result.facts:
    print(f"Fact: {fact.statement}, Label: {fact.label}, Probability: {fact.prob}, Span: {fact.sentence_span}")

Output:

Faithfulness score: 99.971924
Fact: The capital of France is Paris., Label: Safe, Probability: 0.9997192, Span: [0, 30]

Note

The FaithfulnessResponse (and all other response objects) provides both direct string representation and JSON serialization methods for flexible integration with different workflows.