API Reference

This section provides detailed technical documentation for the Trustwise SDK. For conceptual information and best practices, see the individual metric sections.

SDK

class trustwise.sdk.TrustwiseSDK(config)[source]

Bases: object

Main SDK class for Trustwise API access.

Provides access to all metrics through version-specific paths.

Parameters:

config (TrustwiseConfig)

__init__(config)[source]

Initialize the Trustwise SDK with path-based versioning support.

Parameters:

config (TrustwiseConfig) – Trustwise configuration instance.

Return type:

None

get_beta_features()[source]

Get the set of features currently in beta.

Returns:

Set of feature names that are in beta

Return type:

Set[str]

get_versions()[source]

Get the available API versions for the metrics.

Return type:

dict[str, list[str]]

Returns:

Dictionary mapping ‘metrics’ to its available versions. Example: {“metrics”: [“v3”]}

guardrails(thresholds, *, block_on_failure=False, callbacks=None)[source]

Create a guardrail system for response evaluation.

Parameters:

  • thresholds (dict[str, float]) – Dictionary mapping metrics to threshold values.

  • block_on_failure (bool) – Whether to block responses that fail checks.

  • callbacks (dict[str, Callable] | None) – Dictionary mapping metrics to callback functions.

Return type:

Guardrail

Returns:

A configured Guardrail instance.

Raises:

ValueError – If any metric in thresholds is not supported by the current API version.

is_beta_feature(feature_name)[source]

Check if a feature is currently in beta.

Parameters:

feature_name (str) – Name of the feature to check

Returns:

True if the feature is in beta, False otherwise

Return type:

bool

class trustwise.sdk.TrustwiseSDKAsync(config)[source]

Bases: object

Async SDK entrypoint for Trustwise. Use this class to access async metrics.

Parameters:

config (TrustwiseConfig)

__init__(config)[source]

Initialize the Trustwise SDK with path-based versioning support.

Parameters:

config (TrustwiseConfig) – Trustwise configuration instance.

Return type:

None

Configuration

Simple configuration for Trustwise API with environment variable support.

class trustwise.sdk.config.TrustwiseConfig(**data)[source]

Bases: BaseModel

Configuration for Trustwise API.

This class handles configuration for the Trustwise API, including authentication and endpoint URLs. It supports both direct initialization and environment variables.

Parameters:

  • api_key (str, Optional) – API key for authentication. If not provided, will be read from TW_API_KEY environment variable. Required for API access.

  • base_url (str, Optional) – Base URL for API endpoints. If not provided, will be read from TW_BASE_URL environment variable or default to https://api.trustwise.ai.

Environment Variables:

Example

>>> from trustwise.sdk.config import TrustwiseConfig
>>> # Using environment variables
>>> config = TrustwiseConfig()
>>> # Using direct initialization
>>> config = TrustwiseConfig(api_key="your-api-key", base_url="https://api.trustwise.ai")
Raises:

ValueError – If API key is missing or invalid, or if base URL is invalid

Parameters:

  • api_key (str)

  • base_url (str)

api_key: str
__init__(**data)[source]

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Return type:

None

base_url: str
_validate_url()[source]

Validate base URL format and scheme.

Return type:

None

get_safety_url(version)[source]

Get safety API URL for specified version.

Return type:

str

Parameters:

version (str)

get_alignment_url(version)[source]

Get alignment API URL for specified version.

Return type:

str

Parameters:

version (str)

get_performance_url(version)[source]

Get performance API URL for specified version.

Return type:

str

Parameters:

version (str)

get_metrics_url(version)[source]

Get metrics API URL for specified version.

Return type:

str

Parameters:

version (str)

Types

class trustwise.sdk.types.SDKBaseModel(**data)[source]

Bases: BaseModel

Base model for all SDK types.

to_json(**kwargs)[source]

Return a JSON string representation of the model. Ensures valid JSON output regardless of Pydantic version. Always excludes None fields by default.

Return type:

str

Parameters:

kwargs (dict[str, object])

to_dict()[source]

Convert to dictionary.

Return type:

dict

_get_field_description(field_name, field_info=None)[source]

Get the description for a field from its field info.

Return type:

str

Parameters:

  • field_name (str)

  • field_info (object | None)

_format_output(indent=0)[source]

Format the model’s output with proper indentation and descriptions. Used by both __repr__ and __str__ to ensure consistent output.

Return type:

str

Parameters:

indent (int)

_format_nested_dict(data, model_class=None, indent=0)[source]

Format a nested dictionary with field descriptions. If model_class is provided, it will be used to look up field descriptions.

Return type:

list[str]

Parameters:
__repr__()[source]

Return a detailed string representation.

Return type:

str

_format_json_value(value)[source]

Format a value for JSON display with syntax highlighting.

Return type:

str

Parameters:

value (object)

classmethod validate_score_range(v, min_value, max_value, label)[source]
Return type:

object

Parameters:
static format_validation_error(model_cls, validation_error)[source]

Format a Pydantic ValidationError into a user-friendly error message using field types and descriptions. Distinguishes between missing and invalid arguments, and sets the error prefix accordingly.

Return type:

str

Parameters:
_repr_mimebundle_(include=None, exclude=None)[source]

Return multiple representations of the object. This is the preferred way to provide multiple representations in Jupyter.

Return type:

dict[str, str]

Parameters:
_format_html(indent_level=0)[source]

Format the model’s output as HTML with proper styling. Used by __repr_html__ to create a pretty representation in Jupyter notebooks.

Return type:

str

Parameters:

indent_level (int)

_get_value_css_class(value)[source]

Return the appropriate CSS class for a value based on its type.

Return type:

str

Parameters:

value (object)

_format_dict_html(data, indent_level=0)[source]

Format a dictionary as HTML with proper styling and field descriptions.

Return type:

str

Parameters:
__repr_html__()[source]

Return an HTML representation for Jupyter notebooks.

Return type:

str

class trustwise.sdk.types.Fact(**data)[source]

Bases: SDKBaseModel

A verified fact extracted from a response.

Parameters:

  • statement (str) – The extracted statement from the response text that represents a ‘atomically’ fact

  • label (str) – The label indicating the fact’s hallucination status. One of (‘Safe’, ‘Unsafe’, ‘Intrinsic Hallucination’, ‘Extrinsic Hallucination’)

  • prob (float) – The associated probability of the label (0-1)

  • sentence_span (list) – A list of two integers [start, end] indicating the character positions of this statement in the response text

statement: str
label: str
prob: float
sentence_span: list[int]
trustwise.sdk.types.Facts

A list of Fact objects representing verified statements from a response.

alias of list[Fact]

class trustwise.sdk.types.FaithfulnessResponse(**data)[source]

Bases: SDKBaseModel

Response type for faithfulness evaluation.

Faithfulness measures how well the response adheres to the provided context. A higher score indicates better alignment with the source material.

Parameters:

  • score (float) – Faithfulness score (0-100). A higher score, indicating a better response, is correlated with a higher confidence in each statement being true, and also a higher proportion of statements being true with respect to the context.

  • facts (list) – List of extracted ‘atomic’ facts from the response, each containing the statement, label, associated probability, and location in the text.

score: float
facts: list[Fact]
class trustwise.sdk.types.AnswerRelevancyResponse(**data)[source]

Bases: SDKBaseModel

Response type for answer relevancy evaluation.

Answer relevancy measures how well the response addresses the specific query. A higher score indicates the response is more directly relevant to the question asked.

Parameters:

  • score (float) – Answer relevancy score (0-100). A higher score, indicating a better response, is correlated with a higher confidence in the response being relevant to the query.

  • generated_question (str) – The generated question for which the response would be relevant.

score: float
generated_question: str
class trustwise.sdk.types.ContextNode(**data)[source]

Bases: SDKBaseModel

A single context node with its metadata.

Context nodes represent individual pieces of information that can be used to evaluate the faithfulness and relevance of responses.

Parameters:

  • node_id (str) – The unique identifier for the context node.

  • node_score (float) – The score associated with the node.

  • node_text (str) – The text content of the node.

node_id: str
node_score: float
node_text: str
trustwise.sdk.types.Context

A list of ContextNode objects representing the context for evaluation.

alias of list[ContextNode]

class trustwise.sdk.types.ContextRelevancyRequest(**data)[source]

Bases: SDKBaseModel

Request type for context relevancy evaluation.

Parameters:

  • query (str) – The input query string.

  • context (list) – A non-empty list of ContextNode objects.

query: str
context: list[ContextNode]
class trustwise.sdk.types.ContextRelevancyResponse(**data)[source]

Bases: SDKBaseModel

Response type for context relevancy evaluation.

Parameters:

  • score (float) – Context relevancy score (0-100). Combination of the topics scores, where the higher score indicates a higher relevance of the retrieved context to the query.

  • topics (list) – List of topics identified from the query.

  • scores (list) – List of scores for each topic (0-100). Highest score for a topic compared across all context nodes is taken as the topic score.

score: float
topics: list[str]
scores: list[float]
classmethod check_scores(v)[source]
Return type:

list[float]

Parameters:

v (list[float])

class trustwise.sdk.types.SummarizationRequest(**data)[source]

Bases: SDKBaseModel

Request type for summarization evaluation.

Summarization quality measures how well a response captures the key points from the provided context without adding extraneous information.

Parameters:

  • response (str) – The LLM response to evaluate.

  • context (list) – A non-empty list of ContextNode objects.

response: str
context: list[ContextNode]
class trustwise.sdk.types.SummarizationResponse(**data)[source]

Bases: SDKBaseModel

Response type for summarization quality evaluation.

Summarization quality measures how well the response captures and represents the key information from the source context.

Parameters:

score (float) – Summarization score (0-100). A higher score indicates a higher adherence of the response to the context.

score: float
class trustwise.sdk.types.PIIEntity(**data)[source]

Bases: SDKBaseModel

A detected piece of personally identifiable information.

PII entities represent sensitive information that has been identified in text, including its location and category.

Parameters:

  • interval (list) – The [start, end] indices of the PII in the text.

  • string (str) – The detected PII string.

  • category (str) – The PII category.

interval: list[int]
string: str
category: str
class trustwise.sdk.types.PIIRequest(**data)[source]

Bases: SDKBaseModel

Request type for PII detection.

PII detection identifies personally identifiable information in text based on customizable allowlists and blocklists.

Parameters:

  • text (str) – The text to evaluate for PII detection.

  • allowlist (list) – (Optional) List of allowed PII categories.

  • blocklist (list) – List of blocked PII categories.

text: str
allowlist: list[str]
blocklist: list[str]
class trustwise.sdk.types.PIIResponse(**data)[source]

Bases: SDKBaseModel

Response type for PII detection.

Contains all detected PII entities with their locations and categories for further processing or filtering.

Parameters:

identified_pii (list) – List of detected PII occurrences.

identified_pii: list[PIIEntity]
class trustwise.sdk.types.PromptInjectionRequest(**data)[source]

Bases: SDKBaseModel

Request type for prompt injection detection.

Prompt injection detection identifies attempts to manipulate AI systems through specially crafted queries or instructions.

Parameters:

query (str) – The input query string to evaluate for prompt injection.

query: str
class trustwise.sdk.types.PromptInjectionResponse(**data)[source]

Bases: SDKBaseModel

Response type for prompt injection detection.

Higher scores indicate a greater likelihood that the query contains prompt injection attempts that could manipulate the AI system.

Parameters:

score (float) – Prompt injection score (0-100). A higher score indicates a higher likelihood of the query being used as a prompt injection.

score: float
class trustwise.sdk.types.ClarityRequest(**data)[source]

Bases: SDKBaseModel

Request type for clarity evaluation.

Clarity measures how well-written and understandable a response is, considering factors like sentence structure, vocabulary, and coherence.

Parameters:

response (str) – The response text to evaluate for clarity.

response: str
class trustwise.sdk.types.ClarityResponse(**data)[source]

Bases: SDKBaseModel

Response type for clarity evaluation.

Clarity scores indicate how clear and understandable the response is. Higher scores suggest better readability and comprehension.

Parameters:

score (float) – A score from 0-100 indicating how clear and understandable the response is. Higher scores indicate better clarity of the response.

score: float
__str__()[source]

Return a simple string representation focusing on the score.

Return type:

str

class trustwise.sdk.types.HelpfulnessRequest(**data)[source]

Bases: SDKBaseModel

Request type for helpfulness evaluation.

Helpfulness measures how useful and actionable a response is, considering factors like completeness, accuracy, and practical value.

Parameters:

response (str) – The response text to evaluate for helpfulness.

response: str
class trustwise.sdk.types.HelpfulnessResponse(**data)[source]

Bases: SDKBaseModel

Response type for helpfulness evaluation.

Helpfulness scores indicate how useful and actionable the response is. Higher scores suggest the response provides valuable, complete information.

Parameters:

score (float) – Helpfulness score (0-100). A higher score indicates a higher confidence in the response being useful, detailed, and unambiguous.

score: float
class trustwise.sdk.types.FormalityRequest(**data)[source]

Bases: SDKBaseModel

Request type for formality evaluation.

Formality measures the level of formality in written text, distinguishing between casual, conversational, and formal writing styles.

Parameters:

response (str) – The response text to evaluate for formality.

response: str
class trustwise.sdk.types.FormalityResponse(**data)[source]

Bases: SDKBaseModel

Response type for formality evaluation.

Formality scores indicate the level of formality in the text. Higher scores indicate more formal writing, lower scores indicate more casual writing.

Parameters:

  • score (float) – Formality score (0-100). The Formality metric judges how formal a piece of text is. A higher score indicates the text is more formal, and a lower score indicates the text is more informal.

  • sentences (list) – List of sentences analyzed.

  • scores (list) – List of scores for each sentence (0-100).

score: float
sentences: list[str]
scores: list[float]
classmethod check_scores(v)[source]
Return type:

list[float]

Parameters:

v (list[float])

class trustwise.sdk.types.SimplicityRequest(**data)[source]

Bases: SDKBaseModel

Request type for simplicity evaluation.

Simplicity measures how easy a response is to understand, considering factors like sentence length, vocabulary complexity, and structure.

Parameters:

response (str) – The response text to evaluate for simplicity.

response: str
class trustwise.sdk.types.SimplicityResponse(**data)[source]

Bases: SDKBaseModel

Response type for simplicity evaluation.

Simplicity scores indicate how easy the response is to understand. Higher scores suggest simpler, more accessible language and structure.

Parameters:

score (float) – Simplicity score (0-100). A higher score indicates a simpler response, with fewer words and simpler language which is easier to understand.

score: float
class trustwise.sdk.types.SensitivityResponse(**data)[source]

Bases: SDKBaseModel

Response type for sensitivity evaluation.

Sensitivity evaluation checks for the presence of specific topics or themes in the text, providing scores for each requested topic.

Parameters:

scores (dict) – Mapping of user-defined topics to sensitivity score (0-100). A high score for a topic indicates that the topic is present in the text.

scores: dict[str, float]
classmethod check_scores(v)[source]
Return type:

dict[str, float]

Parameters:

v (dict[str, float])

class trustwise.sdk.types.ToxicityRequest(**data)[source]

Bases: SDKBaseModel

Request type for toxicity evaluation.

Toxicity evaluation identifies harmful or inappropriate content across multiple categories like threats, insults, and hate speech.

Parameters:

response (str) – The LLM response to evaluate.

response: str
class trustwise.sdk.types.ToxicityResponse(**data)[source]

Bases: SDKBaseModel

Response type for toxicity evaluation.

Toxicity scores indicate the presence of harmful content across different categories. Higher scores indicate more toxic content, so lower scores are preferred.

Parameters:

  • labels (list) – List of toxicity category labels. Supported labels: (toxix, threat, obscene, insult, identity_hate)

  • scores (list) – List of scores for each label (0-100). A higher score indicates the text is more toxic, and thus a lower score is preferred.

labels: list[str]
scores: list[float]
classmethod check_scores(v)[source]
Return type:

list[float]

Parameters:

v (list[float])

class trustwise.sdk.types.ToneRequest(**data)[source]

Bases: SDKBaseModel

Request type for tone evaluation.

Tone evaluation identifies the emotional tone and sentiment expressed in the text across multiple emotional categories.

Parameters:

response (str) – The response text to evaluate for tone.

response: str
class trustwise.sdk.types.ToneResponse(**data)[source]

Bases: SDKBaseModel

Response type for tone evaluation.

Tone analysis identifies the emotional tones present in the text, with confidence scores for each detected tone category.

Parameters:

  • labels (list) – List of identified tones in the response, ordered by their scores. Supported tones: (amusement, anger, caring, confusion, curiosity, disappointment, disgust, embarrassment, excitement, fear, happiness, nervousness, realization, relief, remorse, sadness, surprise, neutral)

  • scores (list) – Corresponding confidence scores (0-100) for each identified tone. Higher scores indicate stronger presence of that tone.

labels: list[str]
scores: list[float]
classmethod check_scores(v)[source]
Return type:

list[float]

Parameters:

v (list[float])

validate_lists()[source]

Validate that labels and scores lists have the same length.

Return type:

ToneResponse

class trustwise.sdk.types.CostResponse(**data)[source]

Bases: SDKBaseModel

Response type for cost evaluation.

Parameters:

  • cost_estimate_per_run (float) – Represents the expense for each individual LLM call. This is the cost of the model call, including the prompt and response tokens.

  • total_project_cost_estimate (float) – Cumulative cost for 10,000 LLM calls to help estimate costs over high-volume usage patterns.

cost_estimate_per_run: float
total_project_cost_estimate: float
class trustwise.sdk.types.CarbonResponse(**data)[source]

Bases: SDKBaseModel

Response type for carbon emissions evaluation.

Parameters:

  • carbon_emitted (float) – The carbon emissions associated with the user specified processor (kg CO2e)

  • sci_per_api_call (float) – Amount of carbon dioxide (CO2 in KG equivalent) emissions generated per individual LLM call.

  • sci_per_10k_calls (float) – Cumulative carbon emissions associated with 10,000 LLM calls.

carbon_emitted: float
sci_per_api_call: float
sci_per_10k_calls: float
class trustwise.sdk.types.FaithfulnessRequest(**data)[source]

Bases: SDKBaseModel

Request type for faithfulness evaluation.

Parameters:

  • query (str) – The input query string.

  • response (str) – The response to evaluate.

  • context (list) – A non-empty list of ContextNode objects.

query: str
response: str
context: list[ContextNode]
class trustwise.sdk.types.AnswerRelevancyRequest(**data)[source]

Bases: SDKBaseModel

Request type for answer relevancy evaluation.

Parameters:

  • query (str) – The input query string.

  • response (str) – The response to evaluate.

query: str
response: str
class trustwise.sdk.types.SensitivityRequest(**data)[source]

Bases: SDKBaseModel

Request type for sensitivity evaluation.

Parameters:

  • response (str) – The response to evaluate.

  • topics (list) – List of topics to check for sensitivity.

response: str
topics: list[str]
class trustwise.sdk.types.CostRequest(**data)[source]

Bases: SDKBaseModel

Request type for cost evaluation.

Parameters:

  • model_name (str) – Model name must be a non-empty string.

  • model_type (Literal) – Model type must be ‘LLM’ or ‘Reranker’.

  • model_provider (str) – Model provider must be a non-empty string. Should be one of [‘openai’, ‘togetherai’, ‘huggingface’, ‘nvidia’, ‘azure’]

  • number_of_queries (int) – Number of queries must be > 0.

  • total_prompt_tokens (int) – (Optional)Total number of prompt tokens (must be > 0).

  • total_completion_tokens (int) – (Optional)Total number of completion tokens (must be > 0).

  • total_tokens (Optional[Annotated]) – (Optional) Total tokens (optional, must be > 0 if provided).

  • instance_type (str | None) – (Optional) Instance type required for Hugging Face models.

  • average_latency (Optional[Annotated]) – (Optional) Average latency in milliseconds, must be > 0 if provided.

model_name: Annotated[str]
model_type: Literal['LLM', 'Reranker']
model_provider: Annotated[str]
number_of_queries: Annotated[int]
total_prompt_tokens: Annotated[int]
total_completion_tokens: Annotated[int]
total_tokens: Optional[Annotated[int]]
instance_type: str | None
average_latency: Optional[Annotated[float]]
class trustwise.sdk.types.CarbonRequest(**data)[source]

Bases: SDKBaseModel

Request type for carbon evaluation.

Parameters:

  • processor_name (str) – Name of the processor.

  • provider_name (str) – Name of the provider.

  • provider_region (str) – Region of the provider.

  • instance_type (str) – Instance type.

  • average_latency (int) – Average latency in milliseconds.

processor_name: str
provider_name: str
provider_region: str
instance_type: str
average_latency: int
class trustwise.sdk.types.GuardrailResponse(**data)[source]

Bases: SDKBaseModel

Response type for guardrail evaluation.

Parameters:

  • passed (bool) – Whether all metrics passed.

  • blocked (bool) – Whether the response is blocked due to failure.

  • results (dict) – Dictionary of metric results, each containing ‘passed’ and ‘result’.

passed: bool
blocked: bool
results: dict
to_json(**kwargs)[source]

Return a JSON string representation of the guardrail evaluation, recursively serializing all nested SDK types. Use this for logging, API responses, or storage.

Return type:

str

Parameters:

kwargs (dict[str, object])

to_dict()[source]

Return a Python dict representation of the guardrail evaluation, recursively serializing all nested SDK types. Use this for programmatic access, further processing, or conversion to JSON via json.dumps().

Return type:

dict

class trustwise.sdk.types.StabilityRequest(**data)[source]

Bases: SDKBaseModel

Request type for stability evaluation.

Stability measures the consistency of responses to the same prompt.

Parameters:

responses (list) – A list of responses to evaluate for stability. Must contain at least two responses.

responses: list[str]
class trustwise.sdk.types.StabilityResponse(**data)[source]

Bases: SDKBaseModel

Response type for stability evaluation.

Stability measures how consistent the responses are. A higher score indicates greater consistency among the responses.

Parameters:

  • min (int) – An integer, 0-100, measuring the minimum stability between any two pairs of responses (100 is high similarity)

  • avg (int) – An integer, 0-100, measuring the average stability between all two pairs of responses (100 is high similarity)

min: int
avg: int
class trustwise.sdk.types.RefusalRequest(**data)[source]

Bases: SDKBaseModel

Request type for refusal evaluation.

Refusal measures the likelihood that a response is a refusal to answer or comply with the query.

Parameters:

  • query (str) – The input prompt/query sent to the LLM or agent.

  • response (str) – The response from LLM or agent to evaluate.

query: str
response: str
class trustwise.sdk.types.RefusalResponse(**data)[source]

Bases: SDKBaseModel

Response type for refusal evaluation.

Refusal score indicates the likelihood that the response is a refusal (0-100).

Parameters:

score (int) – Refusal score (0-100). A higher score indicates a higher degree (firmness) of refusal.

score: int
class trustwise.sdk.types.CompletionRequest(**data)[source]

Bases: SDKBaseModel

Request type for completion evaluation.

Completion measures how well the response completes or follows the query’s instruction.

Parameters:

  • query (str) – The input prompt/query sent to the LLM or agent.

  • response (str) – The response from LLM or agent to evaluate.

query: str
response: str
class trustwise.sdk.types.CompletionResponse(**data)[source]

Bases: SDKBaseModel

Response type for completion evaluation.

Completion score indicates how well the response completes the query (0-100).

Parameters:

score (int) – Completion score (0-100). A higher score indicates a better completion of the query.

score: int
class trustwise.sdk.types.AdherenceRequest(**data)[source]

Bases: SDKBaseModel

Request type for adherence evaluation.

Adherence measures how well the response follows a given policy or instruction.

Parameters:

  • policy (str) – The policy or instruction the response should adhere to.

  • response (str) – The response from LLM or agent to evaluate.

policy: str
response: str
class trustwise.sdk.types.AdherenceResponse(**data)[source]

Bases: SDKBaseModel

Response type for adherence evaluation.

Adherence score indicates how well the response follows the policy (0-100).

Parameters:

score (int) – Adherence score (0-100). A higher score indicates better adherence to the policy.

score: int

Guardrails

class trustwise.sdk.guardrails.Guardrail(trustwise_client, thresholds, *, block_on_failure=False, callbacks=None)[source]

Bases: object

Guardrail system for Trustwise API responses.

Warning

This feature is currently in beta. The API and functionality may change in future releases.

Parameters:
__init__(trustwise_client, thresholds, *, block_on_failure=False, callbacks=None)[source]

Initialize the guardrail system.

Parameters:

  • trustwise_client (TrustwiseSDK) – The Trustwise client instance.

  • thresholds (dict[str, float]) – Dictionary of metric names and their threshold values (0-100).

  • block_on_failure (bool) – Whether to block responses that fail any metric.

  • callbacks (dict[str, Callable] | None) – Optional callbacks for metric evaluation results.

Raises:

ValueError – If thresholds are invalid.

Return type:

None

_get_metric_kwargs(metric, query, response, context)[source]

Get keyword arguments for a given metric.

Return type:

dict[str, Any]

Parameters:
check_pii(text, allowlist=None, blocklist=None)[source]

Check text for PII and determine if it should be blocked.

Parameters:

  • text (str) – Text to check for PII

  • allowlist (list[str] | None) – Optional list of allowed terms

  • blocklist (list[str] | None) – Optional list of blocked terms

Return type:

dict[str, Any]

Returns:

Dictionary with PII detection results and pass/fail status

evaluate(query, response, context=None)[source]

Evaluate a response against configured metrics.

Return type:

GuardrailResponse

Parameters:

The SDK provides a guardrail system to enforce safety and alignment metrics thresholds. The guardrail system can be used to:

  • Set thresholds for multiple metrics

  • Block responses that fail metric checks

  • Execute callbacks when metrics are evaluated

  • Check for PII content with custom allowlists and blocklists

Example usage:

trustwise = TrustwiseSDK(config)
guardrail = trustwise.guardrails(
    thresholds={
        "faithfulness": 0.8,
        "answer_relevancy": 0.7,
        "clarity": 0.7
    },
    block_on_failure=True
)
evaluation = guardrail.evaluate(
    query="What is the capital of France?",
    response="The capital of France is Paris.",
    context=context
)
# Pythonic access
print(evaluation.passed)
print(evaluation.results['faithfulness']['result'].score)
# Serialization
print(evaluation.to_dict())
print(evaluation.to_json(indent=2))

The guardrail system returns a trustwise.sdk.types.GuardrailResponse