API Reference

This section provides detailed technical documentation for the Trustwise SDK. For conceptual information and best practices, see the individual metric sections.

SDK

class trustwise.sdk.TrustwiseSDK(config)[source]

Bases: object

Main SDK class for Trustwise API access.

Provides access to all metrics through version-specific paths.

Parameters:: config (TrustwiseConfig)

__init__(config)[source]

Initialize the Trustwise SDK with path-based versioning support.

Parameters:: config (TrustwiseConfig) – Trustwise configuration instance.
Return type:: None

get_beta_features()[source]

Get the set of features currently in beta.

Returns:: Set of feature names that are in beta
Return type:: Set[str]

get_versions()[source]

Get the available API versions for the metrics.

Return type:: dict[str, list[str]]
Returns:: Dictionary mapping ‘metrics’ to its available versions. Example: {“metrics”: [“v3”]}

guardrails(thresholds, *, block_on_failure=False, callbacks=None)[source]

Create a guardrail system for response evaluation.

Parameters:

thresholds (dict[str, float]) – Dictionary mapping metrics to threshold values.
block_on_failure (bool) – Whether to block responses that fail checks.
callbacks (dict[str, Callable] | None) – Dictionary mapping metrics to callback functions.

Return type:

Guardrail

Returns:

A configured Guardrail instance.

Raises:

ValueError – If any metric in thresholds is not supported by the current API version.

is_beta_feature(feature_name)[source]

Check if a feature is currently in beta.

Parameters:: feature_name (str) – Name of the feature to check
Returns:: True if the feature is in beta, False otherwise
Return type:: bool

class trustwise.sdk.TrustwiseSDKAsync(config)[source]

Bases: object

Async SDK entrypoint for Trustwise. Use this class to access async metrics.

Parameters:: config (TrustwiseConfig)

__init__(config)[source]

Initialize the Trustwise SDK with path-based versioning support.

Parameters:: config (TrustwiseConfig) – Trustwise configuration instance.
Return type:: None

Configuration

Simple configuration for Trustwise API with environment variable support.

class trustwise.sdk.config.TrustwiseConfig(**data)[source]

Bases: BaseModel

Configuration for Trustwise API.

This class handles configuration for the Trustwise API, including authentication and endpoint URLs. It supports both direct initialization and environment variables.

Parameters:

api_key (str, Optional) – API key for authentication. If not provided, will be read from TW_API_KEY environment variable. Required for API access.
base_url (str, Optional) – Base URL for API endpoints. If not provided, will be read from TW_BASE_URL environment variable or default to https://api.trustwise.ai.

Environment Variables:

TW_API_KEY: API key for authentication
TW_BASE_URL: Base URL for API endpoints (defaults to https://api.trustwise.ai)

Example

>>> from trustwise.sdk.config import TrustwiseConfig
>>> # Using environment variables
>>> config = TrustwiseConfig()
>>> # Using direct initialization
>>> config = TrustwiseConfig(api_key="your-api-key", base_url="https://api.trustwise.ai")

Raises:

ValueError – If API key is missing or invalid, or if base URL is invalid

Parameters:

api_key (str)
base_url (str)

api_key: str

__init__(**data)[source]

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Return type:: None

base_url: str

_validate_url()[source]

Validate base URL format and scheme.

Return type:: None

get_safety_url(version)[source]

Get safety API URL for specified version.

Return type:: str
Parameters:: version (str)

get_alignment_url(version)[source]

Get alignment API URL for specified version.

Return type:: str
Parameters:: version (str)

get_performance_url(version)[source]

Get performance API URL for specified version.

Return type:: str
Parameters:: version (str)

get_metrics_url(version)[source]

Get metrics API URL for specified version.

Return type:: str
Parameters:: version (str)

Types

Note

V4 types are now the default for all metrics. These provide enhanced type safety and improved response structures.

Types for v4 metrics.

class trustwise.sdk.metrics.v4.types.Score(**data)[source]

Bases: SDKBaseModel

A score value between 0 and 100.

Parameters:: value (float) – Score value between 0 and 100.

value: float

__float__()[source]

Convert to float for arithmetic operations.

Return type:: float

__str__()[source]

String representation.

Return type:: str

__repr__()[source]

Representation.

Return type:: str

trustwise.sdk.metrics.v4.types.convert_to_score(v)[source]

Convert various input types to Score.

Return type:: Score
Parameters:: v (float | int | dict | Score)

class trustwise.sdk.metrics.v4.types.ObjectStyleScore(**data)[source]

Bases: SDKBaseModel

Object style score with label.

Parameters:

label (str) – The label for the score.
score (float) – The value of the score. (0-100)

label: str

score: float

class trustwise.sdk.metrics.v4.types.ContextChunk(**data)[source]

Bases: SDKBaseModel

A single context chunk for v4 context relevancy evaluation.

Parameters:

chunk_text (str) – The text content of the context chunk.
chunk_id (str) – The unique identifier for the context chunk.

chunk_text: str

chunk_id: str

class trustwise.sdk.metrics.v4.types.ContextRelevancyRequest(**data)[source]

Bases: SDKRequestModel

Request type for v4 context relevancy evaluation.

Parameters:

metadata (dict) – Optional metadata to be returned in the response.
query (str) – The input query string.
context (list) – A non-empty list of context chunks.
severity (float | None) – (Optional) severity level (0-1).
include_chunk_scores (bool | None) – (Optional) Whether to include individual chunk scores in the response.

query: str

context: list[ContextChunk]

severity: float | None

include_chunk_scores: bool | None

metadata: dict

class trustwise.sdk.metrics.v4.types.PromptManipulationRequest(**data)[source]

Bases: SDKRequestModel

Request type for v4 prompt manipulation evaluation.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
text (str) – The text to evaluate for prompt manipulation.
severity (int | None) – (Optional) Severity level (0-1).

text: str

severity: int | None

class trustwise.sdk.metrics.v4.types.PromptManipulationResponse(**data)[source]

Bases: SDKResponseModel

Response type for prompt manipulation detection.

Higher scores indicate a greater likelihood that the text contains prompt manipulation attempts that could manipulate the AI system.

Parameters:

score (float) – Overall prompt manipulation score (0-100). A higher score indicates a higher likelihood of the text being used for prompt manipulation.
scores (list) – Detailed breakdown of manipulation scores by type.

score: float

scores: list[ObjectStyleScore]

class trustwise.sdk.metrics.v4.types.AnswerRelevancyRequest(**data)[source]

Bases: SDKRequestModel

Request type for answer relevancy evaluation.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
query (str) – The input query string.
response (str) – The response to evaluate.

query: str

response: str

class trustwise.sdk.metrics.v4.types.AnswerRelevancyResponse(**data)[source]

Bases: SDKResponseModel

Response type for answer relevancy evaluation.

Answer relevancy measures how well the response addresses the specific query. A higher score indicates the response is more directly relevant to the question asked.

Parameters:

score (float) – Answer relevancy score (0-100). A higher score, indicating a better response, is correlated with a higher confidence in the response being relevant to the query.
generated_question (str) – The generated question for which the response would be relevant.

score: float

generated_question: str

class trustwise.sdk.metrics.v4.types.FaithfulnessRequest(**data)[source]

Bases: SDKRequestModel

Request type for faithfulness evaluation.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
query (str) – The input query string.
response (str) – The response to evaluate.
context (list) – A non-empty list of context chunks.
severity (float | None) – (Optional) severity level (0-1).
include_citations (bool | None) – Whether to include citations in the response. If true, a unique chunk_id must additionally be provided for each context chunk.

query: str

response: str

context: list[ContextChunk]

severity: float | None

include_citations: bool | None

class trustwise.sdk.metrics.v4.types.Statement(**data)[source]

Bases: SDKBaseModel

A verified fact extracted from a response.

Parameters:

statement (str) – The extracted statement from the response text that represents a ‘atomically’ fact
label (str) – The label indicating the fact’s hallucination status. One of (‘Safe’, ‘Unsafe’, ‘Intrinsic Hallucination’, ‘Extrinsic Hallucination’)
probability (float) – The associated probability of the label (0-1)
sentence_span (list) – A list of two integers [start, end] indicating the character positions of this statement in the response text

statement: str

label: str

probability: float

sentence_span: list[int]

class trustwise.sdk.metrics.v4.types.FaithfulnessResponse(**data)[source]

Bases: SDKBaseModel

Response type for faithfulness evaluation.

Faithfulness measures how well the response adheres to the provided context. A higher score indicates better alignment with the source material.

Parameters:

score (float) – Faithfulness score (0-100). A higher score, indicating a better response, is correlated with a higher confidence in each statement being true, and also a higher proportion of statements being true with respect to the context.
statements (list) – List of extracted ‘atomic’ statements from the response, each containing the statement, label, associated probability, and location in the text.

score: float

statements: list[Statement]

class trustwise.sdk.metrics.v4.types.ContextRelevancyResponse(**data)[source]

Bases: SDKResponseModel

Response type for context relevancy evaluation.

Context relevancy measures how well the provided context is relevant to the query. A higher score indicates the context is more directly relevant to the question asked.

Parameters:

score (float) – Overall context relevancy score (0-100). A higher score indicates better context relevancy to the query.
scores (list) – Detailed breakdown of context relevancy scores by aspect.

score: float

scores: list[ObjectStyleScore]

class trustwise.sdk.metrics.v4.types.ClarityRequest(**data)[source]

Bases: SDKRequestModel

Request type for clarity evaluation.

Clarity measures how easy text is to read. It gives higher scores to writing that contains words which are easier to read, and uses concise, self-contained sentences. It does not measure how well you understand the ideas in the text.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
text (str) – The text to evaluate for clarity.

text: str

class trustwise.sdk.metrics.v4.types.ClarityResponse(**data)[source]

Bases: SDKResponseModel

Response type for clarity evaluation.

Clarity scores indicate how clear and understandable the response is. Higher scores suggest better readability and comprehension.

Parameters:: score (float) – A score from 0-100 indicating how clear and understandable the response is. Higher scores indicate better clarity of the response.

score: float

class trustwise.sdk.metrics.v4.types.FormalityRequest(**data)[source]

Bases: SDKRequestModel

Request type for formality evaluation.

Formality measures the level of formality in written text, distinguishing between casual, conversational, and formal writing styles.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
text (str) – The text to evaluate for formality.

text: str

class trustwise.sdk.metrics.v4.types.FormalityResponse(**data)[source]

Bases: SDKResponseModel

Response type for formality evaluation.

Formality scores indicate the level of formality in the text. Higher scores indicate more formal writing, lower scores indicate more casual writing.

Parameters:: score (float) – Formality score (0-100). The Formality metric judges how formal a piece of text is. A higher score indicates the text is more formal, and a lower score indicates the text is more informal.

score: float

class trustwise.sdk.metrics.v4.types.SimplicityRequest(**data)[source]

Bases: SDKRequestModel

Request type for simplicity evaluation.

Simplicity measures how easy it is to understand the words in a text. It gives higher scores to writing that uses common, everyday words instead of special terms or complicated words. Simplicity looks at the words you choose, not how you put them together in sentences.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
text (str) – The text to evaluate for simplicity.

text: str

class trustwise.sdk.metrics.v4.types.SimplicityResponse(**data)[source]

Bases: SDKResponseModel

Response type for simplicity evaluation.

Simplicity measures how easy it is to understand the words in a text. It gives higher scores to writing that uses common, everyday words instead of special terms or complicated words. Simplicity looks at the words you choose, not how you put them together in sentences.

Parameters:: score (float) – Simplicity score (0-100). A higher score indicates a simpler response, with fewer words and simpler language which is easier to understand.

score: float

class trustwise.sdk.metrics.v4.types.ToneRequest(**data)[source]

Bases: SDKRequestModel

Request type for tone evaluation.

Tone evaluation identifies the emotional tone and sentiment expressed in the text across multiple emotional categories.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
text (str) – The text to evaluate for tone.
tones (list[str] | None) – The tones to evaluate for tone metric.

text: str

tones: list[str] | None

class trustwise.sdk.metrics.v4.types.ToneResponse(**data)[source]

Bases: SDKResponseModel

Response type for tone evaluation.

Tone analysis identifies the emotional tones present in the text, with confidence scores for each detected tone category.

Parameters:: scores (list) – List of detected tones with confidence scores (0-100). Higher scores indicate stronger presence of that tone.

scores: list[ObjectStyleScore]

class trustwise.sdk.metrics.v4.types.ToxicityRequest(**data)[source]

Bases: SDKRequestModel

Request type for toxicity evaluation.

Toxicity evaluation identifies harmful or inappropriate content across multiple categories like threats, insults, and hate speech.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
text (str) – The text to evaluate for toxicity.
severity (int | None) – (Optional) Severity level (0-1).

text: str

severity: int | None

class trustwise.sdk.metrics.v4.types.ToxicityResponse(**data)[source]

Bases: SDKResponseModel

Response type for toxicity evaluation.

Toxicity scores indicate the presence of harmful content across different categories. Higher scores indicate more toxic content, so lower scores are preferred.

Parameters:

score (float) – Toxicity score (0-100). A higher score indicates the text is more toxic, and thus a lower score is preferred.
scores (list) – List of toxicity scores by category (0-100). A higher score indicates the text is more toxic, and thus a lower score is preferred.

score: float

scores: list[ObjectStyleScore]

class trustwise.sdk.metrics.v4.types.PIIEntity(**data)[source]

Bases: SDKBaseModel

A detected piece of personally identifiable information.

PII entities represent sensitive information that has been identified in text, including its location and category.

Parameters:

interval (list) – The [start, end] indices of the PII in the text.
string (str) – The detected PII string.
category (str) – The PII category.

interval: list[int]

string: str

category: str

class trustwise.sdk.metrics.v4.types.PIIRequest(**data)[source]

Bases: SDKRequestModel

Request type for PII detection.

PII detection identifies personally identifiable information in text based on customizable allowlists and blocklists.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
text (str) – The text to evaluate for PII detection.
allowlist (list[str] | None) – (Optional) List of allowed PII strings or regex patterns.
blocklist (list[str] | None) – (Optional) List of blocked PII strings or regex patterns.
categories (list[str] | None) – (Optional) List of PII categories to evaluate.

text: str

allowlist: list[str] | None

blocklist: list[str] | None

categories: list[str] | None

class trustwise.sdk.metrics.v4.types.PIIResponse(**data)[source]

Bases: SDKResponseModel

Response type for PII detection.

Contains all detected PII entities with their locations and categories for further processing or filtering.

Parameters:: pii (list) – List of detected PII occurrences.

pii: list[PIIEntity]

class trustwise.sdk.metrics.v4.types.HelpfulnessRequest(**data)[source]

Bases: SDKRequestModel

Request type for helpfulness evaluation.

Helpfulness measures how useful a given text is. It gives higher scores to texts that fully explain a topic. Helpful responses provide clear, complete information.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
text (str) – The text to evaluate for helpfulness.

text: str

class trustwise.sdk.metrics.v4.types.HelpfulnessResponse(**data)[source]

Bases: SDKResponseModel

Response type for helpfulness evaluation.

Helpfulness measureshow useful a given text is. It gives higher scores to texts that fully explain a topic. Helpful responses provide clear, complete information.

Parameters:: score (float) – A score from 0-100 indicating how helpful the response is. Higher scores indicate better helpfulness of the response.

score: float

class trustwise.sdk.metrics.v4.types.SensitivityRequest(**data)[source]

Bases: SDKRequestModel

Request type for sensitivity evaluation.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
text (str) – The text to evaluate for sensitivity.
topics (list) – List of topics to check for sensitivity.

text: str

topics: list[str]

class trustwise.sdk.metrics.v4.types.SensitivityResponse(**data)[source]

Bases: SDKResponseModel

Response type for sensitivity evaluation.

Sensitivity evaluation checks for the presence of specific topics or themes in the text, providing scores for each requested topic.

Parameters:: scores (list) – List of sensitivity scores for each requested topic (0-100). A high score for a topic indicates that the topic is present in the text.

scores: list[ObjectStyleScore]

class trustwise.sdk.metrics.v4.types.RefusalRequest(**data)[source]

Bases: SDKRequestModel

Request type for refusal evaluation.

Refusal measures the likelihood that a response is a refusal to answer or comply with the query. This metric helps identify when AI systems decline to provide information or perform tasks.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
query (str) – The input prompt/query sent to the LLM or agent.
response (str) – The response from LLM or agent to evaluate.

query: str

response: str

class trustwise.sdk.metrics.v4.types.RefusalResponse(**data)[source]

Bases: SDKResponseModel

Response type for refusal evaluation.

Refusal scores indicate the likelihood that the response is a refusal to answer or comply. Higher scores indicate stronger refusal behavior.

Parameters:: score (float) – Refusal score (0-100). A higher score indicates a higher degree (firmness) of refusal.

score: float

class trustwise.sdk.metrics.v4.types.CompletionRequest(**data)[source]

Bases: SDKRequestModel

Request type for completion evaluation.

Completion measures how well the response completes or follows the query’s instruction.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
query (str) – The input prompt/query sent to the LLM or agent.
response (str) – The response from LLM or agent to evaluate.

query: str

response: str

class trustwise.sdk.metrics.v4.types.CompletionResponse(**data)[source]

Bases: SDKResponseModel

Response type for completion evaluation.

Completion score indicates how well the response completes the query (0-100).

Parameters:: score (float) – Completion score (0-100). A higher score indicates a better completion of the query.

score: float

class trustwise.sdk.metrics.v4.types.AdherenceRequest(**data)[source]

Bases: SDKRequestModel

Request type for adherence evaluation.

Adherence evaluation measures how well the response follows a given policy or instruction.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
policy (str) – The policy or instruction the response should adhere to.
response (str) – The response from LLM or agent to evaluate.

policy: str

response: str

class trustwise.sdk.metrics.v4.types.AdherenceResponse(**data)[source]

Bases: SDKResponseModel

Response type for adherence evaluation.

Adherence score indicates how well the response follows the policy (0-100).

Parameters:: score (float) – Adherence score (0-100). A higher score indicates better adherence to the policy.

score: float

class trustwise.sdk.metrics.v4.types.StabilityRequest(**data)[source]

Bases: SDKRequestModel

Request type for stability evaluation.

Stability evaluation measures the consistency of responses to the same prompt.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
responses (list) – A list of responses to evaluate for stability. Must contain at least two responses.

responses: list[str]

class trustwise.sdk.metrics.v4.types.StabilityResponse(**data)[source]

Bases: SDKResponseModel

Response type for stability evaluation.

Stability measures how consistent the responses are. A higher score indicates greater consistency among the responses.

Parameters:

min (int) – An integer, 0-100, measuring the minimum stability between any two pairs of responses (100 is high similarity)
avg (int) – An integer, 0-100, measuring the average stability between all two pairs of responses (100 is high similarity)

min: int

avg: int

class trustwise.sdk.metrics.v4.types.CarbonValue(**data)[source]

Bases: SDKBaseModel

A carbon value with unit.

Parameters:

value (float) – The carbon value.
unit (str) – The unit of measurement (e.g., ‘kg_co2e’).

value: float

unit: str

class trustwise.sdk.metrics.v4.types.CarbonComponent(**data)[source]

Bases: SDKBaseModel

A carbon component breakdown.

Parameters:

component (str) – The component name (e.g., ‘operational_gpu’, ‘operational_cpu’, ‘embodied_cpu’).
carbon (CarbonValue) – The carbon value for this component.

component: str

carbon: CarbonValue

class trustwise.sdk.metrics.v4.types.CarbonRequest(**data)[source]

Bases: SDKRequestModel

Request type for carbon evaluation.

Carbon evaluation measures the carbon footprint of AI operations based on provider, instance type, region, and latency.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
provider (str) – The cloud provider (e.g., ‘azure’, ‘aws’, ‘gcp’).
region (str) – The region where the instance is located.
instance_type (str) – The instance type.
latency (float | int) – The latency in milliseconds.

provider: str

region: str

instance_type: str

latency: float | int

class trustwise.sdk.metrics.v4.types.CarbonResponse(**data)[source]

Bases: SDKResponseModel

Response type for carbon evaluation.

Carbon evaluation provides the total carbon footprint and breakdown by component (operational GPU, operational CPU, embodied CPU).

Parameters:

carbon (CarbonValue) – The total carbon footprint.
components (list) – Breakdown of carbon footprint by component.

carbon: CarbonValue

components: list[CarbonComponent]

V3 Types (Legacy)

Note

V3 types are deprecated and only include cost-related types and common SDK types. Use V4 types for all new development.

class trustwise.sdk.types.SDKBaseModel(**data)[source]

Bases: BaseModel

Base model for all SDK types with common functionality.

to_json(**kwargs)[source]

Return a JSON string representation of the model. Ensures valid JSON output regardless of Pydantic version. Always excludes None fields by default.

Return type:: str
Parameters:: kwargs (dict[str, object])

to_dict()[source]

Convert to dictionary.

Return type:: dict

classmethod validate_score_range(v, min_value, max_value, label)[source]

Validate that a score or list/dict of scores falls within the specified range.

Return type:

object

Parameters:

v (float | int | list | dict)
min_value (float)
max_value (float)
label (str)

static format_validation_error(model_cls, validation_error)[source]

Format a Pydantic ValidationError into a user-friendly error message using field types and descriptions. Distinguishes between missing and invalid arguments, and sets the error prefix accordingly.

Return type:

str

Parameters:

model_cls (type)
validation_error (Exception)

_get_field_description(field_name, field_info=None)[source]

Get the description for a field from its field info.

Return type:

str

Parameters:

field_name (str)
field_info (object | None)

_format_output(indent=0)[source]

Format the model’s output with proper indentation and descriptions. Used by both __repr__ and __str__ to ensure consistent output.

Return type:: str
Parameters:: indent (int)

_format_nested_dict(data, model_class=None, indent=0)[source]

Format a nested dictionary with field descriptions. If model_class is provided, it will be used to look up field descriptions.

Return type:

list[str]

Parameters:

data (dict)
model_class (type | None)
indent (int)

__repr__()[source]

Return a detailed string representation.

Return type:: str

_format_json_value(value)[source]

Format a value for JSON display with syntax highlighting.

Return type:: str
Parameters:: value (object)

_repr_mimebundle_(include=None, exclude=None)[source]

Return multiple representations of the object. This is the preferred way to provide multiple representations in Jupyter.

Return type:

dict[str, str]

Parameters:

include (object | None)
exclude (object | None)

_format_html(indent_level=0)[source]

Format the model’s output as HTML with proper styling. Used by __repr_html__ to create a pretty representation in Jupyter notebooks.

Return type:: str
Parameters:: indent_level (int)

_get_value_css_class(value)[source]

Return the appropriate CSS class for a value based on its type.

Return type:: str
Parameters:: value (object)

_format_dict_html(data, indent_level=0)[source]

Format a dictionary as HTML with proper styling and field descriptions.

Return type:

str

Parameters:

data (dict)
indent_level (int)

__repr_html__()[source]

Return an HTML representation for Jupyter notebooks.

Return type:: str

class trustwise.sdk.types.SDKRequestModel(**data)[source]

Bases: SDKBaseModel

Base model for all SDK request types.

Parameters:: metadata (dict[str, object] | None)

metadata: dict[str, object] | None

class trustwise.sdk.types.SDKResponseModel(**data)[source]

Bases: SDKBaseModel

Base model for all SDK response types.

class trustwise.sdk.types.CostResponseV3(**data)[source]

Bases: SDKResponseModel

Response type for cost evaluation.

Parameters:

cost_estimate_per_run (float) – Represents the expense for each individual LLM call. This is the cost of the model call, including the prompt and response tokens.
total_project_cost_estimate (float) – Cumulative cost for 10,000 LLM calls to help estimate costs over high-volume usage patterns.

cost_estimate_per_run: float

total_project_cost_estimate: float

class trustwise.sdk.types.CostRequestV3(**data)[source]

Bases: SDKRequestModel

Request type for cost evaluation.

Parameters:

metadata (dict[str, object] | None) – Optional metadata for the request that will be included in the response.
model_name (str) – Model name must be a non-empty string.
model_type (Literal) – Model type must be ‘LLM’ or ‘Reranker’.
model_provider (str) – Model provider must be a non-empty string. Should be one of [‘openai’, ‘togetherai’, ‘huggingface’, ‘nvidia’, ‘azure’]
number_of_queries (int) – Number of queries must be > 0.
total_prompt_tokens (int) – (Optional)Total number of prompt tokens (must be > 0).
total_completion_tokens (int) – (Optional)Total number of completion tokens (must be > 0).
total_tokens (Optional[Annotated]) – (Optional) Total tokens (optional, must be > 0 if provided).
instance_type (str | None) – (Optional) Instance type required for Hugging Face models.
average_latency (Optional[Annotated]) – (Optional) Average latency in milliseconds, must be > 0 if provided.

model_name: Annotated[str]

model_type: Literal['LLM', 'Reranker']

model_provider: Annotated[str]

number_of_queries: Annotated[int]

total_prompt_tokens: Annotated[int]

total_completion_tokens: Annotated[int]

total_tokens: Optional[Annotated[int]]

instance_type: str | None

average_latency: Optional[Annotated[float]]

class trustwise.sdk.types.GuardrailResponse(**data)[source]

Bases: SDKResponseModel

Response type for guardrail evaluation.

Parameters:

passed (bool) – Whether all metrics passed.
blocked (bool) – Whether the response is blocked due to failure.
results (dict) – Dictionary of metric results, each containing ‘passed’ and ‘result’.

passed: bool

blocked: bool

results: dict

to_json(**kwargs)[source]

Return a JSON string representation of the guardrail evaluation, recursively serializing all nested SDK types. Use this for logging, API responses, or storage.

Return type:: str
Parameters:: kwargs (dict[str, object])

to_dict()[source]

Return a Python dict representation of the guardrail evaluation, recursively serializing all nested SDK types. Use this for programmatic access, further processing, or conversion to JSON via json.dumps().

Return type:: dict

Guardrails

class trustwise.sdk.guardrails.Guardrail(trustwise_client, thresholds, *, block_on_failure=False, callbacks=None)[source]

Bases: object

Guardrail system for Trustwise API responses.

Warning

This feature is currently in beta. The API and functionality may change in future releases.

Parameters:

trustwise_client (TrustwiseSDK)
thresholds (dict[str, float])
block_on_failure (bool)
callbacks (dict[str, Callable] | None)

__init__(trustwise_client, thresholds, *, block_on_failure=False, callbacks=None)[source]

Initialize the guardrail system.

Parameters:

trustwise_client (TrustwiseSDK) – The Trustwise client instance.
thresholds (dict[str, float]) – Dictionary of metric names and their threshold values (0-100).
block_on_failure (bool) – Whether to block responses that fail any metric.
callbacks (dict[str, Callable] | None) – Optional callbacks for metric evaluation results.

Raises:

ValueError – If thresholds are invalid.

Return type:

None

_get_metric_kwargs(metric, query, response, context)[source]

Get keyword arguments for a given metric.

Return type:

dict[str, Any]

Parameters:

metric (str)
query (str)
response (str)
context (list[dict[str, Any]] | dict[str, Any] | None)

check_pii(text, allowlist=None, blocklist=None)[source]

Check text for PII and determine if it should be blocked.

Parameters:

text (str) – Text to check for PII
allowlist (list[str] | None) – Optional list of allowed terms
blocklist (list[str] | None) – Optional list of blocked terms

Return type:

dict[str, Any]

Returns:

Dictionary with PII detection results and pass/fail status

evaluate(query, response, context=None)[source]

Evaluate a response against configured metrics.

Return type:

GuardrailResponse

Parameters:

query (str)
response (str)
context (list[dict[str, Any]] | dict[str, Any] | None)

The SDK provides a guardrail system to enforce safety and alignment metrics thresholds. The guardrail system can be used to:

Set thresholds for multiple metrics
Block responses that fail metric checks
Execute callbacks when metrics are evaluated
Check for PII content with custom allowlists and blocklists

Example usage:

trustwise = TrustwiseSDK(config)
guardrail = trustwise.guardrails(
    thresholds={
        "faithfulness": 0.8,
        "answer_relevancy": 0.7,
    },
    block_on_failure=True
)
evaluation = guardrail.evaluate(
    query="What is the capital of France?",
    response="The capital of France is Paris.",
    context=[{"chunk_text": "Paris is the capital of France.", "chunk_id": "doc:idx:1"}]
)
# Pythonic access
print(evaluation.passed)
print(evaluation.results['faithfulness']['result'].score)
# Serialization
print(evaluation.to_dict())
print(evaluation.to_json(indent=2))

The guardrail system returns a trustwise.sdk.types.GuardrailResponse