V3 Metrics

Warning

V3 Metrics are deprecated and removed in this version. Only the cost metric remains available in v3. Please migrate to V4 Metrics for the latest features and continued support.

See the Migration Guide: v3 to v4 for step-by-step migration instructions.

Note

The cost metric is available in both synchronous and asynchronous forms. For async, use TrustwiseSDKAsync and await the evaluate() methods.

The v3 SDK now only provides access to the cost metric through the metrics.v3 namespace. Example usage:

For more details on the metrics, please refer to the Trustwise Metrics Documentation.

# v3 cost metric (deprecated - use v4 instead)
cost = trustwise.metrics.v3.cost.evaluate(model_name="...", model_type="LLM", ...)

Async example:

import asyncio
from trustwise.sdk import TrustwiseSDKAsync
from trustwise.sdk.config import TrustwiseConfig

async def main():
    config = TrustwiseConfig()
    trustwise = TrustwiseSDKAsync(config)
    # v3 cost metric (deprecated - use v4 instead)
    await trustwise.metrics.v3.cost.evaluate(
        model_name="gpt-3.5-turbo",
        model_type="LLM",
        model_provider="openai",
        number_of_queries=5,
        total_prompt_tokens=950,
        total_completion_tokens=50
    )

asyncio.run(main())

Refer to the API Reference for details on the cost metric’s parameters.

Note

Custom types such as CostRequestV3 and CostResponseV3 are defined in trustwise.sdk.types.

Cost

metrics.v3.cost.evaluate(model_name: str, model_type: str, model_provider: str, number_of_queries: int, total_prompt_tokens: int | None = None, total_completion_tokens: int | None = None, total_tokens: int | None = None, instance_type: str | None = None, average_latency: float | None = None) → CostResponseV3

Evaluates the cost of API usage based on token counts, model information, and infrastructure details.

Request:

Returns:

Example response:

{
    "cost_estimate_per_run": 0.0025,
    "total_project_cost_estimate": 0.0125
}

  • cost_estimate_per_run: The estimated cost for a single API call

  • total_project_cost_estimate: The estimated total cost for the specified number of queries