.. _v3_metrics:

V3 Metrics
==========

.. warning::
   **V3 Metrics are deprecated** and removed in this version. Only the cost metric remains available in v3. Please migrate to :doc:`v4_metrics` for the latest features and continued support.

   See the :doc:`migration_guide` for step-by-step migration instructions.

.. note::
   The cost metric is available in both synchronous and asynchronous forms. For async, use ``TrustwiseSDKAsync`` and ``await`` the ``evaluate()`` methods.

The v3 SDK now only provides access to the cost metric through the ``metrics.v3`` namespace. Example usage:

For more details on the metrics, please refer to the `Trustwise Metrics Documentation <https://trustwiseai.github.io/tw-docs/docs/category/understanding-trustwise-metrics>`_.

.. code-block:: python

    # v3 cost metric (deprecated - use v4 instead)
    cost = trustwise.metrics.v3.cost.evaluate(model_name="...", model_type="LLM", ...)

Async example:

.. code-block:: python

    import asyncio
    from trustwise.sdk import TrustwiseSDKAsync
    from trustwise.sdk.config import TrustwiseConfig

    async def main():
        config = TrustwiseConfig()
        trustwise = TrustwiseSDKAsync(config)
        # v3 cost metric (deprecated - use v4 instead)
        await trustwise.metrics.v3.cost.evaluate(
            model_name="gpt-3.5-turbo",
            model_type="LLM",
            model_provider="openai",
            number_of_queries=5,
            total_prompt_tokens=950,
            total_completion_tokens=50
        )

    asyncio.run(main())

Refer to the :doc:`api` for details on the cost metric's parameters.

.. note::
   Custom types such as ``CostRequestV3`` and ``CostResponseV3`` are defined in :mod:`trustwise.sdk.types`.

Cost
~~~~
.. function:: metrics.v3.cost.evaluate(model_name: str, model_type: str, model_provider: str, number_of_queries: int, total_prompt_tokens: Optional[int] = None, total_completion_tokens: Optional[int] = None, total_tokens: Optional[int] = None, instance_type: Optional[str] = None, average_latency: Optional[float] = None) -> CostResponseV3

   Evaluates the cost of API usage based on token counts, model information, and infrastructure details.

   Request:

   - :class:`~trustwise.sdk.types.CostRequestV3`

   Returns:

   - :class:`~trustwise.sdk.types.CostResponseV3`
   
   Example response:

     .. code-block:: json

        {
            "cost_estimate_per_run": 0.0025,
            "total_project_cost_estimate": 0.0125
        }

   - ``cost_estimate_per_run``: The estimated cost for a single API call
   - ``total_project_cost_estimate``: The estimated total cost for the specified number of queries