Trustwise Rest API documentation (v3.1.0)

The Clarity metric judges whether text is easy to read, with a higher score indicating that the provided text is easier to read. It is based on the Flesch Reading Ease.

The Formality metric judges how formal a piece of text is. A higher score indicates the text is more formal, and a lower score indicates the text is more informal.

The Helpfulness metric judges how useful, detailed and unambiguous a response is, with a higher score indicating the response is more helpful. Responses which refuse to answer the question receive lower Helpfulness scores.

The Sensitivity metric judges whether certain topics are present in the provided text. A high score for a topic indicates that the topic is present in the text.

The Simplicity metric judges how easy to understand the words in a response are, with higher scores indicating the words are easier to understand. If lots of industry 'jargon', which is uncommon in standard language, is present, the simplicity score will be low.

The Tone metric identifies the tone of the provided text, from a number of different options. The three tones which score highest are returned to the user.

The Toxicity metric judges whether the input text is toxic. This includes obscenities, threats and insults. A higher score indicates the text is more toxic, and thus a lower score is preferred.

The Answer Relevancy metric judges whether a response attempted to answer the right query. To do this, we take the response and try to generate a query that the response would be a good answer to. This generated query is then compared to the original query to obtain an Answer Relevancy score.

The Context Relevancy metric judges whether the provided context is useful for answering the query posed. First, the key topics discussed in the query are identified. Then, the context is checked to determine whether it discusses each topic, resulting in a Context Relevancy score.

The Faithfulness metric judges whether the response is supported by the context. The response is split into individual, ‘atomic’ claims that contain only a single piece of information. Each of these claims is then verified with respect to the context, and the scores of these verifications are aggregated into a single Faithfulness score.

The PII metric identifies Personally Identifiable Information in a given text, this includes phone numbers, names, bank details and many more categories. This is achieved with a mixture of model-based and deterministic methods. The user can also include a custom RegEx blocklist and allowlist, to allow specific strings/patterns to be blocked or allowed.

Prompt Injection identifies prompts which contain attempts to bypass the security measures of LLMs. A higher score means the text is more likely to be a prompt injection.

The Summarization metric judges whether the response is supported by the context. The response is split into sentences, which are then checked with respect to the context. The scores are then aggregated into a single Summarization score.

The Carbon Emissions metric estimates and tracks the environmental impact of each LLM API call, calculating carbon emissions from processor usage and providing SCI metrics for individual and high-volume queries.

Estimates Cost to provide transparency into expenses related to each LLM call, helping users effectively manage and optimize costs.

Ensure agents respond consistently to reworded prompts. Evaluates consistency of responses to paraphrased inputs.

Evaluate whether the agent completes the task as requested.

Evaluate how clearly and appropriately the agent refuses harmful prompts.

Check if the agent strictly follows defined policies.