Context Relevancy

SDK Usage

Learn how to evaluate this metric programmatically in the Trustwise SDK Documentation.

The Trustwise Context Relevancy metric measures how useful the provided context is for answering a specific query. First, the key topics discussed in the query are identified using the Topic Identification Model¹. Then, the context is checked to determine whether it discusses each topic, resulting in a Context Relevancy score. It gives higher scores when the context contains information that directly helps answer the question being asked. It does not measure whether the context is interesting or detailed, only if it contains what's needed to address the query.

FAQs

How do I know if the retrieved context is good?

For context to be useful, it needs to include information about everything asked in the query. As such, if any of the identified topics are absent from the context (a score of 50 is a good cut-off for this), that may indicate that the retrieved context is insufficient.

Footnotes

1. Built with Llama ↩