Skip to main content

Context Relevancy

SDK Usage

Learn how to evaluate this metric programmatically in the Trustwise SDK Documentation.

The Context Relevancy metric judges whether the provided context is useful for answering the query posed. First, the key topics discussed in the query are identified using the Topic Identification Model1. Then, the context is checked to determine whether it discusses each topic, resulting in a Context Relevancy score.

context-relevancy-flow

FAQs​

How do I know if the retrieved context is good?​

For context to be useful, it needs to include information about everything asked in the query. As such, if any of the identified topics are absent from the context (a score of 50 is a good cut-off for this), that may indicate that the retrieved context is insufficient.

Footnotes​

  1. Built with Llama ↩