Truthfulness Ruleset

Truthfulness is designed to detect falsehoods (by opting into hallucination detection) or off topic responses from the LLM. For example, you can detect when an LLM's response doesn't appear to be related to the input prompt.

The following yaml code can be added to your policy to enable the Truthfulness ruleset.

  - ruleset: score.truthfulness
    options:
      behavior: observe
      sensitivity: medium
      rag_enabled: true
      hallucinations_enabled: false

This ruleset adds the equivalent of the following metric section to your yaml policy and uses those metrics to compute an overall score.

metrics:
    - metric: response.similarity.prompt
    - metric: response.similarity.context