Misuse Ruleset

Misuse is designed to detect the LLM being used in a way that wasn't intended through the detection of various topics and pii. For example, you can make sure that users aren't trying to get medical or financial advice from a chat interface dedicated to customer support.

The following yaml code can be added to your policy to enable the Misuse ruleset.

  - ruleset: score.misuse
    options:
      behavior: observe
      sensitivity: medium
      topics:
        - medicine
        - legal
        - finance

This ruleset adds the equivalent of the following metric section to your yaml policy and uses those metrics to compute an overall score.

metrics:
  - metric: response.pii
  - metric: prompt.topics
    options:
      topics:
        - medicine
        - legal
        - finance