Cost Ruleset

Cost is designed to monitor LLM costs. This ruleset doesn't have an associated score with it at the moment, rather, it enables character and token count metrics.

The following yaml code can be added to your policy to enable the Cost ruleset.

  - ruleset: score.cost
    options:
      behavior: observe
      sensitivity: medium

This ruleset adds the equivalent of the following metric section to your yaml policy and uses those metrics to compute an overall score.

metrics:
    - metric: prompt.stats.char_count
    - metric: prompt.stats.token_count
    - metric: response.stats.char_count
    - metric: response.stats.token_count