📄️ Custom Metrics
Every ML and data pipeline use case is different! It is not unusual for AI Observatory users to supply custom metrics into the platform. Overall, there are three use cases for custom metrics:
📄️ Data Quality Check
Motivation
📄️ Distributional Drift
Motivation
📄️ Embeddings Data
With WhyLabs, you are able to profile embeddings data by comparing them to reference data points. These references can be completely determined by users (helpful when they represent prototypical "ideal" representations of a cluster or scenario) but can also be chosen programmatically.
📄️ Language Models
Motivation
📄️ Image Data
In addition to tabular and textual data, whylogs can generate profiles of image data. whylogs automatically computes a number of metrics relative to image data. These include the following.
