WhyLabs tracks all data health and model health statistics over time. The platform provides the ability to track all model and data health changes over time and to be alerted when changes occur. Data retention configurations are flexible, typically users retain at least 3 months worth of profiles to empower root cause analysis and to build best monitoring baselines.
All dashboards in the AI Observatory can be adjusted to a specific time frame. See dashboard documentation for specific functionality.
When integrated with solutions like MLFlow, the user can enable the archive of model training conditions metadata and create time travel functionality.
It’s not common to have a need for a backfill for a newly configured model. AI Observatory API supports backfill, which is most useful for: Making enough data available to build a good monitoring baseline Fix integration/ingestion hiccups and data outages