Custom Dashboards

Custom dashboards provide flexible options to view and compare metrics from the WhyLabs Observe and WhyLabs Secure portions of the AI Control Platform. Custom views across multiple resources and segments can be built that aren't possible elsewhere in the platform, and they provide the necessary visualization tools to enable debugging and root cause analysis.

A custom dashboard example showing Debt-to-Income (DTI) metrics across different segments

Accessing Custom Dashboards

Navigate from the Project Dashboard by clicking on the "Dashboards" tab located on the right side of the WhyLabs header.

From there you will land on the "My Dashboards" tab, which lists all custom dashboards that have been created by users in the organization, and provides the ability to create new custom dashboards.

The "My dashboards" page showing a list of custom dashboards in an organization

💡 You can access your custom dashboards from anywhere in the platform from the side menu. Simply click on the hamburger menu in the top left corner of the WhyLabs header (next to the logo), and select "My dashboards" from the side menu.

Managing Custom Dashboards

After navigating to the custom dashboard list (in the "My Dashboards" tab) page you can:

• Sort the dashboards by name, created by, created on, or last modified columns
• Access dashboards by clicking on the dashboard name
• Create, edit, and delete any of the dashboards

The date range column lets you understand if dashboard's date range is either "Live" or "Fixed".

• "Live" means the dashboard range is always relative to the current date. This updates when viewing the dashboard.
• "Fixed" means the dashboard range is fixed to a custom date range date with defined start and end dates, which are also shown in the column.

💡 How you set the date range when creating or editing a dashboard determines if its range is live or fixed. Changing between relative and user-defined custom ranges will update the live or fixed range status.

Setting the correct date range

The default range for new dashboards is a live range—set to 7-days trailing window—and the dashboard's date range can be considered a global parameter for all graphs. When building graphs with multiple resources that have different date ranges, you will need to set the dashboard date range to be inclusive of the start and end dates for all resources.

From the graph builder, you can determine the profile lineage (data range) for each resource from the resource dropdown in the plot options. The profile lineage is provided to help you determine the time frame for which data is available for each resource.

Creating a new Custom Dashboard

The new dashboard flow is intuitive and easy to use. After clicking the "New dashboard" button you will be dropped into the dashboard builder flow. Add a name for the dashboard, and then start adding graphs to the dashboard.

The new dashboard flow showing the dashboard name and empty state for the first graph

💡 Remember: custom dashboards are scoped to the organization, so they are visible to all users in the organization

Adding Graphs to a Custom Dashboard

Creating a new graph for your dashboard is as simple as selecting the resources, metrics, columns, and segments you want to visualize. There's no limit to the number of graphs that can be added to a dashboard.

Creating a new plot for the graph

Controls for selecting the type of visualization and adding a new plot to the graph

Select either a time series or pie chart as the type of visualization you want to create, then click the "Add plot" button. If you select the pie chart option, the plot options will be scoped to only the metrics that are available for pie chart visualization.

The plot builder makes it easy to configure the plot parameters for the graph. The following selections are required for configuring the plot:

• Resource selection: choose the resource (model or dataset) you want to visualize. This dropdown groups resources by their batch frequency (hourly, daily, etc.), and includes secondary fields for profile lineage and batch frequency
• Metric selection: choose the metric you want to visualize. This dropdown is scoped to the metrics available for the selected resource, and grouped into categories such as dataset metrics, column metrics, custom metrics, etc. Refer to the "supported metrics types" section below for the full list of metrics
• Column selection: choose the column that you want to visualize. This dropdown is scoped to the columns available for the selected resource and metric. If the selected metric is a dataset-level metric, the column selection will be disabled
• Segment selection (optional): choose the segments you want to visualize. This control is a multi-select dropdown that lets you dynamically build segment definitions based on the segment key-value pairs that are available for the selected resource and column.
  • Segment wildcard is supported, which allows you to select all segments for a given key. Contact WhyLabs support for more information on how enable this feature
• Display name (optional): the display name is used in the legend and tooltip for the plot. The default plot name is a concatenation of the plot parameters and can become long and hard to read, in which case you can provide a custom display name.

After configuring the plot, you can "Save" button to add the plot and the graph, or add another plot to the graph. Additional plots start as a clone of the prior plot, so you can easily make small changes to the plot configuration, such as changing one parameter in the plot.

Secondary fields in the selection dropdowns

The plot selection dropdowns have secondary fields that provide additional context for the resource and metric. These are especially useful when you have multiple resources with different profile lineages, or metrics from different sources.

Diagram showing the location of the secondary fields for the resource selector

Supported metric types

• Column metrics: these are the statistical metrics that are calculated for each column in the dataset. These include metrics such as total count, null count, unique ratio, mean, median, min, max, quartile measures, and standard deviation
• Dataset metrics: these are the performance metrics for the model and will be scoped to the selected resource and be based on the model type (classification metrics, regression metrics, etc.)
• Custom metrics: these are similar to dataset metrics, but are user-defined metrics that have been added to the model
• Drift metrics: these are the computed drift measure that have been calculated by monitors that have been analyzing the model or dataset. These metrics will be scoped to the columns specified in the monitor's target matrix
• Anomaly counts: these are the counts of anomalies that have been detected by monitors that have been analyzing the model or dataset. Similar to drift metrics, these metrics will be scoped to the columns specified in the monitor's target matrix
• LLM Secure metrics: these are aggregate metrics for GenAI applications that are onboarded to LLM Secure, and include policy violation counts, latency metrics, token lengths, and more.

Editing a Custom Dashboard

After adding plots and saving the graph to the dashboard, you can repeat the process to add as many graphs as you need to the dashboard. From the dashboard view, you can modify the order of the graphs with the move button, in addition to being able to edit, clone, or delete a graph. Hovering over a graph will reveal the controls.

Hover over a graph in the dashboard to access its control buttons