Profile Metrics
This section describes the profile metrics available for monitoring in the WhyLabs AI Control Center. Specifically, these are the metrics that can be used in the monitor configuration described in the Advanced Monitor Configuration section. Most of these metrics can also be queried using the WhyLabs Data API.
Column Metrics
Column metrics can be monitored for specific columns in the dataset, and must be used in an analyzer with a targetMatrix
of type column
as shown in the Targeting Columns section.
Distribution Metrics
The distribution metrics can be used in analyzers of type drift
. The frequent_items
metric can also
be used with an analyzer of type frequent_string_comparison
.
Metric name | Description | Available in Data Api? |
---|---|---|
frequent_items | A complex metric representing the counts of the most frequent discrete values in the column | No |
histogram | A complex metric representing the numeric distribution of values in the column using counts of binned numeric values | No |
Statistical Value Metrics
These statistical metrics can be used in analyzers with types of diff
, fixed
, seasonal
and stddev
.
Metric name | Description | Available in Data Api? |
---|---|---|
min | The minimum value in the column | Yes |
max | The maximum value in the column | Yes |
mean | The mean of values in the column | Yes |
median | The median value (i.e. 50th percentile) of the values in the column | Yes |
quantile_5 | The 5th percentile value of the column | Yes |
quantile_25 | The 25th percentile value of the column | Yes |
quantile_75 | The 75th percentile value of the column | Yes |
quantile_95 | The 95th percentile value of the column | Yes |
quantile_99 | The 99th percentile value of the column | Yes |
std_dev | The standard deviation of the values in the column | Yes |
variance | The variance of the values in the column | Yes |
Count Metrics
Metric name | Description | Available in Data Api? |
---|---|---|
count | The count of values in the column | Yes |
count_null | The count of missing/null/NaN values in the column | Yes |
count_null_ratio | The ratio of missing/null/NaN values in the column | Yes |
unique_est | An estimate of the count of unique values in the column | Yes |
unique_est_ratio | The ratio of the unique value count estimate for the column | Yes |
unique_est_lower | The lower bound on the unique value count estimate for the column | Yes |
unique_est_upper | The upper bound on the unique value count estimate for the column | Yes |
count_bool | The count of boolean values in the column | Yes |
count_bool_ratio | The ratio of boolean values in the column | Yes |
count_integral | The count of integer values in the column | Yes |
count_integral_ratio | The ratio of integer values in the column | Yes |
count_fractional | The count of fractional values in the column | Yes |
count_fractional_ratio | The ratio of fractional values in the column | Yes |
count_string | The count of string values in the column | Yes |
count_string_ratio | The ratio of string values in the column | Yes |
Other Column Metrics
The inferred_data_type metric can be used in analyzers of type comparison
.
Metric name | Description | Available in Data Api? |
---|---|---|
inferred_data_type | The inferred data type of the values in the column | No |
Dataset Metrics
These metrics can be monitored for the overall dataset, and must be used in an analyzer with a targetMatrix
of type dataset
as shown in the Targeting Datasets section.
Classification Metrics
These metrics can be used in analyzers of type diff
, fixed
and stddev
, providing classification model metrics have been
uploaded for the dataset.
Metric name | Description | Available in Data Api? |
---|---|---|
classification.f1 | F1 score | Yes |
classification.precision | Precision | Yes |
classification.recall | Recall | Yes |
classification.accuracy | Accuracy | Yes |
classification.auroc | Area under the receiver-operator curve | Yes |
Regression Metrics
These metrics can be used in analyzers of type diff
, fixed
and stddev
, providing regression model metrics have been
uploaded for the dataset.
Metric name | Description | Available in Data Api? |
---|---|---|
regression.mse | Mean squared error | Yes |
regression.mae | Mean absolute error | Yes |
regression.rmse | Root mean square error | Yes |
Integration Health Metrics
These metrics should only be used in an analyzer of type fixed
.
Metric name | Description | Available in Data Api? |
---|---|---|
missingDatapoint | 0 if the last batch contained profile data; 1 if it is missing | No |
secondsSinceLastUpload | Number of seconds elapsed since the last profile upload | No |