Skip to main content

Glossary

The following is a glossary of terms commonly used throughout the WhyLabs Documentation. Note that some of these terms have a special meaning in the context of whylogs/WhyLabs.

whylogs is a data logging library that captures statistical properties of data and ML models.

A batch is a collection of data points, often grouped by time.

In batch mode, whylogs processes a dataset in batches.

In streaming mode, whylogs processes individual data points.

A dataset is a collection of related data that will be analyzed together. In case of tabular data: each column of the table represents a particular variable, and each row represents a record of the dataset. When used alongside a statistical model, the dataset often represents features as columns, with additional columns for the output. For non-structured/complex data, the representation depends on the datatype.

A DatasetProfile is a collection of summary statistics and related metadata for a dataset that whylogs has processed.

Data Sketches are a class of algorithms that efficiently extract information from large or streaming datasets in a single pass. This term is sometimes used to refer specifically to the Apache DataSketches project.

A logger represents the whylogs tracking object for a given dataset (in batch mode) or a collection of data points (in streaming mode). A logger is always associated with a timestamp for its creation and a timestamp for the dataset. Different loggers may write to different storage systems using different output formats.

Metadata is data that describes either a dataset or information from whylogs’ processing of the dataset.

The whylogs output is available in the following formats: protobuf, JSON, and flat. Protobuf is a lightweight binary format that maps one-to-one with the memory representation of a whylogs object. JSON displays the protobuf data in JSON format. Flat outputs multiple files with both CSV and JSON content to represent different views of the data, including histograms, upper-bound, lower-bound, and frequent values. To apply advanced transformation on whylogs, we recommend using Protobuf.

A pipeline consists of the components data moves through, as well as any infrastructure associated with those components. A project may have multiple ML pipelines, but it’s common to have one pipeline for a multi-stage project.

A Project is any ML model, data pipeline, data stream, or dataset that you want to monitor in WhyLabs. Profiles generated by whylogs can be regularly uploaded to a particular WhyLabs project for ongoing monitoring. There are multiple types of projects available, including Models and Datasets.

A record is an observation of data. whylogs represents this as a map of keys (string data - feature names) to values (numerical/textual data).

A session represents your configuration for how your application interacts with whylogs, including logger configuration, input and output formats. Using a single session for your application is recommended.

A statistical profile is a collection of statistical properties of a feature. Properties can be different for discrete and continuous features. In the context of this documentation, "DatasetProfile" is a specific instance of a statistical profile which is specific to whylogs.

Prefooter Illustration Mobile
Run AI With Certainty
Get started for free
Prefooter Illustration