Table of Contents

• whylogs.app.writers
  • Writer
    • write
    • path_suffix
    • file_name
    • template_params
  • LocalWriter
    • write
    • ensure_path
  • S3Writer
    • write
  • WhyLabsWriter
    • write
  • writer_from_config

Classes for writing whylogs output

Writer Objects

class Writer(ABC)

Class for writing to disk

Parameters

output_path : str Prefix of where to output files. A directory for type = 'local', or key prefix for type = 's3' formats : list All output formats. See :data:whylogs.app.config.ALL_SUPPORTED_FORMATS path_template : str, optional Templatized path output using standard python string templates. Variables are accessed via $identifier or ${identifier}. See :func:Writer.template_params for a list of available identifers. Default = :data:DEFAULT_PATH_TEMPLATE filename_template : str, optional Templatized output filename using standardized python string templates. Variables are accessed via $identifier or ${identifier}. See :func:Writer.template_params for a list of available identifers. Default = :data:DEFAULT_FILENAME_TEMPLATE

write

 | @abstractmethod
 | write(profile: DatasetProfile, rotation_suffix: str = None)

Abstract method to write a dataset profile to disk. Must be implemented

path_suffix

 | path_suffix(profile: DatasetProfile)

Generate a path string for an output path from a dataset profile by applying the path templating defined in self.path_template

file_name

 | file_name(profile: DatasetProfile, file_extension: str, rotation_suffix: Optional[str] = None)

For a given DatasetProfile, generate an output filename based on the templating defined in self.filename_template

template_params

 | @staticmethod
 | template_params(profile: DatasetProfile) -> dict

Return a dictionary of dataset profile metadata which can be used for generating templatized variables or paths.

Parameters

profile : DatasetProfile The dataset profile

Returns

params : dict Variables which can be substituted into a template string.

Notes

Template params:

• name: name of the dataset
• session_timestamp: session time in UTC epoch milliseconds
• dataset_timestamp: timestamp for the data in UTC epoch ms
• session_id: Unique identifier for the session

LocalWriter Objects

class LocalWriter(Writer)

whylogs Writer class that can write to disk.

See :class:Writer for a description of arguments

write

 | write(profile: DatasetProfile, rotation_suffix: Optional[str] = None)

Write a dataset profile to disk

ensure_path

 | ensure_path(suffix: str, addition_part: typing.Optional[str] = None) -> str

Ensure that a path exists, creating it if not

S3Writer Objects

class S3Writer(Writer)

whylogs Writer class that can write to S3.

See :class:Writer for a description of arguments

write

 | write(profile: DatasetProfile, rotation_suffix: str = None)

Write a dataset profile to S3

WhyLabsWriter Objects

class WhyLabsWriter(Writer)

write

 | write(profile: DatasetProfile, rotation_suffix: str = None)

Write a dataset profile to WhyLabs

writer_from_config

writer_from_config(config: WriterConfig)

Construct a whylogs Writer from a WriterConfig

Returns

writer : Writer whylogs writer