Table of Contents

• whylogs.io.file_loader
  • valid_file
  • extension_file
  • image_loader
  • json_loader
  • file_loader

valid_file

valid_file(fname: str)

simple check if extension is part of the implemented ones

Arguments:

• fname str - file path

Returns:

bool

extension_file

extension_file(path: str)

Check the enconding format based on the magic number if file has no magic number we simply use extension. More advance analytics of file content is needed, potentially extendind to a lib like libmagic

Arguments:

• path str - File path

Returns:

• file_extension_given - str: extension of encoding data magic_data : dic : any magic data information available including magic number : byte
• mime_type - str name : str

image_loader

image_loader(path: str)

tries to load image using the PIL lib

Arguments:

• path str - path to image files

Returns:

PIL.Image.Image : image data and image encoding format

json_loader

json_loader(path: str = None) -> Union[Dict, list]

Loads json or jsonl data

Arguments:

• path str, optional - path to file

Returns:

objs : Union[Dict, list]: Returns a list or dict of json data json_format : format of file (json or jsonl)

file_loader

file_loader(path: str, valid_file: Callable[[str], bool] = valid_file) -> Any

Factory for file data

Arguments:

• path str - path to file
• valid_file Callable[[str], bool], optional - Optional valid file check,

Returns:

data : Tuple( [] Dataframe or Image data (PIL format), or Dict],

• magic_data - Dict of magic number data)

Raises:

• NotImplementedError - Description