palgen.ingest.loader

Module Contents

class palgen.ingest.loader.Ingest

Ingest interface.

__slots__ = ()

abstract ingest(files)

Ingests files.

Parameters:

files (Iterable[Path]) – An iterable of Paths that should be ingested.

Yields:

tuple[Path, Any] – Tuple of a path to every ingested file and its content.

Return type:

Iterable[tuple[pathlib.Path, Any]]

__call__(files)

Parameters:

files (Iterable[pathlib.Path]) –

Return type:

Iterable[tuple[pathlib.Path, Any]]

class palgen.ingest.loader.Empty

Ingest interface.

__slots__ = ()

ingest(files)

Ingests files if they are empty. Warns if a file in the files iterable is not empty.

Parameters:

files (Iterable[Path]) – An iterable of Paths that should be ingested.

Yields:

tuple[Path, None] – Tuple of a path to every ingested file and None.

Return type:

Iterable[tuple[pathlib.Path, None]]

__call__(files)

Parameters:

files (Iterable[pathlib.Path]) –

Return type:

Iterable[tuple[pathlib.Path, Any]]

class palgen.ingest.loader.Raw

Ingest interface.

__slots__ = ()

ingest(files)

Ingests files as raw bytes.

Parameters:

files (Iterable[Path]) – An iterable of Paths that should be ingested.

Yields:

tuple[Path, byte] – Tuple of a path to every ingested file and its content.

Return type:

Iterable[tuple[pathlib.Path, bytes]]

__call__(files)

Parameters:

files (Iterable[pathlib.Path]) –

Return type:

Iterable[tuple[pathlib.Path, Any]]

class palgen.ingest.loader.Text(encoding='utf-8')

Ingest interface.

Text ingest.

Parameters:

encoding (str, optional) – Encoding to use when reading files. Defaults to ‘utf-8’.

__slots__ = ('encoding',)

ingest(files)

Ingests files as text.

Parameters:

files (Iterable[Path]) – An iterable of Paths that should be ingested.

Yields:

tuple[Path, Any] – Tuple of a path to every ingested file and its content.

Return type:

Iterable[tuple[pathlib.Path, str]]

__call__(files)

Parameters:

files (Iterable[pathlib.Path]) –

Return type:

Iterable[tuple[pathlib.Path, Any]]

class palgen.ingest.loader.Json

Ingest interface.

__slots__ = ()

ingest(files)

Ingests JSON files.

Parameters:

files (Iterable[Path]) – An iterable of Paths that should be ingested.

Yields:

tuple[Path, Any] – Tuple of a path to every ingested file and its content.

__call__(files)

Parameters:

files (Iterable[pathlib.Path]) –

Return type:

Iterable[tuple[pathlib.Path, Any]]

class palgen.ingest.loader.Toml

Ingest interface.

__slots__ = ()

ingest(files)

Ingests TOML files. This uses Python’s standard library tomllib if available.

Parameters:

files (Iterable[Path]) – An iterable of Paths that should be ingested.

Yields:

tuple[Path, dict[str, Any]] – Tuple of a path to every ingested file and its content.

Return type:

Iterable[tuple[pathlib.Path, dict[str, Any]]]

__call__(files)

Parameters:

files (Iterable[pathlib.Path]) –

Return type:

Iterable[tuple[pathlib.Path, Any]]