Utils¶

Iterators¶

datagrowth.utils.iterators.ibatch(iterable, batch_size, progress_bar=False, total=None)¶

Creates an iterator that iterates over an iterable in batches of a constant size. Each batch will be held in memory. Optionally this function will display a progress bar, showing the amount of iterated batches.

Parameters:

iterable – (iter) the iterator to batchify
batch_size – (int) the size of one batch
progress_bar – (bool) whether to display a progress bar
total – (int) the size of the iterator (only used for the progress bar)

Returns:

Iterator

Datetime¶

datagrowth.utils.datetime.format_datetime(datetime)¶

Formats a datetime into a string using DATAGROWTH_DATETIME_FORMAT parse_datetime_string and format_datetime consistently cast between strings and datetimes when used together.

Parameters:: datetime – a datetime object
Returns:: string representing the datetime

datagrowth.utils.datetime.parse_datetime_string(time_str)¶

Parses a time string to a datetime using DATAGROWTH_DATETIME_FORMAT. parse_datetime_string and format_datetime consistently cast between strings and datetimes when used together.

Parameters:: time_str – (str) a string representing a datetime
Returns:: datetime

Input/Output¶

datagrowth.utils.io.get_media_path(app_label, media_type='', absolute=True)¶

Returns a directory path for a particular app to store media in. Optionally this path can include a media type to further separate media files in subdirectories. By default the path is absolute and inside the DATAGROWTH_WEB_MEDIA_ROOT, but you can also return a path relative to the media root directory.

Parameters:

app_label – (str) the app label that you’re getting a media path for
media_type – (str) an optional media type that can further group media within apps
absolute – (bool) whether to return an absolute or relative path

Returns:

path to media directory without a trailing slash

datagrowth.utils.io.get_model_path(app_label, model_type='')¶

Returns a path to a directory inside the global data directory specified by DATAGROWTH_DATA_DIR. The idea is to store computer models inside this directory. The path will contain the app_label to make sure that related models are stored together. It optionally also includes a subdirectory for the model type in order to separate different computer models.

Parameters:

app_label – (str) the app label that is related to the computer model you’re getting a path for
model_type – (str) an optional model type that can further group models within apps

Returns:

path to models directory without a trailing slash

Data¶

datagrowth.utils.data.override_dict(parent, child)¶

A convenience function that will copy parent and then copy any items of child to that copy.

Parameters:

parent – (dict) the source dictionary to use as a base
child – (dict) a dictionary with items that should be added/overridden

Returns:

a copy of parent with added/overridden items from child

datagrowth.utils.data.reach(path: str | None, data: Any, default: any = None, default_factory: Callable[[], Any] | None = None) → Any¶

Reach takes a path and data structure. It will return the value from the data structure belonging to the path.

Paths are essentially multiple keys or indexes separated by . and start with $. Each part of a path should correspond to another level in the structure given.

Example data structure:

{
    "test": {"test": "second level test"},
    "list of tests": ["test0","test1","test2"]
}

In the example above $.test.test as path would return “second level test” while $.test.1 as path would return “test1”.

Reach will return None if path does not lead to a value in the data structure or the data structure entirely if path matches $.

Parameters:

path – (str) a key path starting with $ to find in the data structure
data – (dict, list or tuple) a data structure to search
default – (any) default value to return if no value was found, defaults to None
default_factory – (callable) factory for default value to return if no value was found

Returns:

value corresponding to path in data structure or the default