FigshareDataLoader

class polpo.preprocessing.load.FigshareDataLoader(figshare_id, remote_path, data_dir=None, use_cache=True, local_basename=None, version=1, remove_id=True)

Bases: PreprocessingStep, CacheableDataLoader

Transfer files and directories from figshare.

Parameters:

• figshare_id (int) – Id of figshare article.

• remote_path (str) – Path to retrieve from remote host.

• data_dir (str) – Directory where to store data.

• use_cache (bool) – Whether to verify if data is already available locally.

• local_basename (str) – Basename of transferred file/folder if different from remote host.

• version (int) – Dataset version.

• remove_id (bool) – Whether to remove figshare added id when downloading items that are within a folder.

Notes

• If remote_path does not exist in figshare, it automatically downloads

the full dataset. Code will fail after download. Warning will give information about downloaded zip. Can’t know in advance if this will happen.

apply(data=None)

Apply step.

load()

Load data.