ScpDataLoader#

class polpo.preprocessing.ssh.ScpDataLoader(remote_path, ssh_client, data_dir=None, recursive=True, use_cache=True, local_basename=None)[source]#

Bases: PreprocessingStep, CacheableDataLoader

Transfer files and directories from remote host to localhost.

Parameters:

  • remote_path (str) – Path to retrieve from remote host.

  • ssh_client (SshClient) – Client handling connection.

  • data_dir (str) – Directory where to store data.

  • recursive (str) – Whether to transfer files and directories recursively.

  • use_cache (bool) – Whether to verify if data is already available locally.

  • local_basename (str) – Basename of transferred file/folder if different from remote host.

apply(data=None)[source]#

Apply step.

load()[source]#

Load data.