ibridges.data_operations.sync

ibridges.data_operations.sync(source, target, max_level=None, dry_run=False, on_error='fail', copy_empty_folders=False, resc_name='', options=None, metadata=None, progress_bar=True)

Synchronize data between local and remote copies.

The command can be in one of the two modes: synchronization of data from the client’s local file system to iRODS, or from iRODS to the local file system. The mode is determined by the type of the values for source and target: objects with type ibridges.path.IrodsPath will be interpreted as remote paths, while types str and Path with be interpreted as local paths.

Files/data objects that have the same checksum will not be synchronized.

Parameters:

  • source (Union[str, Path, IrodsPath]) – Existing local folder or iRODS collection. An exception will be raised if it doesn’t exist.

  • target (Union[str, Path, IrodsPath]) – Existing local folder or iRODS collection. An exception will be raised if it doesn’t exist.

  • max_level (Optional[int]) – Controls the depth up to which the file tree will be synchronized. A max level of 1 synchronizes only the source’s root, max level 2 also includes the first set of subfolders/subcollections and their contents, etc. Set to None, there is no limit (full recursive synchronization).

  • dry_run (bool) – List all source files and folders that need to be synchronized without actually performing synchronization.

  • ignore_err – If an error occurs during the transfer, and ignore_err is set to True, any errors encountered will be transformed into warnings and iBridges will continue to transfer the remaining files.

  • on_error (str) – When a transfer of a file fails, by default the whole transfer will stop and print the error message(fail). By setting ‘on-error’ to ‘warn’, those errors will be turned into warnings and the transfer continues with the next file. Setting ‘on-error’ to ‘skip’ will omit any message and simply proceed.

  • copy_empty_folders (bool) – Controls whether folders/collections that contain no files or subfolders/subcollections will be synchronized.

  • resc_name (str) – Name of the resource from which data is downloaded, by default the server will decide.

  • options (Optional[dict]) – Python-irodsclient options found in irods.keywords. The following keywords will be ignored since they are set by iBridges: FORCE_FLAG_KW, RESC_NAME_KW, NUM_THREADS_KW, REG_CHKSUM_KW, VERIFY_CHKSUM_KW.

  • metadata (Union[None, str, Path, dict]) – If not None, the location to get the metadata from or store it to.

  • progress_bar (bool) – Whether to display a progress bar.

Raises:

  • CollectionDoesNotExistError: – If the source collection does not exist

  • NotACollectionError: – If the source is a data object.

  • NotADirectoryError: – If the local source is not a directory.

Return type:

Operations

Returns:

An operations object to execute the sync if dry-run is True.

Examples

>>> # Below, all files/dirs in "some_local_dir" will be transferred into "some_remote_coll"
>>> sync("some_local_dir", IrodsPath(session, "~/some_remote_col")

>>> # Below, all data objects/collections in "col" will tbe transferred into "some_local_dir"
>>> sync(IrodsPath(session, "~/col"), "some_local_dir")