Best Practices

This section presents a selection of best practices for using the RDMS. Adhering to these best practices will ensure the most optimal user experience with the RDMS.

The section will be gradually updated with new usage examples and tips.

If you believe that important information should be added to this section, please contact RDMS support with your request!

For the optimal usage of the RDMS, it is highly recommended to follow these best practices for naming your files/folders:

• Do not use special characters (e.g. $%^*&#=!) in your file/folder names.
• Do not use periods (.) in your file/folder names.
• Do not use quote symbols (' or “) in your file/folder names.
• Prefer usage of underscore (_) or hyphen (-) instead of white spaces in file/folder names.

Example of a folder structure with correct naming:

$ itree project_name
project_name
  analytical_data
    machine_01
      20231223_analysis.ext
      20240111_analysis.ext
      20240325_analysis.ext
    machine_02
      20230222_analysis.ext
      20230710_analysis.ext
      20240109_analysis.ext
    machine_03
      20231020_analysis.ext
      20231120_analysis.ext
      20231212_analysis.ext
  manuscripts
    publication_v01.odt

Example of a folder structure with incorrect naming:

$ itree "Project with XXX and YYY"
Project with XXX and YYY
  analytical_data
    analytical devices @ building 1
      experiment 100% scan rate.ext
      experiment 74% scan rate.ext
      experiment 80% scan rate.ext
    analytical devices @ building 2
      Experiment 01 by user.name@rug.nl.ext
      Experiment 02 by user.name@rug.nl.ext
      Experiment 03 by user.name@rug.nl.ext
    analytical devices @ building 3
      $1-100%.ext
      Versuch #1.ext
      Versuch_Öldiffusion_erste_Möglichkeit.ext
  manuscripts
    publication.final version.odt

To improve the performance of the RDMS, it is recommended to store data sets in numerous small files in a structured format like *.tar, *.tar.gz, *.tar.bz, or *.zip. This significantly improves transfer rates as the system engages in multi-threaded transfers after reaching a minimal file size threshold (32 MB). Transferring multiple smaller files furthermore results in big overhead, diminishing performance.

Best practices to handle such cases are:

1. First, collect all data locally.
2. Before archiving in the RDMS, bundle the data set or its subsets into a structured data format (as mentioned above).
3. Upload the bundled format to the RDMS.
4. (Optional) Add metadata if desired.
5. (Optional) Unbundle the data on the RDMS.

For extraction on the RDMS, CLI users can use the ibun -x command as also described in the iCommands for (Meta)data Management section of this wiki.

For RDMS web interface users, the “Uncompress tar” function, accessible via right-click on a *.tar file, enables extraction. Currently, this function supports only *.tar formats.

In rare cases, data may arrive in an incomplete form in the RDMS. This usually happens if a data transfer abruptly interrupted, for example due to connection problems, without proper finalization.

Restarting the data transfer may solve this issue. However, it is possible that the already transferred data remains in a locked state, causing problems when the transfer is restarted as those files cannot be overwritten directly.

If you experience these issues, it is recommended to contact RDMS-Support.

Users of the command-line tool iCommands have furthermore the possibility to detect such locked files directly using an appropriate CLI command.

In general, these issues manifest in HIERARCHY_ERRORs when a data transfer to the RDMS (e.g. via iput or irsync) is attempted via CLI.

To check all files at a RDMS location /rug/home/path/to/folder including all its subfolders, and to detect just those files that are marked as locked, the following command can be executed:

 $ iquest "status: %s, name: %s/%s" "SELECT DATA_REPL_STATUS, COLL_NAME, DATA_NAME WHERE COLL_NAME LIKE '/rug/home/path/to/folder%' AND DATA_REPL_STATUS > '1'"

This command will check the specified location for files which have a replica status of 2 (“read-locked”) or 3 (“write-locked”), and then output it in the format:

status: <2/3>. name: <path_to_folder>/<name_of_file>

While the locked files cannot be directly removed, they can still be moved first to another location in your home/team location, for example as a separate folder for locked files. Afterwards, the data transfer can be restarted.

Best practices to handle locked files and resolve the HIERARCH_ERROR are:

1. Create a new folder in your home or team drive to contain all locked files.
2. Use the 'iquest' command to identify locked files and move them to the newly created location. CLI users can utilize 'imv' for this purpose.
3. Restart the data transfer. The HIERARCHY_ERROR should be resolved.
4. If you accumulated multiple locked files in your folder which you cannot delete, please contact RDMS-Support and we will help you remove these.

Note: It is recommended not to contact RDMS support for every locked file, but instead first try to resolve it as described above. However, if numerous locked files are detected, you can directly contact RDMS support.