Configure iCommands
Installing iCommands
- Installation on Ubuntu and Debian
- Installation on EL-based Distributions
Using iCommands

iCommands

The complete functionality to the RDMS comes with iCommands on the command-line interface (CLI).

iCommands is the official CLI client by iRODS, the open-source software upon which the RDMS is built.
iCommands consist of a whole set of different commands that allow to interact with the RDMS from the CLI. Next to commands relevant to general users (navigation, data transfer, etc.), it has commands specifically for metadata, and commands for management of groups.
iCommands are already installed on the LWP and Habrók HPC cluster. In these environments, you just need to:
1. Create a ~\.irods directory in your home directory.
2. Create an irods_environment.json file in that directory.
For other Linux distributions, explanation to install iCommands are also explained in this section.

Configure iCommands

For iCommands to work properly, a configuration file is needed. If the file does not exists on your system, the configuration file can be setup in the following way:

1. Create the .irods directory in your Linux home directory (~ is short for /home/<username>/ under Linux):

$ mkdir ~/.irods

2. Create the configuration file:

Create the file ~/.irods/irods_environment.json using your preferred text editor. The file must contain plain ASCII characters. Add the following lines to the file, modifying as instructed below:

Click to display configuration file

{
    "irods_authentication_scheme": "pam_password",
    "irods_client_server_negotiation": "request_server_negotiation",
    "irods_client_server_policy": "CS_NEG_REQUIRE",
    "irods_default_resource": "rootResc",
    "irods_encryption_algorithm": "AES-256-CBC",
    "irods_encryption_key_size": 32,
    "irods_encryption_num_hash_rounds": 16,
    "irods_encryption_salt_size": 8,
    "irods_host": "store.data.rug.nl",
    "irods_port": 1247,
    "irods_ssl_verify_server": "cert",
    "irods_user_name": "<emailadress>",
    "irods_zone_name": "rug"
}

Notes:

The iRODS username <emailadress> should be replaced by your UG email address.
Your username is case-sensitive, make sure to use all lowercase letters!

To see if everything was properly configures, you can now start the session by invoking the iinit command which will initialize your connection to the RDMS:

$ iinit
Enter your current PAM password:

If you do not see an error after specifying your password, you are set to start using iCommands. See the section below for more information on using iCommands including some examples.

Note:
The iinit command will not just initialize your connection to the RDMS, but will also store your password in an obfuscated format under ~/.irods/.irodsA. Therefore, you do not have to re-type your password each time a subsequent command is executed during the same session.

Installing iCommands

Packages for iCommands are currently provided for CentOS 7, EL 8/9 (e.g., Rocky Linux or AlmaLinux), Ubuntu 18/20/22 and Debian 11/12 (https://irods.org/download/). iCommands can be installed on other Linux distributions too, but special configurations will be needed. For such cases, we recommend the use of a virtual machine that runs one of the officially supported Linux distributions.

Installation on Ubuntu and Debian

Install the public key and add the iRODS repository

$ wget -qO - https://packages.irods.org/irods-signing-key.asc | sudo apt-key add -
$ echo "deb [arch=amd64] https://packages.irods.org/apt/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/renci-irods.list
$ sudo apt-get update

Install iCommands

$ sudo apt install irods-icommands

Additional Notes for Users of Ubuntu 20+ and Debian 11+

On Debian-based operating systems, the apt-key add feature was deprecated, and you might see the following message in the first step:

$ wget -qO - https://packages.irods.org/irods-signing-key.asc | sudo apt-key add -
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
OK

As of now, the warning can be still ignored and the steps can be performed as described, but in the longer term it is highly advisable to add the repository signing key in another manner as described, for example, here. As of now, iRODS has not yet decided on the best way to handle this in the case of the iRODS package repository, and this information will be updated accordingly in the future.

Installation on EL-based Distributions

Before you can install iCommands on a RedHat-based Linux distributions (CentOS, Rocky Linux, and AlmaLinux), be sure that the necessery dependencies can be satisfied.

Install the Extra Packages for Enterprise Linux (EPEL) repository

$ sudo yum install epel-release

Install external dependencies

$ sudo yum install python-psutil python-jsonschema

Install the public key and add the iRODS repository

$ sudo rpm --import https://packages.irods.org/irods-signing-key.asc
$ wget -qO - https://packages.irods.org/renci-irods.yum.repo | sudo tee /etc/yum.repos.d/renci-irods.yum.repo

Install iCommands

$ sudo yum install irods-icommands

Using iCommands

In this section, we will describe the usage of different available iCommands in more detail. Examples will be given for the different iCommands showing their possible usage. Please keep in mind that the iCommands provide extensive functionality and not all will be described in this wiki.

Furthermore, different iCommands can be included in (computing) scripts to automate tasks if needed which will not covered in this section as it is specific for the respective use case.

If you have additional question about functionality of iCommands in general or specific subcommands, you can contact rdms-support@rug.nl.

Next to the information provided on this wiki, a plethora of external guides on iCommands and its usage exists. Some important to mention are:

Getting Help for specific iCommands

As with linux commands, there is built in help for all iCommands which can be shown by using the addition -h flag with the respective command. For example, help for the ils command is displayed like:

$ ils -h

which will display the help of the respective command.

Click to display ⇲

Click to hide ⇱

$ ils -h
Usage: ils [-ArlLv] dataObj|collection ...
Usage: ils --bundle [-r] dataObj|collection ...
Display data objects and collections stored in iRODS.

The following is typical output for ils with no arguments:
$ ils
/tempZone/public:
big
foo
The only information displayed are the names of data objects in the current
working collection.

The long format (given by -l) will display some more information:
$ ils -l
/tempZone/public:
rods 0 demoResc 41943040 2021-04-29.17:57 & big
rods 2 comp;arch 40236581 2021-04-19.03:00 X big
alice 2 repl1;child1 283 2021-04-29.17:54 ? foo
alice 3 repl1;child2 283 2021-04-29.17:56 ? foo
alice 4 repl1;child3 283 2021-04-29.17:57 ? foo
This displays replica information and system metadata for the replicas of
the data objects in the current working collection. This information includes
the owner of the data object and the replica's number, resource hierarchy, size,
last modified time, and status.

The very long format (given by -L) will display even more information:

$ ils -L
/tempZone/public:
rods 0 demoResc 41943040 2021-04-29.17:57 & big
sha2:gKNyEYjkAhiwiyZ3a8U72ugeR4T/9x1xRQoZcxnLoRM= generic /var/lib/irods/Vault/public/big
rods 2 comp;arch 40236581 2021-04-19.03:00 X big
sha2:KNyEYjkhiwyZ3a8U72ugeR4T/9x1xRQoZcxnLoRMigg= generic /var/lib/irods/archVault/public/big
alice 2 repl1;child1 283 2021-04-29.17:54 ? foo
generic /var/lib/irods/child1vault/public/foo
alice 3 repl1;child2 283 2021-04-29.17:56 ? foo
generic /var/lib/irods/child2vault/public/foo
alice 4 repl1;child3 283 2021-04-29.17:57 ? foo
generic /var/lib/irods/child3vault/public/foo
This displays even more replica information and system metadata such as each
replica's checksum, data type, and physical location in the storage resource.

The different replica statuses are represented by the following characters
which can be seen between the last modified time and the data object name.
0. X: stale - The replica is no longer known to be good. Usually
indicates that a more recently written-to replica exists.
This does not necessarily mean that the data is incorrect
but merely that it may not reflect the "truth" of this
data object.
1. &: good - The replica is good. Usually the latest replica to be
written. When a replica is good, all bytes and system
metadata (size and checksum, if present) are understood
to have been recorded correctly.
2. ?: intermediate - The replica is actively being written to. The state of
the replica cannot be determined because the client
writing to it still has the replica opened. Replicas
which are intermediate cannot be opened for read or
write, nor can they be unlinked or renamed.
3. ?: write-locked - One of this replica's sibling replicas is actively
being written to but is itself at rest. Replicas which
are write-locked cannot be opened for read or write
nor can they be unlinked or renamed.
These replica statuses correspond with the data_is_dirty column in r_data_main.

Options are:
-A ACL (access control list) and inheritance format
-d List collections themselves, not their contents
-l long format
-L very long format
-r recursive - show subcollections
-t ticket - use a read (or write) ticket to access collection information
-v verbose
-V Very verbose
-h this help
--bundle - list the subfiles in the bundle file (usually stored in the
/myZone/bundle collection) created by iphybun command.

iRODS Version 4.3.0 ils

Furthermore, you can run ihelp to get an overview of all available iCommands.

Common iCommands Usage Examples

For the explanation of commonly used iCommands, we will split the iCommands into three different sections:

iCommands for Navigation will describe the iCommands that are used to navigate the RDMS from the CLI, but also contain information about starting/ending of the connection.
iCommands for (Meta)data Management will describe how to transfer data in/out of the RDMS, explain how to set permissions for other users, how to work with metadata from the CLI, including a more detailed description on how to query the database for more extensive searches.
iCommands for Group Management will focus on the respective commands that can be used by 'groupadmins' to manage RDMS groups from the CLI. As this section is not relevant for all users, it will be discussed separately.

iCommands for Navigation

We already saw the iinit command in action which initializes a new RDMS session and also stored the obfuscated password for re-use with following iCommands.

To again close a session from the CLI, the iexit command can be used.

If a session is initialized, we can use the ipwd subcommand to display our current working directory within the RDMS:

Click to display ⇲

Click to hide ⇱

$ ipwd 
/rug/home/rdms-testers@rug.nl

To reveal the content of a directory, we can use the ils <path>. When executing ils without specifying an additional RDMS path, it will show us the content of our current working directory.

Click to display ⇲

Click to hide ⇱

$ ils 
/rug/home/rdms-testers@rug.nl:
  example_file
  C- /rug/home/rdms-testers@rug.nl/some_example_folder
  C- /rug/home/rdms-testers@rug.nl/some_other_folder

When used without additional flags, ils will just list files and folder (folders = collections in iRODS terms, marked by 'C-' in front of data name), but will not provide any additional information about file sizes or permission information. If we want to see these information, we can use the additional -A flag to get information about file permissions and inheritance, and the -l or -L flag for (very) long output which will display additional information, for example file sizes. As usual, flags can be combined for a specific command.

Click to display ⇲

Click to hide ⇱

$ ils -A
/rug/home/rdms-testers@rug.nl:
        ACL - rdms-testers@rug.nl#rug:own   
        Inheritance - Disabled
  example_file
        ACL - rdms-testers@rug.nl#rug:own   
  example_file2
        ACL - rdms-testers@rug.nl#rug:own   
  C- /rug/home/rdms-testers@rug.nl/some_example_folder
  C- /rug/home/rdms-testers@rug.nl/some_other_folder

$ ils -l
/rug/home/rdms-testers@rug.nl:
  rdms-testers      0 rootResc;rootRandy;ptC;replC;randy20;pt223;mnt_irods223            0 2023-07-21.13:05 & example_file
  rdms-testers      1 rootResc;rootRandy;ptC;replC;randy11;pt124;mnt_irods124            0 2023-07-21.13:05 & example_file
  rdms-testers      0 rootResc;rootRandy;ptC;replC;randy11;pt103;mnt_irods103           44 2023-07-21.13:08 & example_file2
  rdms-testers      1 rootResc;rootRandy;ptC;replC;randy20;pt223;mnt_irods223           44 2023-07-21.13:08 & example_file2
  C- /rug/home/rdms-testers@rug.nl/some_example_folder  
  C- /rug/home/rdms-testers@rug.nl/some_other_folder 

$ ils -lA
/rug/home/rdms-testers@rug.nl:
        ACL - rdms-testers@rug.nl#rug:own   
        Inheritance - Disabled
  rdms-testers      0 rootResc;rootRandy;ptC;replC;randy20;pt223;mnt_irods223            0 2023-07-21.13:05 & example_file
        ACL - rdms-testers@rug.nl#rug:own   
  rdms-testers      1 rootResc;rootRandy;ptC;replC;randy11;pt124;mnt_irods124            0 2023-07-21.13:05 & example_file
        ACL - rdms-testers@rug.nl#rug:own   
  rdms-testers      0 rootResc;rootRandy;ptC;replC;randy11;pt103;mnt_irods103           44 2023-07-21.13:08 & example_file2
        ACL - rdms-testers@rug.nl#rug:own   
  rdms-testers      1 rootResc;rootRandy;ptC;replC;randy20;pt223;mnt_irods223           44 2023-07-21.13:08 & example_file2
        ACL - rdms-testers@rug.nl#rug:own   
  C- /rug/home/rdms-testers@rug.nl/some_example_folder  
  C- /rug/home/rdms-testers@rug.nl/some_other_folder

The ils command can be also used with the additional -r to perform a recursive operation, in which case it will not just display the output of the current RDMS directory, but instead also the content of its lower-level folders.

Click to display ⇲

Click to hide ⇱

$ ils -r
/rug/home/rdms-testers@rug.nl:
  example_file
  example_file2
  C- /rug/home/rdms-testers@rug.nl/some_example_folder
/rug/home/rdms-testers@rug.nl/some_example_folder:
  C- /rug/home/rdms-testers@rug.nl/some_other_folder
/rug/home/rdms-testers@rug.nl/some_other_folder:
  C- /rug/home/rdms-testers@rug.nl/some_other_folder/a_subfolder1
/rug/home/rdms-testers@rug.nl/some_other_folder/a_subfolder1:
  more_example_files_1
  more_example_files_10
  more_example_files_2
  more_example_files_3
  more_example_files_4
  more_example_files_5
  more_example_files_6
  more_example_files_7
  more_example_files_8
  more_example_files_9

Alternatively, the itree command can be executed to get a similar overview of the folder/file structure at a specific RDMS location. Moreover, itree also displays a summary of files (data objects) and folder (collections) that were found at the specified location. Running itree without specifying a path will again display results for the current working directory.

Click to display ⇲

Click to hide ⇱

$ itree /rug/home/rdms-testers@rug.nl/some_other_folder/a_subfolder1
a_subfolder1
  more_example_files_1
  more_example_files_10
  more_example_files_2
  more_example_files_3
  more_example_files_4
  more_example_files_5
  more_example_files_6
  more_example_files_7
  more_example_files_8
  more_example_files_9
Found 0 collections and 10 data objects

If we want to navigate to another location inside of the RDMS, we can use the icd command. For navigation with icd the destination path can be either specified relatively to the current working directory or given as an absolute path. Furthermore, executing icd without any additional path, will always navigate to your RDMS home location.

Click to display ⇲

Click to hide ⇱

# Example using icd and relative file path
$ ils
/rug/home/rdms-testers@rug.nl:
  example_file
  example_file2
  C- /rug/home/rdms-testers@rug.nl/some_example_folder
  C- /rug/home/rdms-testers@rug.nl/some_other_folder
$ icd some_example_folder
$ ipwd
/rug/home/rdms-testers@rug.nl/some_example_folder

# Example using icd and absolute file path
$ icd /rug/home/rdms-testers@rug.nl/some_other_folder/a_subfolder1
$ ipwd
/rug/home/rdms-testers@rug.nl/some_other_folder/a_subfolder1

# Using icd without path, always navigates to RDMS Home directory
$ icd
$ ipwd
/rug/home/rdms-testers@rug.nl

iCommands for (Meta)data Management

Upload to the RDMS
To transfer data to the the RDMS, the iput command can be used. It is of the general syntax iput [flags] <source> <dest> where <source> points to files/folders on your local environment while <dest> specifies the destination on the RDMS. The destination entry can be also left empty in which case the data will be transferred to the current RDMS working directory. Specified paths can be relative or absolute.

Click to display ⇲

Click to hide ⇱

# Transfer a file to the RDMS
$ iput test.txt /rug/home/rdms-testers@rug.nl/some_example_folder
$ ils /rug/home/rdms-testers@rug.nl/some_example_folder
/rug/home/rdms-testers@rug.nl/some_example_folder:
  test.txt

If you want to transfer a whole folder structure, the additional -r flag has to be used.

Click to display ⇲

Click to hide ⇱

$ iput -r Testdir_L1
Running recursive pre-scan... pre-scan complete... transferring data...
$ itree Testdir_L1
Testdir_L1
  Testdir_L2a
    Testfile_1
    Testfile_2
    Testfile_3
    Testfile_4
    Testfile_5
  Testdir_L2b
    Testfile_1
    Testfile_2
    Testfile_3
    Testfile_4
    Testfile_5
  Testdir_L2c
    Testfile_1
    Testfile_2
    Testfile_3
    Testfile_4
    Testfile_5
  Testdir_L2d
    Testfile_1
    Testfile_2
    Testfile_3
    Testfile_4
    Testfile_5
  testfile
Found 4 collections and 21 data objects

If you want to see the transfer of the data transfer displayed, you can use the additional -P flag.

Click to display ⇲

Click to hide ⇱

$ iput -P 1GB_Testfile_1
0/1 -  0.00% of files done   0.000/953.674 MB -  0.00% of file sizes done
Processing 1GB_Testfile_1 - 953.674 MB   2023-07-21.14:21:16
1GB_Testfile_1 - 360.000/953.674 MB - 37.75% done   2023-07-21.14:21:23
1GB_Testfile_1 - 796.837/953.674 MB - 83.55% done   2023-07-21.14:21:26
1GB_Testfile_1 - 953.674/953.674 MB - 100.00% done   2023-07-21.14:21:26

Notes:

It is also recommended to use the -K which will calculate and compare the checksum of the files that are transferred.
For long-lasting transfers, it is furthermore recommended to use the -T flag which will renew the socket connection after 10 minutes which can eliminate possible errors resulting from time-outs due to firewall settings.
An additional option exists to specify a restart file using the -X flag. For uploading directories, this can be used to restart the transfer if needed. This can be combined with the –retries [N] flag to automatically restart the transfer [N] times if needed.
For improved transfer performance, especially for data sets containing a lot of small, individual files, it is recommend to upload the files in a bundled format (e.g., tar archive) and then bundle within the RDMS if desired (see the section about (un)bundling below).
If you want to overwrite a file that is already present in the system, use the -f (force) flag to overwrite, but be cautious when overwriting data.

Download from the RDMS
For downloads of data from the RDMS, the iget command can be used. Its general syntax, iget [flags] <source> <dest>, is identical to iput, except that <source> is this time the RDMS side while <dest> specifies the destination on your local file system.

Also, the same flags can be used and the above mentioned notes for the usage of iput also apply for iget.

Synchronization with the RDMS
Another method for the transfer of data is the usage of the irsync command which synchronizes a source directory with a target directory. The distinction is made by placing a i: in front of the source/destination paths.

Click to display ⇲

Click to hide ⇱

# Synchronize the content of local folder to a folder on the RDMS:
$ irsync -r <source> i:<dest>

Synchronize the content of remote folder on the RDMS with a local folder:
$ irsync -r i:<source> <dest>

Synchronize the content of two remote RDMS folders:
$ irsync -r i:<source> i:<dest>

Notes:

The irsync command also allows to specify multiple sources that can be synced to one destination.
To see detailed information about the progress of the synchronization, the additional -v (verbose) flag can be used.
The -K flag can be used to calculate and compare checksum for the transferred files on the source and destination side. This is recommended.
The irsync command does not have a specific force flag to overwrite files. Instead, irsync works in a way that it detects new/changed files on the specified source and uploads them to the target destination. Already existing files that were not changed are not touched/transferred by irsync while changed files get overwritten on the destination!

With this note in mind, a typical irsync command with recommend flags would look like:

Click to display ⇲

Click to hide ⇱

$ irsync -rvK git_projects/nmr_meta_parser/ i:/rug/home/rdms-testers@rug.nl/nmr_meta_parser/
Running recursive pre-scan... pre-scan complete... transferring data...
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser:
   nmr_parser.py                   0.001 MB | 2.293 sec | 0 thr |  0.000 MB/s
   nmr_meta_main.py                0.001 MB | 1.270 sec | 0 thr |  0.001 MB/s
   nmr_meta_binder.py              0.007 MB | 2.514 sec | 0 thr |  0.003 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/parsers:
   experiment_parser.py            0.001 MB | 1.650 sec | 0 thr |  0.000 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/parsers/__pycache__:
   bruker.cpython-310.pyc          0.004 MB | 1.205 sec | 0 thr |  0.003 MB/s
   experiment_parser.cpython       0.001 MB | 1.613 sec | 0 thr |  0.001 MB/s
   varian.cpython-310.pyc          0.004 MB | 0.417 sec | 0 thr |  0.009 MB/s
   varian.py                       0.005 MB | 0.241 sec | 0 thr |  0.020 MB/s
   bruker.py                       0.004 MB | 0.242 sec | 0 thr |  0.017 MB/s
   __init__                        0.000 MB | 0.201 sec | 0 thr |  0.000 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/__pycache__:
   parse_utils.cpython-310.p       0.002 MB | 0.321 sec | 0 thr |  0.006 MB/s
   nmr_meta_cl_parser.cpytho       0.003 MB | 0.341 sec | 0 thr |  0.008 MB/s
   nmr_meta_binder.cpython-3       0.006 MB | 0.232 sec | 0 thr |  0.027 MB/s
   nmr_parser.cpython-310.py       0.001 MB | 0.378 sec | 0 thr |  0.002 MB/s
   requirements.txt                0.000 MB | 0.483 sec | 0 thr |  0.000 MB/s
   .gitignore                      0.000 MB | 0.254 sec | 0 thr |  0.000 MB/s
   README.md                       0.004 MB | 0.440 sec | 0 thr |  0.010 MB/s
   parse_utils.py                  0.001 MB | 1.770 sec | 0 thr |  0.001 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git:
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/objects:
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/objects/5c:
   d67aca348a0e4758231637733       0.000 MB | 0.522 sec | 0 thr |  0.001 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/objects/a6:
   244a57cf70a69c6731243888c       0.000 MB | 0.523 sec | 0 thr |  0.001 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/objects/c4:
   69d8c5b6502b25ea021967a40       0.000 MB | 0.623 sec | 0 thr |  0.000 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/objects/01:
   caaf0720045c0265dd835cca0       0.001 MB | 0.809 sec | 0 thr |  0.002 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/objects/8a:
   df0d2bfd9d1f3da44afb66775       0.000 MB | 0.836 sec | 0 thr |  0.000 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/objects/17:
   65841eab7cea2807e8be2915d       0.001 MB | 0.403 sec | 0 thr |  0.001 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/objects/4c:
   061cd396da8e40b03325fa520       0.000 MB | 0.364 sec | 0 thr |  0.000 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/objects/info:
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/objects/d7:
   337483d3a05065e3fffe4cf53       0.000 MB | 0.284 sec | 0 thr |  0.001 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/objects/9f:
   759a7d969b2e19f619cbc7c14       0.000 MB | 0.367 sec | 0 thr |  0.000 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/objects/80:
   f8218ef959908ebc47b02a24d       0.001 MB | 0.257 sec | 0 thr |  0.005 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/objects/97:
   bd6ee70910937174233828393       0.001 MB | 0.272 sec | 0 thr |  0.002 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/objects/15:
   43b4fcb9a03324daca1a5d87d       0.000 MB | 0.273 sec | 0 thr |  0.001 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/objects/4b:
   83c7605922dd90e79cfc1e145       0.000 MB | 0.334 sec | 0 thr |  0.001 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/objects/pack:
   pack-08f6731e945a0424f737       0.079 MB | 0.313 sec | 0 thr |  0.251 MB/s
   pack-08f6731e945a0424f737       0.011 MB | 0.249 sec | 0 thr |  0.042 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/branches:
   config                          0.000 MB | 0.209 sec | 0 thr |  0.001 MB/s
   FETCH_HEAD                      0.000 MB | 0.202 sec | 0 thr |  0.000 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/logs:
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/logs/refs:
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/logs/refs/remotes:
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/logs/refs/remotes/origin:
   develop                         0.000 MB | 0.446 sec | 0 thr |  0.000 MB/s
   main                            0.000 MB | 0.287 sec | 0 thr |  0.001 MB/s
   HEAD                            0.000 MB | 0.268 sec | 0 thr |  0.001 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/logs/refs/heads:
   main                            0.000 MB | 0.477 sec | 0 thr |  0.001 MB/s
   HEAD                            0.000 MB | 0.302 sec | 0 thr |  0.001 MB/s
   description                     0.000 MB | 0.209 sec | 0 thr |  0.000 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/refs:
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/refs/remotes:
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/refs/remotes/origin:
   develop                         0.000 MB | 0.240 sec | 0 thr |  0.000 MB/s
   main                            0.000 MB | 0.244 sec | 0 thr |  0.000 MB/s
   HEAD                            0.000 MB | 0.248 sec | 0 thr |  0.000 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/refs/tags:
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/refs/heads:
   main                            0.000 MB | 0.258 sec | 0 thr |  0.000 MB/s
   packed-refs                     0.000 MB | 0.220 sec | 0 thr |  0.001 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/info:
   exclude                         0.000 MB | 0.246 sec | 0 thr |  0.001 MB/s
   ORIG_HEAD                       0.000 MB | 0.225 sec | 0 thr |  0.000 MB/s
   HEAD                            0.000 MB | 0.529 sec | 0 thr |  0.000 MB/s
   index                           0.001 MB | 0.227 sec | 0 thr |  0.006 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/.git/hooks:
   pre-commit.sample               0.002 MB | 0.232 sec | 0 thr |  0.007 MB/s
   post-update.sample              0.000 MB | 0.542 sec | 0 thr |  0.000 MB/s
   push-to-checkout.sample         0.003 MB | 0.257 sec | 0 thr |  0.010 MB/s
   applypatch-msg.sample           0.000 MB | 0.233 sec | 0 thr |  0.002 MB/s
   pre-rebase.sample               0.005 MB | 0.246 sec | 0 thr |  0.019 MB/s
   pre-receive.sample              0.001 MB | 0.248 sec | 0 thr |  0.002 MB/s
   pre-push.sample                 0.001 MB | 0.486 sec | 0 thr |  0.003 MB/s
   prepare-commit-msg.sample       0.001 MB | 0.261 sec | 0 thr |  0.005 MB/s
   update.sample                   0.003 MB | 0.257 sec | 0 thr |  0.014 MB/s
   pre-applypatch.sample           0.000 MB | 0.260 sec | 0 thr |  0.002 MB/s
   pre-merge-commit.sample         0.000 MB | 0.243 sec | 0 thr |  0.002 MB/s
   fsmonitor-watchman.sample       0.004 MB | 0.238 sec | 0 thr |  0.019 MB/s
   commit-msg.sample               0.001 MB | 0.231 sec | 0 thr |  0.004 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/utils:
   __init__.py                     0.000 MB | 0.243 sec | 0 thr |  0.000 MB/s
   utils.py                        0.012 MB | 0.217 sec | 0 thr |  0.055 MB/s
   DummyIrodsConnector.py          0.007 MB | 0.242 sec | 0 thr |  0.029 MB/s
C- /rug/home/rdms-testers@rug.nl/nmr_meta_parser/utils/__pycache__:
   utils.cpython-310.pyc           0.013 MB | 0.539 sec | 0 thr |  0.024 MB/s
   DummyIrodsConnector.cpyth       0.009 MB | 0.255 sec | 0 thr |  0.034 MB/s
   IrodsConnector.cpython-31       0.040 MB | 0.292 sec | 0 thr |  0.137 MB/s
   __init__.cpython-310.pyc        0.000 MB | 0.251 sec | 0 thr |  0.001 MB/s
   IrodsConnector.py               0.051 MB | 0.261 sec | 0 thr |  0.195 MB/s
   nmr_meta_cl_parser.py           0.003 MB | 0.232 sec | 0 thr |  0.014 MB/s

(Un)bundling
The ibun command allows to bundle directories or extract a bundled directories within the RDMS. Without specifying additional flags, tar bundles will be created when bundling a directory, but it is also possible to specify other types (available are: tar, gzip, bzip2 and zip).

The additional -c flag is used to indicate bundling. Additionally, you have to also specify which RDMS directory you want to bundle as well as a name for the created bundle

Click to display ⇲

Click to hide ⇱

# Bundle directory to a tar archive
$ ibun -c bundled_folder.tar /rug/home/rdms-testers@rug.nl/some_other_folder

# Bundle directory to a gzip archive
$ ibun -cDgzip bundled_folder.gz /rug/home/rdms-testers@rug.nl/some_other_folder

# Bundle directory to a bzip2 archive
$ ibun -cDbzip2 bundled_folder.bz2 /rug/home/rdms-testers@rug.nl/some_other_folder

# Bundle directory to a zip archive
$ ibun -cDzip bundled_folder.zip /rug/home/rdms-testers@rug.nl/some_other_folder

The additional -x flag is used to indicate extraction. Additionally, you have to also specify which bundle you want to extract as well as specify a RDMS directory to which the content of the bundle is extracted.

Click to display ⇲

Click to hide ⇱

# Extract tar archive
$ ibun -x bundled_folder.tar unbundled_tar

# Extract gzip archive
$ ibun -x bundled_folder.gz unbundled_gzip

# Extract bzip2 archive
$ ibun -x bundled_folder.bz2 unbundled_bzip2

# Extract zip archive
$ ibun -x bundled_folder.zip unbundled_zip

Notes:

The bundling and extraction operations do not display a progress. For large folders/archives, it might take a longer time to finish while your terminal seems to be stuck.
If you want to overwrite, you can again use the additional -f flag.

Assigning Data Permission
For the assignment of access permissions, the ichmod command is used. Currently, it is possible to assign the following four permission levels: 'Null', 'Read', 'Write', 'Own'.

The general syntax to assign permission is ichmod [-r] <access_level> <user-/groupname> <path> where <access_level> is one of the mentioned four permission levels, <user/-groupname> the name of the user or group that should get the new permissions, and <path> is the RDMS path for the files/folders that you want to assign the new permissions.

If you plan to assign the permission recursively on a folders, meaning also to assign the new permissions for all its subfolders and contained files, you can run the command with the additional -r flag.

It is also possible to enable/disable permission inheritance of a folder using the ichmod command.

Please refer to the Wiki section about permissions and inheritance to get more information about the different permission levels as well as the meaning of inheritance.

Click to display ⇲

Click to hide ⇱

# Assign new permissions (here: read) for a user
$ ichmod write rdms-testers@rug.nl /rug/home/rdms-testers@rug.nl/example_file

# Assign new permissions (here: own) for a group  on a folder and it's content (see the added '-r' flag!)
$ ichmod own rdms-testers@rug.nl /rug/home/rdms-testers@rug.nl/some_example_folder

# Revoking permissions = Assign 'null' permission
$ ichmod null rdms-testers@rug.nl /rug/home/rdms-testers@rug.nl/example_file

# Activate Inheritance on a folder
$ ichmod inherit /rug/home/rdms-testers@rug.nl/some_example_folder

# Deactivate Inheritance on a folder
$ ichmod noninherit /rug/home/rdms-testers@rug.nl/some_example_folder

Notes: If you are unsure about the currently set permissions on a file/folder or you are unsure if inheritance is off/on, run the ils -A <path> to display these information.

Click to display ⇲

Click to hide ⇱

$ ils -A /rug/home/rdms-testers@rug.nl/some_example_folder
/rug/home/rdms-testers@rug.nl/some_example_folder:
        ACL - rdms-testers@rug.nl#rug:own   
        Inheritance - Disabled
  test.txt
        ACL - rdms-testers@rug.nl#rug:own   

# In this example, we can see that the folder in question has inheritance disabled, and just one user has 'own' permission assigned to the main folder as well as the one document contained in it.

Working with Metadata
The iCommands can be also used to add metadata to a file/folder, list already existing metadata, modify existing metadata, and also query (search) for objects that contain certain metadata.

Within the RDMS, metadata is always added as so called 'attribute-value-units' triples, AVUs. The 'attribute' specifies the metadata key (e.g. 'Author') while the 'value' specifies the corresponding value for this key (e.g. 'John Doe'). The 'unit' entry is optional, but can be useful in cases where the entries could be otherwise ambiguous (e.g. A: 'Distance:, V: '100', U: 'km'). This can be left blank if no unit is desired.

The command that is used to work with metadata from the CLI, is the imeta command, which has additional subcommands depending on the desired operation :

List current metadata: imeta [-d, -C] <path>
Add new metadata: imeta add [-d, -C] <path>
Modify metadata: imeta mod [-d, -C] <path> <old_attribute> <old_value> <old_unit> n:<new_attribute> v:<new_value> u:<new_unit>
Query for metadata: imeta qu <attribute> <Operator> <value>

Click to display ⇲

Click to hide ⇱

# List current metadata on a folder
$ imeta ls -C /rug/home/rdms-testers@rug.nl/some_example_folder
AVUs defined for collection /rug/home/rdms-testers@rug.nl/some_example_folder:
None

# Add new metadata (AVU) to a folder
$ imeta add -C /rug/home/rdms-testers@rug.nl/some_example_folder Author Wiki_User

# Modify metadata of a folder
$ imeta mod -C /rug/home/rdms-testers@rug.nl/some_example_folder Author Wiki_User n:Author v:Example_User

# Query (search) for metadata
$ imeta qu -C Author = Example_User
collection: /rug/home/rdms-testers@rug.nl/some_example_folder

Notes:

The imeta command requires that you also specify the additional -C or -d flag.
Use -C if you want to work with folder (collection). For example imeta ls -C <foldername>.
Use -d if you want to work with a file (data object). For example imeta ls -d <filename>.
There are additional help pages available for the different imeta subcommands which can be displayed by executing imeta help <subcommand>. For example: imeta help qu to display more information of the query (search) command.

Search for Data
The iCommands provide different options to search for files/folders:

If you know the name of the file/folder, you can simply use the ilocate command which will display you the full RDMS path if files/folders by this name were found in the system.
If you want to do simple search by using metadata, you can use the imeta [-C, -d] qu query. With this, you can also combine multiple metadata entries (see example below) for a more granular query/search.
The iquest command can be used for even more powerful queries/searches within the database. This is a very powerful search that can be (for example) also used to count the amount of files in a certain RDMS location. It uses SQL-like terminology for the queries and also allows to use wildcards for the queries. The help for this command (execute iquest -h contains a more detailed description about the syntax as well as some examples.

Click to display ⇲

Click to hide ⇱

# Locate files/folders by name
$ ilocate example_file
/rug/home/rdms-testers@rug.nl/example_file
/rug/home/rdms-testers@rug.nl/some_other_folder/example_file

# Query for one or multiple metadata entries
$  imeta qu -d Author = Wiki_User
collection: /rug/home/rdms-testers@rug.nl
dataObj: example_file
----
collection: /rug/home/rdms-testers@rug.nl
dataObj: example_file2

$ imeta qu -d Author = Wiki_User and Purpose = Example
collection: /rug/home/rdms-testers@rug.nl
dataObj: example_file

$ imeta qu -d Author = Wiki_User and Purpose = Something_Else
collection: /rug/home/rdms-testers@rug.nl
dataObj: example_file2

# Use iquest to get the total size of the user's home folder as well as all contained folders
$ iquest "SELECT SUM(DATA_SIZE) WHERE COLL_NAME LIKE '/rug/home/rdms-testers@rug.nl%'"
DATA_SIZE = 4001838012

iCommands for Group Management

To manage RDMS Groups, the igroupadmin command is used. This command can be used to list group members (igroupadmin lg <groupname>), add/remove users from groups (igroupadmin <atg/rfg> <groupname> <emailadress>), or even to create a new group (igroupadmin mkgroup <groupname>).

Click to display ⇲

Click to hide ⇱

# Create a new group
$ igroupadmin mkgroup an_example_group

# List members of group
$ igroupadmin lg an_example_group
Members of group an_example_group:
No rows found

# Add member to group
$ igroupadmin atg an_example_group rdms-testers@rug.nl
$ igroupadmin lg an_example_group
Members of group an_example_group:
rdms-testers@rug.nl#testrug

# Remove member from group
$ igroupadmin rfg an_example_group rdms-testers@rug.nl
Members of group an_example_group:
No rows found

iCommands Cheat Sheet

The following table contains an overview of the most useful iCommands with a short description.

iCommand	Meaning
`ihelp <command>`	Print help of `<command>` or displays all available `iCommands` if no command is specified
`iinit`	Initialize connection to the RDMS and also save the password in scrambled format for use with subsequent `iCommands`
`iexit`	Close connection to the RDMS and also remove the scrambled password file
`ipwd`	Displays your current RDMS working directory
`icd`	Change the current working directory
`ils [-A, -L, -r]`	Listing files and folders `-A`: Also display access and inheritance information `-L`: Displays more information (long format) `-r`: List recursively (also display content of sub folders)
`itree [-c, -f, -s]`	Display folder structure as a tree view `-c`: Just display folders (collections), no files `-f`: Display full path of listed items `-s`: Also display file sizes in the tree view
`imkdir [-p]`	Create a new folder in the RDMS `-p`: Also create parent folders if they do not exist
`irm [-r, -f]`	Move files/folders to trash `-r`: Recursive (delete whole folder structures) `-f`: Do not move to trash, but delete permanently
`irmtrash`	Empty your trash
`imv`	Move files/folders within the RDMS
`icp [-K, -r, -f]`	Copy files/folders within the RDMS `-K`: Calculate and verify checksum during copying `-r`: Recursive (copy whole folder structures) `-f`: Force, overwrites if files/folders already exist
`ilocate`	Locate files in the RDMS
`ichmod [-r]`	Change access permission and enable/disable permission inheritance `-r`: Recursive (change for whole folder structures)
`imeta add [-d, -C]`	Add metadata `-d`: Used for files (data objects) `-C`: Used for folders (collections)
`imeta ls [-d, -C]`	List metadata `-d`: Used for files (data objects) `-C`: Used for folders (collections)
`iget [-K, -r, -P, -T, -X]`	Download data from the RDMS `-K`: Calculate and verify checksum `-r`: Download whole folder structure (recursive) `-P`: Display progress during transfer `-T`: Renew socket connection (long transfers) `-X`: Specify restart file (long transfers)
`iput [-K, -r, -P, -T, -X]`	Upload data to the RDMS `-K`: Calculate and verify checksum `-r`: Upload whole folder structure (recursive) `-P`: Display progress during transfer `-T`: Renew socket connection (long transfers) `-X`: Specify restart file (long transfers)
`irsync [-K]`	Synchronize data between RDMS and local
`ibun [-x, -c, -D]`	(Un)bundle data in the RDMS
`iuserinfo <user>`	Display information (name, groups, etc) of a user, displays information about yourself if no user is specified
`iquest`	Execute complex queries using metadata
`igroupadmin`	Manage RDMS Groups (just for group admins)

Further Tips and Tricks

This section contains some additional tips and tricks that make working with iCommands more effective.

Enable Bash Auto-Completion

Most Linux distribution come by default with the so called Bourne Again Shell (Bash) which has an internal auto-completion function via “tab key”. This can be used to, for example, auto-complete certain paths when using the standard Linux commands (e.g., ls, cd).

This auto-completion does not work by default for the iCommands which means that for example using ils with the “tab key” to auto-complete an RDMS path will not work. Luckily, the iRODS community provides a helpful irods_completion.bash shell script via its iRODS contrib Github page that can be used to enable this functionality.

Temporarily enable Bash Auto-Completion

As a first step, you need to download the irods_completion.bash shell script from the mentioned Github repository. The Linux commands cURL or wget can be used for this directly from the terminal.

In this example, we will assume that the auto-completion script will be downloaded and saved in your Linux home directory (abbreviated as ~).

To download the shell script to your home location, open a terminal window and then execute:

# Using wget
$ wget https://raw.githubusercontent.com/irods/contrib/main/irods_completion.bash  -P ~/

# Using cURL
$ curl  https://raw.githubusercontent.com/irods/contrib/main/irods_completion.bash > ~/irods_completion.bash

After successfully downloading the script, you can enable it temporarily (for your current shell session) by executing:

$ source ~/irods_completion.bash

Afterwards, the “tab key” can be used for auto-completion on the RDMS side. If you hit the “tab key” two times in a row, your terminal will display all available paths in the current location that fits your auto-completion criteria.

Permanently enable Bash Auto-Completion

It is also possible to automatically “source” the auto_completion.bash script every time a new Bash shell is started.

To do so, you have to first download the shell script as described above. Afterwards, we will edit the ~/.bashrc file which is automatically loaded when a Bash session is started so that it automatically loads/sources the auto-completion script.

While you can use your favorite text editor for this, we will use nano in this example. In your Linux terminal, execute:

$ nano ~/.bashrc

Now add source ~/irods_completion.bash to the end of the opened file, and then save (nano: Ctrl + o) the file and exit your text editor (nano: Ctrl + x).

From now on the auto-completion script will be loaded by default when you start a new shell session. If you also want to enable the auto-completion for your current session, you have to once execute:

$ source ~/.bashrc

Notes

The auto-completion script that is described in this section is specific for the Bash shell. If you are unsure if you are using the correct shell, you can execute echo $SHELL in your terminal which will show you the default shell in your system.
If you want to disable the automated loading of the auto-completion script, you can edit the ~/.bashrc file again. Either delete the part that contains source ~/irods_completion.bash or comment it out by adding a “#” in front of the line.

Table of Contents

iCommands

Configure iCommands

Installing iCommands

Installation on Ubuntu and Debian

Installation on EL-based Distributions

Using iCommands

Getting Help for specific iCommands

Common iCommands Usage Examples

iCommands for Navigation

iCommands for (Meta)data Management

iCommands for Group Management

iCommands Cheat Sheet

Further Tips and Tricks

Enable Bash Auto-Completion