| Both sides previous revision Previous revision Next revision | Previous revision |
| rdms:access:linux:icommands [2024/05/06 09:50] – Added text about apt key add deprecation jelte | rdms:access:linux:icommands [2026/05/21 12:58] (current) – burcu |
|---|
| ====== iCommands ====== | ====== iCommands ====== |
| |
| The complete functionality to the RDMS comes with ''iCommands'' on the command-line interface (CLI). | The complete functionality of the RDMS comes with ''iCommands'' on the command-line interface (CLI). |
| |
| * ''iCommands'' is the official CLI client by iRODS, the open-source software upon which the RDMS is built. | * ''iCommands'' is the official CLI client by iRODS, the open-source software upon which the RDMS is built. |
| * ''iCommands'' consist of a whole set of different commands that allow to interact with the RDMS from the CLI. Next to [[https://docs.irods.org/4.3.0/icommands/user/|commands relevant to general users]] (navigation, data transfer, etc.), it has [[https://docs.irods.org/4.3.0/icommands/metadata/|commands specifically for metadata]], and [[https://docs.irods.org/4.3.0/icommands/groupadmin/|commands for management of groups]]. | * ''iCommands'' consist of a whole set of different commands that allow interaction with the RDMS from the CLI. Next to [[https://docs.irods.org/4.3.0/icommands/user/|commands relevant to general users]] (navigation, data transfer, etc.), it has [[https://docs.irods.org/4.3.0/icommands/metadata/|commands specifically for metadata]], and [[https://docs.irods.org/4.3.0/icommands/groupadmin/|commands for management of groups]]. |
| * ''iCommands'' are already installed on the LWP and Habrók HPC cluster. In these environments, you just need to: | * ''iCommands'' are already installed on the LWP and Habrók HPC cluster. In these environments, you just need to: |
| - Create a ''~\.irods'' directory in your home directory. | - Create a ''~\.irods'' directory in your home directory. |
| - Create an ''irods_environment.json'' file in that directory. | - Create an ''irods_environment.json'' file in that directory. |
| * For other Linux distributions, explanation to install ''iCommands'' are also explained in this section. | * For other Linux distributions, instructions to install ''iCommands'' are also explained in this section. |
| |
| ===== Configure iCommands ===== | ===== Configure iCommands ===== |
| |
| For ''iCommands'' to work properly, a configuration file is needed. If the file does not exists on your system, the configuration file can be setup in the following way: | For ''iCommands'' to work properly, a configuration file is needed. If the file does not exist on your system, the configuration file can be set up in the following way: |
| |
| | |
| 2. Create the configuration file: | 2. Create the configuration file: |
| |
| Create the file ''~/.irods/irods_environment.json'' using your preferred text editor. The file must contain plain ASCII characters. Add the following lines to the file, modifying as instructed below: | Create the file ''~/.irods/irods_environment.json'' using your preferred text editor. The file must contain plain ASCII characters. [[https://wiki.hpc.rug.nl/rdms/solution/multizone|The Default and Sensitive data (SD) zones]] have different configurations. Add the following lines to the file, modifying as instructed below: |
| |
| <hidden Click to display configuration file><code> | <hidden Click to display configuration file for Default zone><code> |
| { | { |
| "irods_authentication_scheme": "PAM", | "irods_authentication_scheme": "pam_password", |
| "irods_client_server_negotiation": "request_server_negotiation", | "irods_client_server_negotiation": "request_server_negotiation", |
| "irods_client_server_policy": "CS_NEG_REQUIRE", | "irods_client_server_policy": "CS_NEG_REQUIRE", |
| "irods_user_name": "<emailadress>", | "irods_user_name": "<emailadress>", |
| "irods_zone_name": "rug" | "irods_zone_name": "rug" |
| | } |
| | </code></hidden>\\ |
| | |
| | <hidden Click to display configuration file for Sensitive data zone><code> |
| | { |
| | "irods_authentication_scheme": "pam_password", |
| | "irods_client_server_negotiation": "request_server_negotiation", |
| | "irods_client_server_policy": "CS_NEG_REQUIRE", |
| | "irods_default_resource": "rootResc", |
| | "irods_encryption_algorithm": "AES-256-CBC", |
| | "irods_encryption_key_size": 32, |
| | "irods_encryption_num_hash_rounds": 16, |
| | "irods_encryption_salt_size": 8, |
| | "irods_host": "store-sd.data.rug.nl", |
| | "irods_port": 1247, |
| | "irods_ssl_verify_server": "cert", |
| | "irods_user_name": "<emailadress>", |
| | "irods_zone_name": "sdrugZone" |
| } | } |
| </code></hidden>\\ | </code></hidden>\\ |
| **Notes:** | **Notes:** |
| * The iRODS username ''<emailadress>'' should be replaced by your UG email address. | * The iRODS username ''<emailadress>'' should be replaced by your UG email address. |
| * Your username is case-sensitive, make sure to use all lowercase letters! | * Your username is case-sensitive; make sure to use all lowercase letters! |
| |
| To see if everything was properly configures, you can now start the session by invoking the ''iinit'' command which will initialize your connection to the RDMS: | To see if everything was properly configured, you can now start the session by invoking the ''iinit'' command, which will initialize your connection to the RDMS: |
| |
| <code> | <code> |
| If you do not see an error after specifying your password, you are set to start using ''iCommands''. See the section below for more information on using ''iCommands'' including some examples. | If you do not see an error after specifying your password, you are set to start using ''iCommands''. See the section below for more information on using ''iCommands'' including some examples. |
| |
| **Note:**\\ The ''iinit'' command will not just initialize your connection to the RDMS, but will also store your password in an obfuscated format under ''~/.irods/.irodsA''. Therefore, you do not have to re-type your password each time a subsequent command is executed during the same session. | **Note:**\\ The ''iinit'' command will not just initialize your connection to the RDMS, but will also store your password in an obfuscated format under ''~/.irods/.irodsA''. Therefore, you do not have to retype your password each time a subsequent command is executed during the same session. |
| |
| ===== Installing iCommands ===== | ===== Installing iCommands ===== |
| |
| Packages for iCommands are currently provided for **CentOS 7**, **EL 8/9 (e.g., Rocky Linux or AlmaLinux)**, **Ubuntu 18/20/22** and **Debian 11/12** (https://irods.org/download/). iCommands can be installed on other Linux distributions too, but special configurations will be needed. For such cases, we recommend the use of a virtual machine that runs one of the officially supported Linux distributions. | Packages for iCommands are currently provided for **CentOS 7**, **EL 8/9 (e.g., Rocky Linux or AlmaLinux)**, **Ubuntu 18/20/22** and **Debian 11/12** (https://irods.org/download/). iCommands can be installed on other Linux distributions, too, but special configurations will be needed. For such cases, we recommend the use of a virtual machine that runs one of the officially supported Linux distributions. |
| |
| |
| **Additional Notes for Users of Ubuntu 20+ and Debian 11+** | **Additional Notes for Users of Ubuntu 20+ and Debian 11+** |
| |
| On Debian-based operating systems, the `apt-key add` feature was deprecated, and you might see the following message in the first step: | On Debian-based operating systems, the ''apt-key add'' feature was deprecated, and you might see the following message in the first step: |
| <code> | <code> |
| $ wget -qO - https://packages.irods.org/irods-signing-key.asc | sudo apt-key add - | $ wget -qO - https://packages.irods.org/irods-signing-key.asc | sudo apt-key add - |
| </code> | </code> |
| |
| As of now, the warning can be still ignored and the steps can be performed as described, but in the longer term it is highly advisable to add the repository signing key in another manner as described, for example, [[https://www.linuxuprising.com/2021/01/apt-key-is-deprecated-how-to-add.html|here]]. As of now, iRODS has not yet decided on the best way to handle this in the case of the iRODS package repository. this information will be updated accordingly. | As of now, the warning can still be ignored, and the steps can be performed as described, but in the longer term, it is highly advisable to add the repository signing key in another manner as described, for example, [[https://www.linuxuprising.com/2021/01/apt-key-is-deprecated-how-to-add.html|here]]. As of now, iRODS has not yet decided on the best way to handle this in the case of the iRODS package repository, and this information will be updated accordingly in the future. |
| |
| ==== Installation on EL-based Distributions ==== | ==== Installation on EL-based Distributions ==== |
| Before you can install iCommands on a RedHat-based Linux distributions (CentOS, Rocky Linux, and AlmaLinux), be sure that the necessery dependencies can be satisfied. | Before you can install iCommands on a Red Hat-based Linux distribution (CentOS, Rocky Linux, and AlmaLinux), be sure that the necessary dependencies can be satisfied. |
| |
| |
| In this section, we will describe the usage of different available ''iCommands'' in more detail. Examples will be given for the different ''iCommands'' showing their possible usage. Please keep in mind that the ''iCommands'' provide extensive functionality and not all will be described in this wiki. | In this section, we will describe the usage of different available ''iCommands'' in more detail. Examples will be given for the different ''iCommands'' showing their possible usage. Please keep in mind that the ''iCommands'' provide extensive functionality and not all will be described in this wiki. |
| |
| Furthermore, different ''iCommands'' can be included in (computing) scripts to automate tasks if needed which will not covered in this section as it is specific for the respective use case. | Furthermore, different ''iCommands'' can be included in (computing) scripts to automate tasks if needed, which will not be covered in this section as it is specific to the respective use case. |
| |
| If you have additional question about functionality of ''iCommands'' in general or specific subcommands, you can contact <rdms-support@rug.nl>. | If you have additional questions about the functionality of ''iCommands'' in general or specific subcommands, you can contact <rdms-support@rug.nl>. |
| |
| Next to the information provided on this wiki, a plethora of external guides on ''iCommands'' and its usage exists. Some important to mention are: | Next to the information provided on this wiki, a plethora of external guides on ''iCommands'' and their usage exist. Some important things to mention are: |
| |
| * [[https://docs.vscentrum.be/en/latest/data/iCommands.html|iCommands Guide by the Vlaams Supercomputing Center]] | * [[https://docs.vscentrum.be/en/latest/data/iCommands.html|iCommands Guide by the Vlaams Supercomputing Center]] |
| |
| ==== Getting Help for specific iCommands ==== | ==== Getting Help for specific iCommands ==== |
| As with linux commands, there is built in help for all ''iCommands'' which can be shown by using the addition ''-h'' flag with the respective command. For example, help for the ''ils'' command is displayed like: | As with Linux commands, there is built-in help for all ''iCommands'' which can be shown by using the addition ''-h'' flag with the respective command. For example, help for the ''ils'' command is displayed like: |
| |
| <code> | <code> |
| big | big |
| foo | foo |
| The only information displayed are the names of data objects in the current | The only information displayed is the names of data objects in the current |
| working collection. | working collection. |
| | |
| alice 4 repl1;child3 283 2021-04-29.17:57 ? foo | alice 4 repl1;child3 283 2021-04-29.17:57 ? foo |
| generic /var/lib/irods/child3vault/public/foo | generic /var/lib/irods/child3vault/public/foo |
| This displays even more replica information and system metadata such as each | This displays even more replica information and system metadata, such as each |
| replica's checksum, data type, and physical location in the storage resource. | replica's checksum, data type, and physical location in the storage resource. |
| | |
| but merely that it may not reflect the "truth" of this | but merely that it may not reflect the "truth" of this |
| data object. | data object. |
| 1. &: good - The replica is good. Usually the latest replica to be | 1. &: good - The replica is good. Usually, the latest replica is |
| written. When a replica is good, all bytes and system | written. When a replica is good, all bytes and system |
| metadata (size and checksum, if present) are understood | metadata (size and checksum, if present) are understood |
| to have been recorded correctly. | to have been recorded correctly. |
| 2. ?: intermediate - The replica is actively being written to. The state of | 2. ?: intermediate - The replica is actively being written to. The state of |
| the replica cannot be determined because the client | The replica cannot be determined because the client |
| writing to it still has the replica opened. Replicas | writing to it still has the replica open. Replicas |
| which are intermediate cannot be opened for read or | which are intermediate cannot be opened for read or |
| write, nor can they be unlinked or renamed. | write, nor can they be unlinked or renamed. |
| 3. ?: write-locked - One of this replica's sibling replicas is actively | 3. ?: write-locked - One of this replica's sibling replicas is actively |
| being written to but is itself at rest. Replicas which | being written to, but is itself at rest. Replicas, which |
| are write-locked cannot be opened for read or write | are write-locked, cannot be opened for read or write |
| nor can they be unlinked or renamed. | , nor can they be unlinked or renamed. |
| These replica statuses correspond with the data_is_dirty column in r_data_main. | These replica statuses correspond with the data_is_dirty column in r_data_main. |
| | |
| -h this help | -h this help |
| --bundle - list the subfiles in the bundle file (usually stored in the | --bundle - list the subfiles in the bundle file (usually stored in the |
| /myZone/bundle collection) created by iphybun command. | /myZone/bundle collection) created by the iphybun command. |
| |
| iRODS Version 4.3.0 ils | iRODS Version 4.3.0 ils |
| For the explanation of commonly used ''iCommands'', we will split the ''iCommands'' into three different sections: | For the explanation of commonly used ''iCommands'', we will split the ''iCommands'' into three different sections: |
| |
| - **iCommands for Navigation** will describe the iCommands that are used to navigate the RDMS from the CLI, but also contain information about starting/ending of the connection. | - **iCommands for Navigation** will describe the ''iCommands'' that are used to navigate the RDMS from the CLI, but also contain information about starting/ending the connection. |
| - **iCommands for (Metda)data Management** will describe how to transfer data in/out of the RDMS, explain how to set permissions for other users, how to work with metadata from the CLI, including a more detailed description on how to query the database for more extensive searches. | - **iCommands for (Meta)data Management** will describe how to transfer data in/out of the RDMS, explain how to set permissions for other users, and how to work with metadata from the CLI, including a more detailed description on how to query the database for more extensive searches. |
| - **iCommands for Group Management** will focus on the respective commands that can be used by 'groupadmins' to manage RDMS groups from the CLI. As this section is not relevant for all users, it will be discussed separately. | - **iCommands for Group Management** will focus on the respective commands that can be used by 'groupadmins' to manage RDMS groups from the CLI. As this section is not relevant for all users, it will be discussed separately. |
| |
| === iCommands for Navigation === | === iCommands for Navigation === |
| |
| We already saw the ''iinit'' command in action which initializes a new RDMS session and also stored the obfuscated password for re-use with following ''iCommands''. | We already saw the ''iinit'' command in action, which initializes a new RDMS session and also stores the obfuscated password for re-use with the following ''iCommands''. |
| |
| To again close a session from the CLI, the ''iexit'' command can be used. | To again close a session from the CLI, the ''iexit'' command can be used. |
| </code> | </code> |
| </hidden>\\ | </hidden>\\ |
| When used without additional flags, ''ils'' will just list files and folder (folders = collections in iRODS terms, marked by 'C-' in front of data name), but will not provide any additional information about file sizes or permission information. If we want to see these information, we can use the additional ''-A'' flag to get information about file permissions and inheritance, and the ''-l'' or ''-L'' flag for (very) long output which will display additional information, for example file sizes. As usual, flags can be combined for a specific command. | When used without additional flags, ''ils'' will just list files and folders (folders = collections in iRODS terms, marked by 'C-' in front of data name), but will not provide any additional information about file sizes or permission information. If we want to see this information, we can use the additional ''-A'' flag to get information about file permissions and inheritance, and the ''-l'' or ''-L'' flag for (very) long output, which will display additional information, for example, file sizes. As usual, flags can be combined for a specific command. |
| |
| <hidden> | <hidden> |
| </code> | </code> |
| </hidden>\\ | </hidden>\\ |
| The ''ils'' command can be also used with the additional ''-r'' to perform a //recursive// operation, in which case it will not just display the output of the current RDMS directory, but instead also the content of its lower-level folders. | The ''ils'' command can also be used with the additional ''-r'' to perform a //recursive// operation, in which case it will not just display the output of the current RDMS directory, but instead also the content of its lower-level folders. |
| <hidden> | <hidden> |
| <code> | <code> |
| </code> | </code> |
| </hidden>\\ | </hidden>\\ |
| Alternatively, the ''itree'' command can be executed to get a similar overview of the folder/file structure at a specific RDMS location. Moreover, ''itree'' also displays a summary of files (data objects) and folder (collections) that were found at the specified location. Running ''itree'' without specifying a path will again display results for the current working directory. | Alternatively, the ''itree'' command can be executed to get a similar overview of the folder/file structure at a specific RDMS location. Moreover, ''itree'' also displays a summary of files (data objects) and folders (collections) that were found at the specified location. Running ''itree'' without specifying a path will again display results for the current working directory. |
| |
| <hidden> | <hidden> |
| </code> | </code> |
| </hidden>\\ | </hidden>\\ |
| If we want to navigate to another location inside of the RDMS, we can use the ''icd'' command. For navigation with ''icd'' the destination path can be either specified relatively to the current working directory or given as an absolute path. Furthermore, executing ''icd'' without any additional path, will always navigate to your RDMS home location. | If we want to navigate to another location inside the RDMS, we can use the ''icd'' command. For navigation with ''icd'' the destination path can be either specified relatively to the current working directory or given as an absolute path. Furthermore, executing ''icd'' without any additional path will always navigate to your RDMS home location. |
| |
| <hidden> | <hidden> |
| /rug/home/rdms-testers@rug.nl/some_other_folder/a_subfolder1 | /rug/home/rdms-testers@rug.nl/some_other_folder/a_subfolder1 |
| |
| # Using icd without path, always navigates to RDMS Home directory | # Using icd without a path always navigates to the RDMS Home directory |
| $ icd | $ icd |
| $ ipwd | $ ipwd |
| </hidden> | </hidden> |
| |
| === iCommands for (Metda)data Management === | === iCommands for (Meta)data Management === |
| **Upload to the RDMS**\\ | **Upload to the RDMS**\\ |
| To transfer data to the the RDMS, the ''iput'' command can be used. It is of the general syntax ''iput [flags] <source> <dest>'' where ''<source>'' points to files/folders on your local environment while ''<dest>'' specifies the destination on the RDMS. The destination entry can be also left empty in which case the data will be transferred to the current RDMS working directory. Specified paths can be relative or absolute. | To transfer data to the RDMS, the ''iput'' command can be used. It is of the general syntax ''iput [flags] <source> <dest>'' where ''<source>'' points to files/folders on your local environment while ''<dest>'' specifies the destination on the RDMS. The destination entry can also be left empty, in which case the data will be transferred to the current RDMS working directory. Specified paths can be relative or absolute. |
| |
| <hidden> | <hidden> |
| **Notes:** | **Notes:** |
| * It is also recommended to use the ''-K'' which will calculate and compare the checksum of the files that are transferred. | * It is also recommended to use the ''-K'' which will calculate and compare the checksum of the files that are transferred. |
| * For long-lasting transfers, it is furthermore recommended to use the ''-T'' flag which will renew the socket connection after 10 minutes which can eliminate possible errors resulting from time-outs due to firewall settings. | * For long-lasting transfers, it is furthermore recommended to use the ''-T'' flag, which will renew the socket connection after 10 minutes, which can eliminate possible errors resulting from time-outs due to firewall settings. |
| * An additional option exists to specify a restart file using the ''-X'' flag. For uploading directories, this can be used to restart the transfer if needed. This can be combined with the ''--retries [N]'' flag to automatically restart the transfer [N] times if needed. | * An additional option exists to specify a restart file using the ''-X'' flag. For uploading directories, this can be used to restart the transfer if needed. This can be combined with the ''--retries [N]'' flag to automatically restart the transfer [N] times if needed. |
| * For improved transfer performance, especially for data sets containing a lot of small, individual files, it is recommend to upload the files in a bundled format (e.g., tar archive) and then bundle within the RDMS if desired (see the section about (un)bundling below). | * For improved transfer performance, especially for data sets containing a lot of small, individual files, it is recommended to upload the files in a bundled format (e.g., tar archive) and then bundle within the RDMS if desired (see the section about (un)bundling below). |
| * If you want to overwrite a file that is already present in the system, use the ''-f'' (force) flag to overwrite, but be cautious when overwriting data. | * If you want to overwrite a file that is already present in the system, use the ''-f'' (force) flag to overwrite, but be cautious when overwriting data. |
| |
| For downloads of data from the RDMS, the ''iget'' command can be used. Its general syntax, ''iget [flags] <source> <dest>'', is identical to ''iput'', except that ''<source>'' is this time the RDMS side while ''<dest>'' specifies the destination on your local file system. | For downloads of data from the RDMS, the ''iget'' command can be used. Its general syntax, ''iget [flags] <source> <dest>'', is identical to ''iput'', except that ''<source>'' is this time the RDMS side while ''<dest>'' specifies the destination on your local file system. |
| |
| Also, the same flags can be used and the above mentioned notes for the usage of ''iput'' also apply for ''iget''. | Also, the same flags can be used, and the above-mentioned notes for the usage of ''iput'' also apply for ''iget''. |
| |
| |
| |
| **Synchronization with the RDMS**\\ | **Synchronization with the RDMS**\\ |
| Another method for the transfer of data is the usage of the ''irsync'' command which synchronizes a source directory with a target directory. The distinction is made by placing a ''i:'' in front of the source/destination paths. | Another method for the transfer of data is the usage of the ''irsync'' command, which synchronizes a source directory with a target directory. The distinction is made by placing a ''i:'' in front of the source/destination paths. |
| |
| <hidden> | <hidden> |
| <code> | <code> |
| # Synchronize the content of local folder to a folder on the RDMS: | # Synchronize the content of the local folder to a folder on the RDMS: |
| $ irsync -r <source> i:<dest> | $ irsync -r <source> i:<dest> |
| |
| Synchronize the content of remote folder on the RDMS with a local folder: | Synchronize the content of the remote folder on the RDMS with a local folder: |
| $ irsync -r i:<source> <dest> | $ irsync -r i:<source> <dest> |
| |
| </hidden>\\ | </hidden>\\ |
| **Notes:** | **Notes:** |
| * The ''irsync'' command also allows to specify multiple sources that can be synced to one destination. | * The ''irsync'' command also allows specifying multiple sources that can be synced to one destination. |
| * To see detailed information about the progress of the synchronization, the additional ''-v'' (verbose) flag can be used. | * To see detailed information about the progress of the synchronization, the additional ''-v'' (verbose) flag can be used. |
| * The ''-K'' flag can be used to calculate and compare checksum for the transferred files on the source and destination side. This is recommended. | * The ''-K'' flag can be used to calculate and compare checksums for the transferred files on the source and destination sides. This is recommended. |
| * The ''irsync'' command does not have a specific force flag to overwrite files. Instead, ''irsync'' works in a way that it detects new/changed files on the specified source and uploads them to the target destination. Already existing files that were not changed are not touch/transferred by ''irsync'' while changed files get overwritten on the destination! | * The ''irsync'' command does not have a specific force flag to overwrite files. Instead, ''irsync'' works in a way that it detects new/changed files on the specified source and uploads them to the target destination. Already existing files that were not changed are not touched/transferred by ''irsync'' while changed files get overwritten on the destination! |
| |
| With this notes in mind, a typical ''irsync'' command with recommend flags would look like: | With this note in mind, a typical ''irsync'' command with recommended flags would look like: |
| |
| <hidden> | <hidden> |
| |
| **(Un)bundling**\\ | **(Un)bundling**\\ |
| The ''ibun'' command allows to bundle directories or extract a bundled directories within the RDMS. Without specifying additional flags, tar bundles will be created when bundling a directory, but it is also possible to specify other types (available are: tar, gzip, bzip2 and zip). | The ''ibun'' command allows bundling directories or extracting bundled directories within the RDMS. Without specifying additional flags, tar bundles will be created when bundling a directory, but it is also possible to specify other types (available are: tar, gzip, bzip2, and zip). |
| |
| |
| The additional ''-c'' flag is used to indicate bundling. Additionally, you have to also specify which RDMS directory you want to bundle as well as a name for the created bundle | The additional ''-c'' flag is used to indicate bundling. Additionally, you have to specify which RDMS directory you want to bundle, as well as a name for the created bundle |
| |
| <hidden> | <hidden> |
| </code> | </code> |
| </hidden>\\ | </hidden>\\ |
| The additional ''-x'' flag is used to indicate extraction. Additionally, you have to also specify which bundle you want to extract as well as specify a RDMS directory to which the content of the bundle is extracted. | The additional ''-x'' flag is used to indicate extraction. Additionally, you have to specify which bundle you want to extract, as well as specify an RDMS directory to which the content of the bundle is extracted. |
| <hidden> | <hidden> |
| <code> | <code> |
| </hidden>\\ | </hidden>\\ |
| **Notes:** | **Notes:** |
| * The bundling and extraction operations do not display a progress. For large folders/archives, it might take a longer time to finish while your terminal seems to be stuck. | * The bundling and extraction operations do not display progress. For large folders/archives, it might take a longer time to finish, while your terminal seems to be stuck. |
| * If you want to overwrite, you can again use the additional ''-f'' flag. | * If you want to overwrite, you can again use the additional ''-f'' flag. |
| | * Notes that extraction via ''ibun -x'' will fail if your archive contains symlinks, as they are not supported by ''ibun'' (also see info in the [[rdms:bestpractices|Best Practices]] section). |
| |
| **Assigning Data Permission**\\ | **Assigning Data Permission**\\ |
| For the assignment of access permissions, the ''ichmod'' command is used. Currently, it is possible to assign the following four permission levels: 'Null', 'Read', 'Write', 'Own'. | For the assignment of access permissions, the ''ichmod'' command is used. Currently, it is possible to assign the following four permission levels: 'Null', 'Read', 'Write', and 'Own'. |
| |
| The general syntax to assign permission is ''ichmod [-r] <access_level> <user-/groupname> <path>'' where ''<access_level>'' is one of the mentioned four permission levels, ''<user/-groupname>'' the name of the user or group that should get the new permissions, and ''<path>'' is the RDMS path for the files/folders that you want to assign the new permissions. | The general syntax to assign permission is ''ichmod [-r] <access_level> <user-/groupname> <path>'' where ''<access_level>'' is one of the mentioned four permission levels, ''<user/-groupname>'' the name of the user or group that should get the new permissions, and ''<path>'' is the RDMS path for the files/folders that you want to assign the new permissions. |
| |
| If you plan to assign the permission recursively on a folders, meaning also to assign the new permissions for all its subfolders and contained files, you can run the command with the additional ''-r'' flag. | If you plan to assign the permission recursively on a folder, meaning also to assign the new permissions for all its subfolders and contained files, you can run the command with the additional ''-r'' flag. |
| |
| It is also possible to enable/disable permission inheritance of a folder using the ''ichmod'' command. | It is also possible to enable/disable permission inheritance of a folder using the ''ichmod'' command. |
| $ ichmod write rdms-testers@rug.nl /rug/home/rdms-testers@rug.nl/example_file | $ ichmod write rdms-testers@rug.nl /rug/home/rdms-testers@rug.nl/example_file |
| |
| # Assign new permissions (here: own) for a group on a folder and it's content (see the added '-r' flag!) | # Assign new permissions (here: own) for a group on a folder, and its content (see the added '-r' flag!) |
| $ ichmod own rdms-testers@rug.nl /rug/home/rdms-testers@rug.nl/some_example_folder | $ ichmod own rdms-testers@rug.nl /rug/home/rdms-testers@rug.nl/some_example_folder |
| |
| </hidden>\\ | </hidden>\\ |
| **Notes:** | **Notes:** |
| If you are unsure about the currently set permissions on a file/folder or you are unsure if inheritance is off/on, run the ''ils -A <path>'' to display these information. | If you are unsure about the currently set permissions on a file/folder or you are unsure if inheritance is off/on, run the ''ils -A <path>'' to display this information. |
| |
| <hidden> | <hidden> |
| </code> | </code> |
| </hidden>\\ | </hidden>\\ |
| | |
| | **How to manage files in the RDMS trash folder**\\ |
| | Files that have been deleted either via the web interface or using the command ''irm'' are not permanently removed from the system, but are stored in the ''trash'' folder of the RDMS. Should you wish to empty your ''trash'' folder, you can simply run the ''irmtrash'' command without any further parameters. |
| | |
| | If you wish to restore files in the ''trash'' folder using ''iCommands'', you can do so by running the following commands: |
| | <hidden> |
| | <code> |
| | #To locate a file in the trash folder, you will need to know the filename: |
| | $ ilocate -t <filename> |
| | |
| | # where the -t switch indicates to search in trash. Copy the path given by ilocate, then run any of the suggested commands. |
| | |
| | # To move the file out of the trash folder (and preserve previously attached metadata): |
| | $ imv <logical path(s) from ilocate> <path to RDMS destination folder> |
| | |
| | # To restore a copy of the file present in the trash folder (without previously attached metadata): |
| | $ icp <logical path(s) from ilocate> <path to RDMS destination folder> |
| | |
| | # To download a copy of the file to a folder on your local system (without previously attached metadata): |
| | $ iget <logical path(s) from ilocate> <path to local destination folder> |
| | |
| | </code> |
| | </hidden>\\ |
| | |
| | You can also use the **RDMS web interface to restore a file** from the ''trash''. If you would prefer to use the web interface for this operation, please refer to the section of the wiki [[https://wiki.hpc.rug.nl/rdms/webapp/databrowser#how_to_restore_deleted_files|How to restore deleted files]]. |
| | |
| **Working with Metadata**\\ | **Working with Metadata**\\ |
| The ''iCommands'' can be also used to add metadata to a file/folder, list already existing metadata, modify existing metadata, and also query (search) for objects that contain certain metadata. | The ''iCommands'' can also be used to add metadata to a file/folder, list already existing metadata, modify existing metadata, and also query (search) for objects that contain certain metadata. |
| |
| Within the RDMS, metadata is always added as so called 'attribute-value-units' triples, AVUs. The 'attribute' specifies the metadata key (e.g. 'Author') while the 'value' specifies the corresponding value for this key (e.g. 'John Doe'). The 'unit' entry is optional, but can be useful in cases where the entries could be otherwise ambiguous (e.g. A: 'Distance:, V: '100', U: 'km'). This can be left blank if no unit is desired. | Within the RDMS, metadata is always added as so-called 'attribute-value-units' triples, AVUs. The 'attribute' specifies the metadata key (e.g., 'Author') while the 'value' specifies the corresponding value for this key (e.g., 'John Doe'). The 'unit' entry is optional, but can be useful in cases where the entries could be otherwise ambiguous (e.g., A: 'Distance:, V: '100', U: 'km'). This can be left blank if no unit is desired. |
| |
| The command that is used to work with metadata from the CLI, is the ''imeta'' command, which has additional subcommands depending on the desired operation : | The command that is used to work with metadata from the CLI is the ''imeta'' command, which has additional subcommands depending on the desired operation : |
| |
| * List current metadata: ''imeta [-d, -C] <path>'' | * List current metadata: ''imeta [-d, -C] <path>'' |
| **Notes:** | **Notes:** |
| * The ''imeta'' command requires that you also specify the additional ''-C'' or ''-d'' flag. | * The ''imeta'' command requires that you also specify the additional ''-C'' or ''-d'' flag. |
| * Use ''-C'' if you want to work with folder (collection). For example ''imeta ls -C <foldername>''. | * Use ''-C'' if you want to work with a folder (collection). For example ''imeta ls -C <foldername>''. |
| * Use ''-d'' if you want to work with a file (data object). For example ''imeta ls -d <filename>''. | * Use ''-d'' if you want to work with a file (data object). For example ''imeta ls -d <filename>''. |
| * There are additional help pages available for the different ''imeta'' subcommands which can be displayed by executing ''imeta help <subcommand>''. For example: ''imeta help qu'' to display more information of the query (search) command. | * There are additional help pages available for the different ''imeta'' subcommands, which can be displayed by executing ''imeta help <subcommand>''. For example: ''imeta help qu'' to display more information about the query (search) command. |
| |
| **Search for Data**\\ | **Search for Data**\\ |
| The iCommands provide different options to search for files/folders: | The iCommands provide different options to search for files/folders: |
| |
| - If you know the name of the file/folder, you can simply use the ''ilocate'' command which will display you the full RDMS path if files/folders by this name were found in the system. | - If you know the name of the file/folder, you can simply use the ''ilocate'' command, which will display you the full RDMS path if files/folders by this name were found in the system. |
| - If you want to do simple search by using metadata, you can use the ''imeta [-C, -d] qu'' query. With this, you can also combine multiple metadata entries (see example below) for a more granular query/search. | - If you want to do a simple search by using metadata, you can use the ''imeta [-C, -d] qu'' query. With this, you can also combine multiple metadata entries (see example below) for a more granular query/search. |
| - The ''iquest'' command can be used for even more powerful queries/searches within the database. This is a very powerful search that can be (for example) also used to count the amount of files in a certain RDMS location. It uses SQL-like terminology for the queries and also allows to use wildcards for the queries. The help for this command (execute ''iquest -h'' contains a more detailed description about the syntax as well as some examples. | - The ''iquest'' command can be used for even more powerful queries/searches within the database. This is a very powerful search that can be (for example) also used to count the number of files in a certain RDMS location. It uses SQL-like terminology for the queries and also allows the use of wildcards for the queries. The help for this command (execute ''iquest -h'' contains a more detailed description about the syntax as well as some examples. |
| |
| <hidden> | <hidden> |
| $ igroupadmin mkgroup an_example_group | $ igroupadmin mkgroup an_example_group |
| |
| # List members of group | # List members of the group |
| $ igroupadmin lg an_example_group | $ igroupadmin lg an_example_group |
| Members of group an_example_group: | Members of group an_example_group: |
| === Enable Bash Auto-Completion === | === Enable Bash Auto-Completion === |
| |
| Most Linux distribution come by default with the so called [[https://en.wikipedia.org/wiki/Bash_(Unix_shell)|Bourne Again Shell (Bash)]] which has an internal auto-completion function via "tab key". This can be used to, for example, auto-complete certain paths when using the standard Linux commands (e.g., ''ls'', ''cd''). | Most Linux distributions come by default with the so-called [[https://en.wikipedia.org/wiki/Bash_(Unix_shell)|Bourne Again Shell (Bash)]], which has an internal auto-completion function via "tab key". This can be used to, for example, auto-complete certain paths when using the standard Linux commands (e.g., ''ls'', ''cd''). |
| |
| This auto-completion does not work by default for the ''iCommands'' which means that for example using ''ils'' with the "tab key" to auto-complete an RDMS path will not work. Luckily, the iRODS community provides a helpful ''irods_completion.bash'' shell script via its [[https://github.com/irods/contrib/tree/main|iRODS contrib Github]] page that can be used to enable this functionality. | This auto-completion does not work by default for the ''iCommands'' which means that, for example, using ''ils'' with the "tab key" to auto-complete an RDMS path will not work. Luckily, the iRODS community provides a helpful ''irods_completion.bash'' shell script via its [[https://github.com/irods/contrib/tree/main|iRODS contrib Github]] page that can be used to enable this functionality. |
| |
| **Temporarily enable Bash Auto-Completion** | **Temporarily enable Bash Auto-Completion** |
| |
| As a first step, you need to download the ''irods_completion.bash'' shell script from the mentioned Github repository. The Linux commands ''cURL'' or ''wget'' can be used for this directly from the terminal. | As a first step, you need to download the ''irods_completion.bash'' shell script from the mentioned GitHub repository. The Linux commands ''cURL'' or ''wget'' can be used for this directly from the terminal. |
| |
| In this example, we will assume that the auto-completion script will be downloaded and saved in your Linux home directory (abbreviated as ''~''). | In this example, we will assume that the auto-completion script will be downloaded and saved in your Linux home directory (abbreviated as ''~''). |
| </code> | </code> |
| |
| Afterwards, the "tab key" can be used for auto-completion on the RDMS side. If you hit the "tab key" two times in a row, your terminal will display all available paths in the current location that fits your auto-completion criteria. | Afterwards, the "tab key" can be used for auto-completion on the RDMS side. If you hit the "tab key" two times in a row, your terminal will display all available paths in the current location that fit your auto-completion criteria. |
| |
| **Permanently enable Bash Auto-Completion** | **Permanently enable Bash Auto-Completion** |
| It is also possible to automatically "source" the ''auto_completion.bash'' script every time a new Bash shell is started. | It is also possible to automatically "source" the ''auto_completion.bash'' script every time a new Bash shell is started. |
| |
| To do so, you have to first download the shell script as described above. Afterwards, we will edit the ''~/.bashrc'' file which is automatically loaded when a Bash session is started so that it automatically loads/sources the auto-completion script. | To do so, you have to first download the shell script as described above. Afterwards, we will edit the ''~/.bashrc'' file, which is automatically loaded when a Bash session is started, so that it automatically loads/sources the auto-completion script. |
| |
| While you can use your favorite text editor for this, we will use ''nano'' in this example. In your Linux terminal, execute: | While you can use your favorite text editor for this, we will use ''nano'' in this example. In your Linux terminal, execute: |
| Now add ''source ~/irods_completion.bash'' to the end of the opened file, and then save (''nano'': Ctrl + o) the file and exit your text editor (''nano'': Ctrl + x). | Now add ''source ~/irods_completion.bash'' to the end of the opened file, and then save (''nano'': Ctrl + o) the file and exit your text editor (''nano'': Ctrl + x). |
| |
| From now on the auto-completion script will be loaded by default when you start a new shell session. If you also want to enable the auto-completion for your current session, you have to once execute: | From now on, the auto-completion script will be loaded by default when you start a new shell session. If you also want to enable the auto-completion for your current session, you have to execute: |
| |
| <code> | <code> |
| |
| **Notes** | **Notes** |
| * The auto-completion script that is described in this section is specific for the Bash shell. If you are unsure if you are using the correct shell, you can execute ''echo $SHELL'' in your terminal which will show you the default shell in your system. | * The auto-completion script that is described in this section is specific to the Bash shell. If you are unsure if you are using the correct shell, you can execute ''echo $SHELL'' in your terminal, which will show you the default shell in your system. |
| * If you want to disable the automated loading of the auto-completion script, you can edit the ''~/.bashrc'' file again. Either delete the part that contains ''source ~/irods_completion.bash'' or comment it out by adding a "#" in front of the line. | * If you want to disable the automated loading of the auto-completion script, you can edit the ''~/.bashrc'' file again. Either delete the part that contains ''source ~/irods_completion.bash'' or comment it out by adding a "#" in front of the line. |