Differences

This shows you the differences between two versions of the page.

--- rdms:data:filesize [2026/08/28 07:53] – [Large Number of Files] removed reference to unbundling giulio
+++ rdms:data:filesize [2026/08/31 07:34] (current) – burcu
@@ Line 2: / Line 2: @@
 ====== Data Transfer and File Size ======
-Depending of the data size and amount of files, we advise different possibilities for files transfer in/out of the RDMS.
+Depending on the data size and number of files, we advise different possibilities for file transfer in/out of the RDMS.
 ==== Data (Sets) under 50 MB ====
@@ Line 18: / Line 18: @@
 === Upload Files in the Web Interface ===
-To upload a file directly inside the web interface, you first navigate to the destination folder. Afterwards, select the {{:rdms:data:three_points.png?direct&15|}} and select ''Upload File'' from the  menu. You can upload maximum 10 files at once, as long as the total size of the selected files does not exceed 50 MB.
+To upload a file directly inside the web interface, you first navigate to the destination folder. Afterwards, select the {{:rdms:data:three_points.png?direct&15|}} and select ''Upload File'' from the  menu. You can upload a maximum of 10 files at once, as long as the total size of the selected files does not exceed 50 MB.
 {{ :rdms:data:upload_interface.png?direct&600 |}}
@@ Line 24: / Line 24: @@
 ==== Data (Sets) over 50 MB ====
-The data transfer of larger files is not possible via the RDMS web interface. There are various clients and command-line tools such as ''iCommands'' to transfer data to RDMS, add metada or perform more advanced tasks.
+The data transfer of larger files is not possible via the RDMS web interface. There are various clients and command-line tools such as ''iCommands'' to transfer data to RDMS, add metadata, or perform more advanced tasks.
 === iCommands ===
-''iCommands'' is an Unix utility that gives users a command-line interface. Currently it is the most efficient way of transferring data, creating metadata and perform more advanced tasks. In order to use the ''iCommands'' you need to work in a command-line tool such as a console, shell or a terminal. For further information please check the [[https://wiki.hpc.rug.nl/rdms/access/linux/createprofile|iCommands]] section of this manual.
+''iCommands'' is a Unix utility that gives users a command-line interface. Currently, it is the most efficient way of transferring data, creating metadata, and performing more advanced tasks. In order to use the ''iCommands'' you need to work in a command-line tool such as a console, shell, or a terminal. For further information, please check the [[https://wiki.hpc.rug.nl/rdms/access/linux/createprofile|iCommands]] section of this manual.
 === WebDAV ===
-WebDAV is a common file transfer protocol. For Windows, MacOS and Linux there are multiple WebDAV clients you can use.
+WebDAV is a common file transfer protocol. For Windows, MacOS and Linux, there are multiple WebDAV clients you can use.
 We currently recommend and provide support for:
@@ Line 42: / Line 42: @@
 ==== Large Data Sets ====
-For large data (sets) in the order of several hundreds of GB or higher, the iRODS-native ''iCommands'' are the best solution.
+For large data (sets) in the order of several hundred GB or higher, the iRODS-native ''iCommands'' are the best solution.
 Here are three different ways to use them:
-  * Via Hábrók cluster: The Hábrók HPC cluster has a [[https://wiki.hpc.rug.nl/habrok/introduction/cluster_description#network|fast network connection]] to the RDMS and the ''iCommands'' are already pre-installed on the cluster. It is therefore also possible, to first transfer the data to Hábrók via [[https://wiki.hpc.rug.nl/habrok/data_management/transferring_data|SSH file transfer]]. Of course you do need an [[https://wiki.hpc.rug.nl/peregrine/introduction/policies#getting_access|account for Hábrók]], but then you can upload to [[https://wiki.hpc.rug.nl/habrok/data_management/storage_areas?s[]=scratch|one of the storage locations]] that are available on the cluster. For the intermediate storage on the cluster for transfer to the RDMS, we recommend that you use the ''/scratch'' directory on Hábrók.
+  * Via Hábrók cluster: The Hábrók HPC cluster has a [[https://wiki.hpc.rug.nl/habrok/introduction/cluster_description#network|fast network connection]] to the RDMS and the ''iCommands'' are already pre-installed on the cluster. It is therefore also possible to first transfer the data to Hábrók via [[https://wiki.hpc.rug.nl/habrok/data_management/transferring_data|SSH file transfer]]. Of course, you do need an [[https://wiki.hpc.rug.nl/peregrine/introduction/policies#getting_access|account for Hábrók]], but then you can upload to [[https://wiki.hpc.rug.nl/habrok/data_management/storage_areas?s[]=scratch|one of the storage locations]] that are available on the cluster. For intermediate storage on the cluster for transfer to the RDMS, we recommend that you use the ''/scratch'' directory on Hábrók.
-  * Via a Linux OS: See the manual on installing and using [[rdms:access:linux|iCommands on Linux]].\\ If you do not have a Linux OS available, you can also use the [[https://iris.service.rug.nl/tas/public/ssp/content/detail/service?unid=8b36d7efb51048fa82ce480ff0206177|Linux workplaces (LWP)]] provided by the university. Within the LWP, the `iCommands` are pre-installed. There also exists a [[https://vlwp.rug.nl/nxwebplayer|virtual LWP version]] that can be access via the browser. For more details on the usage of the LWP, you can also refer to the [[https://lwpwiki.webhosting.rug.nl/index.php/Main_Page|LWP wiki]].
+  * Via a Linux OS: See the manual on installing and using [[rdms:access:linux|iCommands on Linux]].\\ If you do not have a Linux OS available, you can also use the [[https://iris.service.rug.nl/tas/public/ssp/content/detail/service?unid=8b36d7efb51048fa82ce480ff0206177|Linux workplaces (LWP)]] provided by the university. Within the LWP, the `iCommands` are pre-installed. There also exists a [[https://vlwp.rug.nl/nxwebplayer|virtual LWP version]] that can be accessed via the browser. For more details on the usage of the LWP, you can also refer to the [[https://lwpwiki.webhosting.rug.nl/index.php/Main_Page|LWP wiki]].
-  * Under Windows: Microsoft offers the possibility to install the 'Windows Subsystem for Linux', or short [[https://docs.microsoft.com/en-us/windows/wsl/wsl2-about|WSL2]].  This is a virtualization of the Linux kernel in the Windows OS. After successful [[https://docs.microsoft.com/en-us/windows/wsl/wsl2-install|installation]] of WSL2, you can [[rdms:access:linux:icommands|install iCommands]] within the virtual Linux environment and use it for data transfer.\\ **Note:** On university-managed Windows installation, so called 'UWP', the WSL cannot be installed and used.
+  * Under Windows: Microsoft offers the possibility to install the 'Windows Subsystem for Linux', or, in short, [[https://docs.microsoft.com/en-us/windows/wsl/wsl2-about|WSL2]].  This is a virtualization of the Linux kernel in the Windows OS. After successful [[https://docs.microsoft.com/en-us/windows/wsl/wsl2-install|installation]] of WSL2, you can [[rdms:access:linux:icommands|install iCommands]] within the virtual Linux environment and use it for data transfer.\\ **Note:** On university-managed Windows installations, so-called 'UWP', WSL cannot be installed and used.
 ==== Large Number of Files ====
-Independent of the total size of the data, it is recommended that data sets that contain large number of single files (thousands of files and more), are uploaded as an archive, for example as a zip or a tar archives. Uploading the files while in a zip/tar archive will tremendously reduce the operation time.
+Independent of the total size of the data, it is recommended that data sets that contain a large number of single files (thousands of files and more) are uploaded as an archive, for example as a zip or a tar archive. Uploading the files while in a zip/tar archive will tremendously reduce the operation time.