====== Cyberduck ====== Cyberduck is an open-source client for FTP, SFTP, and WebDAV available for macOS and Windows. It is pre-installed on the University Workspace (UWP) PCs and on the [[https://uwp.rug.nl/|virtual UWP]]. If you work on a user-managed workplace or any other device you can [[https://cyberduck.io/download/|download Cyberduck]] from its official website. {{ :rdms:access:cyberducklogo.png?direct&100|}} Cyberduck is a solution to transfer files to and from the RDMS. To manage data, metadata, users, and projects on the RDMS, you need to connect to the [[rdms:webapp|RDMS WebApp]]. Cyberduck can connect to the RDMS both by using a **WebDAV** and a **native iRODS** protocol. **Notes:** * In case you want to transfer folders with many files (of the order of thousand or more), it is recommended to compress/bundle these folders (ZIP/TAR) in order to have more stable transfers, or transfer such folders one-by-one instead of selecting all of them at once. * If you receive a ''Hierarchy error'' message while transferring data to RDMS, first select the ''Continue'' option. Then, if known, delete the file(s) that caused the error in your RDMS environment and restart the transfer. If you cannot delete a file, move it to another location and contact RDMS support. * For tips regarding optimum transfer settings, please also take a look at the [[https://wiki.hpc.rug.nl/rdms/access/windows/cyberduck#transfer_preferences|Cyberduck transfer preferences]] section which includes recommended Cyberduck settings which could resolve your issues. ===== Configuration of Cyberduck using the WebDAV Protocol ===== In order to access the RDMS via Cyberduck with the WebDAV protocol, click **Open Connection**, select the WebDAV (HTTPS) protocol, and configure the login credentials as follows: * **Protocol:** ''WebDAV (HTTPS)'' * **Port:** ''443'' * **Server:** ''webdav.data.rug.nl'' * **Username:** ''@rug.nl'' * **Password:** Your UG password {{ :rdms:access:windows:cyberduck_webdav_new.png?direct&500 |}} **Note:** If you experience issues with this, be sure that you specified your username in lowercase. The login to the RDMS is case sensitive! ===== Configuration of Cyberduck using the native iRODS Protocol ===== If you would like to use the native iRODS protocol for connecting Cyberduck to the RDMS, you can download the iRODS (RUG) Cyberduck profile directly by clicking on the link below or by copying the content to a text editor of your choice and then save as a ''*.cyberduckprofile'' file. Afterwards, double click the downloaded/created ''*.cyberduckprofile'' file to install add it to Cyberduck. **iRODS Cyberduck profile**: ++++ Click to display section | Protocol irods Vendor rug Description RDMS (rug) Default Nickname RDMS rug zone Default Hostname store.data.rug.nl Hostname Configurable Region rug:rootResc Default Port 1247 Port Configurable Username Configurable Default Path /rug/home Path Configurable Authorization PAM ++++ If everything went right, you will see the login screen below. It will connect you to the RDMS "rug" zone using the native iRODS protocol. Simply insert your UG email address (lowercase!) and associated password to connect to the system. It should have also created an editable bookmark for your connection. {{ :rdms:access:windows:cyberduck_rug_connection.png?direct&500 |}} ===== Transferring Data to and from the RDMS via Cyberduck ===== The instructions below give details on how to use the transfer dialogs to upload, download, and synchronize your data. Be aware that drag-and-drop from/to file explorer is also supported with the same results. ==== Upload ==== The main functionality you will wish to use with Cyberduck is transferring your files from your local storage to the RDMS. Follow the instructions below to see how this is possible. ++++ Click to display section | Once you are connected to the RDMS via Cyberduck, navigate through the Cyberduck interface to the desired target collection for uploading data. Your Home Drive is available at ''/rug/home/@rug.nl'', while [[rdms:solution:team|Team Drives]] or your available Projects are located under ''/rug/home/'' and ''/rug/home/Projects/'', respectively. Once you are in the desired collection, click on the upload button in the top bar (red outlined). {{ :rdms:access:windows:cyberduck_step_1.png?direct&600 |}} This will open a new window. Here you can navigate to the file(s) you want to upload and, once selected, you can start the upload by clicking the red outlined button. {{ :rdms:access:windows:cyberduck_step_2.png?direct&600 |}} The upload will now start and the window shown in the picture will appear. {{ :rdms:access:windows:cyberduck_step_3.png?direct&600 |}} If the upload finished successfully, Cyberduck will show the message below. {{ :rdms:access:windows:cyberduck_step_4b.png?direct&600 |}} Finally, if you closed the window above, but would still want to check the history of your recent uploads, you can click on the ''Transfers'' button in the top bar (red outlined). The window will then reopen and show you the recent transfers. {{ :rdms:access:windows:cyberduck_step_5.png?direct&600 |}} ++++ ==== Download ==== If instead you wish to retrieve data previously stored on the RDMS, you can follow the instructions below to initiate your transfer. ++++ Click to display section | To download data from the RDMS using Cyberduck, you need to follow a slightly different procedure compared to the upload of data. For instance, there is no download button at the top of the GUI, but the download functionality becomes available by right-clicking the file or folder you wish to download. So, navigate to what you wish to download using the file explorer of Cyberduck and right-click it. This will display the following menu. {{ :rdms:access:windows:down_1.png?direct&600 |}} Here you can select either of the three red underlined options: ''Download'', ''Download As...'', or ''Download To..''. The first option, ''Download'', will let you download the file in the specified download directory, which you can set yourself by following the relevant step described in the [[https://wiki.hpc.rug.nl/rdms/access/windows/cyberduck#transfer_preferences|Transfer Preferences]] section below. The second option, ''Download As...'', will open up a window that will let you navigate through your local file structure and also rename the file you wish to download. Here you can potentially also change its file extension, though we recommend you do not do so. Finally, ''Download To...'' will open a window that will let you choose where to save the file, but will not let you rename it. Whichever option you choose, starting the download will open up the transfer window, where you can follow the progress of your download. If it succeeded, you will see the green icon displayed below and the confirmation that the download is complete. Should you have any trouble finding the file you downloaded in your local file structure, you can find it or open it by clicking on the download so it gets blue highlighted and then clicking one of the two icons in the red box. {{ :rdms:access:windows:down_2.png?direct&600 |}} ++++ ==== Synchronization ==== Apart from up- or downloading files or entire folders manually in Cyberduck, you can also give the client the command to synchronize one of your local folders to a collection on the RDMS. The advantage of using synchronizing rather than up- or downloading the data, is that the synchronization will not transfer every file but only act on files that have been changed or that don't exist in the target directory. This will reduce the amount of data you will transfer at any one time. ++++ Click to display section | To synchronize your folder, simply right-click the collection you wish to sync and the following menu will be displayed. {{ :rdms:access:windows:sync_1.png?direct&600 |}} Click on ''Synchronize...'' (red underlined) and the next window will open. There you can navigate to the folder on your local machine that you wish to sync to the RDMS collection. Once you have selected it and confirmed, the transfer window will open and after a few seconds, this window will appear. {{ :rdms:access:windows:sync_2.png?direct&600 |}} Among other things, the window will show the collection you wish to sync once again. Please make sure that it is indeed the collection you want to sync by checking the red-underlined entry. You can also check under ''Details'' that the remote and the local files correspond to the collection and folder you selected. **N.B.**: Cyberduck talks of "files" here, but this obviously also applies to folders. {{ :rdms:access:windows:sync_3.png?direct&600 |}} Finally, if you click the red box shown above, you will be given three options of how Cyberduck is going to handle your sync. Your options are: * **Download**: Changes to the RDMS collection will be reflected in your local folder. Changes to your local folder will not be reflected in the RDMS collection. * **Upload**: Changes to your local folder will be reflected in the RDMS collection. Changes to the RDMS collection will not be reflected in your local folder. * **Mirror**: Any difference between the folders will be transferred, no matter if it was changed in your local folder or in the RDMS. We suggest that you either use "Upload" or "Download" when using the Sync option, as you have a better handle of what is going to be changed. Please also keep in mind that Cyberduck's synchronization feature is a one-time action. If you wish to sync your folder again, it will not happen automatically, but you will need to repeat the steps above. ++++ ==== Transfer Preferences ==== Cyberduck lets you edit the way your transfers happen through choosing certain preferences. Below we list the main preferences you might want to adjust before starting your first transfer. ++++ Click to display section | In order to ensure that your transfer completes correctly, we suggest that you navigate to the ''Edit --> Preferences...'' tab in the top menu and adjust your preferences. The picture below shows what you need to click to make the preference menu appear. {{ :rdms:access:windows:pref_1.png?direct&600 |}} Once the menu is open, navigate to the transfer tab (red box) and click it. There are other preferences available at this stage, but we recommend that you only change how your transfers are handled. {{ :rdms:access:windows:pref_2.png?direct&600 |}} This is not a necessary change, but if you are running into connection timeout issues with Cyberduck, navigate the the "General" tab in the "Transfers" window. Changing the option under "Transfer Files" from "Open Multiple Connections" to "Open Single Connection" can help solve that problem. Furthermore, be sure that you **disable the option for segmented downloads** as this can introduce issues with your downloads. {{ :rdms:access:windows:pref_3_new.png?direct&600 |}} In the same section of Cyberduck's preferences, you can also adjust your default download folder. If you wish to change the current setting, just click on the ''Choose...'' button to the right and then navigate to the folder you wish to set as preferred download destination. Finally, the one option we recommend you turn on in your preferences is to **always verify the checksum of your files**, both in upload and in download, otherwise only the size is used. To do this, simply check the two boxes as shown in the picture. {{ :rdms:access:windows:pref_4.png?direct&600 |}} ++++