| Both sides previous revision Previous revision Next revision | Previous revision |
| rdms:webapp:processes [2025/08/19 14:27] – adjusted a sentance a little, added a comma at another place. jelte | rdms:webapp:processes [2026/08/05 15:01] (current) – burcu |
|---|
| | {{indexmenu_n>6}} |
| | |
| ====== Processes and Jobs ====== | ====== Processes and Jobs ====== |
| {{indexmenu_n>5}} | |
| |
| The RDMS web interface allows you to observe the status of the processes/jobs that you started via the web interface. For that, you can use the ''Processes/jobs'' view on the left-side Menu bar. Moreover, this view allows you to observe the status of your **delayed rules**, a concept that will be explained in this section as well. | You can use the RDMS Web interface to view the status of processes and jobs that you started through the Web interface. Select the "Processes/jobs" icon in the main menu on the left to open this view. |
| | |
| | You can also use this view to check the status of your delayed rules. The "Delayed Rules" section below explains this concept. |
| |
| {{ :rdms:webapp:rdms_procces_0.png?direct&850 |}} | {{ :rdms:webapp:rdms_procces_0.png?direct&850 |}} |
| |
| ===== Processes/Jobs in the Web Interface ===== | ===== Processes and Jobs in the Web Interface ===== |
| If you perform certain operations in the RDMS web interface, these get registered as processes. The status of these processes can be observed here. | |
| |
| Currently, the following operations are visible as processes: | When you perform certain operations through the RDMS Web interface, the RDMS registers them as processes. You can use the "Processes/jobs" view to check their status. |
| - Uploading of data via the web interface, | |
| - Bundling operations (compress to tar or decompress), | |
| - Extraction of metadata, | |
| - Different steps of the [[rdms:workflows:archiving|Archiving Workflow]]. | |
| |
| By clicking on a listed process, you can get more details about it, as shown below. | The following operations are currently displayed as processes: |
| |
| {{ :rdms:webapp:rdms_procces_4.png?direct&850 |}} | * Uploading files through the Web interface |
| | * Compressing folders into tar archives and extracting tar archives |
| | * Extracting metadata |
| | * Performing steps in the [[rdms:workflows:archiving|Archiving Workflow]] |
| |
| For example, this view can show you the kind of process and the data on which it was executed. Moreover, every process gets a process number as well as an ID. You can use the ID to correlate the job to a delayed rule (see below), if applicable. | Select a process to view more information about it, as shown below. |
| |
| **Note:** Not all jobs are executed as delayed rules, as will be described below. For example, the upload operation will not be triggered as a delayed rule, but the computation of the data checksum following the upload will be executed as a delayed rule in the background. | {{ :rdms:webapp:rdms_procces_4.png?direct&850 |}} |
| |
| | The process details show information such as the type of process and the files or folders on which it was performed. Each process has both an "RDMS process ID" and an "ID". When applicable, you can use the "RDMS process ID" to match the process to the corresponding delayed rule. |
| | |
| | **Note:** The RDMS does not execute every process as a delayed rule. For example, uploading a file is not a delayed rule. However, the checksum calculation that starts after the upload runs as a delayed rule in the background. |
| |
| ===== Delayed Rules ===== | ===== Delayed Rules ===== |
| |
| The RDMS is built on top of the data management software [[rdms:rdmsproject:irods|iRODS]]. The iRODS system has the built-in capability to execute certain tasks when a certain event is registered in the system. This automated tasks are called **rules** and they can be delayed, meaning being queued and executed in the background even if the user is not logged in anymore. | The RDMS is built on the [[https://irods.org/|iRODS]] data-management software. iRODS can automatically perform tasks when specific events occur in the system. These automated tasks are called rules. |
| |
| We use rules in several places in the RDMS, often as delayed rules. Important example of delayed rules are: | The RDMS can queue a rule and execute it in the background at a later time, even after you have logged out. This type of rule is called a delayed rule. |
| |
| * **Automated checksum calculations of your data**: If you upload files to the RDMS, an automated checksum calculation for your files with be executed as a delayed rule. This guarantees that all your files have a checksum registered which gives you and us the capability to check the [[rdms:data:integrity|integrity of your data]]. | The RDMS uses delayed rules for several operations, including: |
| * **Bundling/Unbundling via the web interface**: If you use the function to compress a folder to a tar archive or to decompress a tar archive, it will also be executed as a delayed rule. | |
| |
| You can observe the status of your rules via the second tab in the processes/jobs view. | * **Calculating checksums automatically:** When you upload files to the RDMS, the system calculates their checksums by using a delayed rule. The registered checksums allow you and the RDMS administrators to verify the [[rdms:data:integrity|integrity of your data]]. |
| | * **Compressing and extracting files through the Web interface:** When you compress a folder into a tar archive or extract a tar archive, the RDMS performs the operation as a delayed rule. |
| | |
| | To view your delayed rules, select the second tab in the "Processes/jobs" view. |
| |
| {{ :rdms:webapp:rdms_procces_5.png?direct&850 |}} | {{ :rdms:webapp:rdms_procces_5.png?direct&850 |}} |
| |
| The RDMS process ID (highlighted in red) is visible for every delayed rule, which allows you to correlate a job with its respective delayed rule. For example, in the screenshots above, you can see that the process ID is the same for both, the delayed rule and for the related job presented in the section above. | Each delayed rule displays its "RDMS process ID". You can use this value to match the delayed rule to the corresponding process or job. The "ID" is a separate identifier shown in the Web interface. |
| |
| The delayed rule shown here is an example for the creation of a tar archive (data bundling). You will notice that below the first line (''ID:''), a **Priority** of 9 is assigned to this rule. The RDMS assigns **priority 9** to all the delayed rules that **should be executed first**, **priority 5** to delayed rules that have **medium priority**, and finally **priority 1** to all the rules that should be **queued last** (i.e. checksum calculations). | In the examples above, the delayed rule and its related job have the same "RDMS process ID". |
| | |
| | The example below shows a delayed rule for creating a tar archive. The rule details include its assigned priority. The RDMS uses the following priority levels to determine queue preference: |
| | |
| | * **Priority 9:** Rules with high queue priority |
| | * **Priority 5:** Rules with medium queue priority |
| | * **Priority 1:** Rules with low queue priority, such as checksum calculations |
| |
| {{ :rdms:webapp:rdms_procces_8.png?direct&850 |}} | {{ :rdms:webapp:rdms_procces_8.png?direct&850 |}} |
| |
| The priorities are set such that critical tasks, tasks initiated manually by the user, and tasks that are part of a workflow are allowed to jump the queue and be executed before tasks that are all not essential to the functioning of the system, numerous, common, and automatically scheduled. For example, it is important that the checksum of a file is calculated after the file is uploaded, but this calculation should not stop the archiving workflow from moving to the next step. Even if the file was uploaded way before the workflow started, the calculation of the checksum is given minimal priority, so that it may happen when the system is mostly idle. | The priority system allows critical tasks, manually started tasks, and tasks that are part of an [[..:workflows:|RDMS workflow]] to receive a higher queue preference than common, automatically scheduled tasks. |
| | |
| | For example, the RDMS must calculate a checksum after you upload a file. However, this calculation should not delay an Archiving Workflow. The RDMS therefore assigns checksum calculations a lower queue priority, allowing higher-priority operations to be processed first when resources are available. |
| |
| You can also check the status of delayed rules via the CLI tool [[rdms:access:linux:icommands|iCommands]] using the `iqstat` command, which shows the user's delayed rules that await execution. | You can also check the status of delayed rules through the command-line interface (CLI). Use the ''iqstat'' command in ''iCommands'' to display your delayed rules that are waiting to run. |
| |
| {{ :rdms:webapp:rdms_procces_9.png?direct&850 |}} | {{ :rdms:webapp:rdms_procces_9.png?direct&850 |}} |