This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision |
| habrok:data_management:storage_areas [2025/10/24 10:32] – [/scratch] pedro | habrok:data_management:storage_areas [2026/10/15 12:35] (current) – Move /userapps to Storage areas pedro |
|---|
| ===== /scratch ===== | ===== /scratch ===== |
| |
| Each user of the system also has a directory in ''/scratch'', which is meant for storing large amounts of data that needs to be processed. **Please be aware that backups of this data are not made, and that, because of this, /scratch is not suitable for long term data storage.** This means that you have to copy important results to other storage yourself regularly. This storage can for example be the ''/projects'' or RDMS storage system. | Each user of the system also has a directory in ''/scratch'', which is meant for storing large amounts of data that needs to be processed. <wrap important>**Please be aware that we do not make backups of this data, thus, ''/scratch'' is not suitable for long term data storage.**</wrap> This means that you have to copy important results to other storage yourself regularly. This storage can for example be the ''/projects'' or RDMS storage system. |
| |
| Also on ''/scratch'' quotas are applied to prevent the system from running out of space. Currently the limit is 250GB by default. If this limit is too low for your research purposes, you can request us to change this limit. The limit can be increased to a "fair use" value without issues. When more space is required it is expected that ''/scratch'' is still only used as a staging area for data that will immediately be processed, and that a suitable storage system is available elsewhere for storing the full data collection. These storage systems can again be the ''/projects'' or RDMS systems described below. | Also on ''/scratch'' quotas are applied to prevent the system from running out of space. Currently the limit is 250GB by default. If this limit is too low for your research purposes, you can request us to change this limit. The limit can be increased to a "fair use" value without issues. When more space is required it is expected that ''/scratch'' is still only used as a staging area for data that will immediately be processed, and that a suitable storage system is available elsewhere for storing the full data collection. These storage systems can again be the ''/projects'' or RDMS systems described below. |
| cp -r outputdata /scratch/$USER/mydataset/ | cp -r outputdata /scratch/$USER/mydataset/ |
| </code> | </code> |
| | |
| | ===== /userapps ===== |
| | |
| | Since ''/scratch'' is optimized for large files, storing software (which normally consists of a set of many small files) on ''/scratch'' is not recommended. For large or shared software installations a NFS based file system share has been setup, which is available as ''/userapps''. <wrap important>Since we assume that most software installations use downloads from external sites (like e.g. Python virtual environments) we do not make a backup of ''/userapps''!</wrap> |
| | |
| | Please contact [[hpc@rug.nl]] if you need additional space on ''/userapps'' for your installations, or when you need to share your software stack with multiple users. For the latter you should answer the questions for a group directory above. |
| |
| ====== Relevant external data stores ===== | ====== Relevant external data stores ===== |