Differences

This shows you the differences between two versions of the page.

--- dcc:itsol:whisper:datamanage [2024/08/12 08:04] – Moved content from LibGuides to here giulio
+++ dcc:itsol:whisper:datamanage [2026/09/25 10:26] (current) – alba
@@ Line 1: / Line 1: @@
 {{indexmenu_n>5}}
-===== Data management safety measures =====
-Last but not least, it is important that you ensure proper management of your data. HPC is a computing cluster and therefore only intended for the short term storage of mutable data. In order to ensure proper performance, data should not be stored long-term in the cluster. Therefore, after you have processed your files, you should move your processed data to a more suitable location (like University network storage), where you can perform quality control and any text editing. The source audio files and their transcription are considered potentially sensitive data and should not be retained on HPC, but kept in a secure location for future reference.
+====== Data management safety measures ======
+Because you are handling data containing the **voices of (multiple) people**, your data is considered one of the most sensitive kinds of data. For this reason, data uploaded to Hábrók to perform the transcription should not be left idling there. As soon as you know that the transcription has been performed, **download the results to your UG work environment and remove all traces of the data from Hábrók**.
+Hábrók is a computing cluster, intended for the short-term storage of data you are actively working on. Scratch is not backed up, so anything you want to keep must be copied elsewhere first; deleting the only copy is irreversible, so check that your saved copy opens correctly before you remove anything.
+Three things have to be cleared before you are done:
+^What                           ^Where                            ^
+|The recordings you uploaded    |''%%whisper/input/%%''           |
+|The transcripts Whisper created|''%%whisper/output/%%''          |
+|The job log                    |''%%whisper/slurm-<JobID>.out%%''|
+Once all three are downloaded and cleared, you are ready for a new round of transcriptions.
+===== The recordings =====
+Check the transcripts before you delete the audio. If a job has to be run again — because the wrong language was detected, for example — having the recordings still on Hábrók saves you from uploading them a second time. A quick check that the transcription succeeded is enough; the detailed reading can happen in your own work environment.
+Once the transcripts are what you expect, remove the recordings promptly. Leaving transcribed audio on the cluster serves no purpose and increases the risk of a data leak.
+===== The transcripts =====
+Download everything you need from ''%%whisper/output/%%'' and then empty it. The output files can be removed at any time, but clear them before starting a new job so that one run's results are never mistaken for another's.
+===== The job log =====
+The job log sits next to the job script, in the ''%%whisper/%%'' directory rather than in the output directory, and is named after the job: ''%%slurm-<JobID>.out%%''.
+It records what Whisper did — the model it loaded, the files it processed, and any errors. It does not contain the transcript itself. It does contain the file names of your recordings, which can be identifying on their own, so delete it along with the rest.
+<WRAP half column>[[dcc:itsol:whisper:troubleshooting|← Previous: Troubleshooting]]</WRAP>
+<WRAP half column rightalign>[[dcc:itsol:start|Return to DCC IT Solutions Guides →]]</WRAP>
+<WRAP clear></WRAP>
-[[dcc:itsol:start | → Return to DCC IT Solutions Guides]]