Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
dcc:itsol:whisper:scripts [2024/08/12 10:15] – fixed spacing in the "Specialized scripts" section giuliodcc:itsol:whisper:scripts [2025/09/10 13:40] (current) – Minor text changes alba
Line 2: Line 2:
 ===== Building the scripts ===== ===== Building the scripts =====
  
-We will **run Whisper using a script** in order to facilitate the use of the tool. Follow the steps here to set up the script and run it, read the next section of the guide to learn more about the content of the script file itself.+We will **run Whisper using a script** in order to facilitate the use of the tool. Follow the steps here to set up the script and run it. Read the next section of the guide to learn more about the content of the script file itself.
  
 In order to run the script, you will first have to create it. Open your text editor of choice and copy the highlighted code below into the new file. Save the file with the name: ''whisper_runall.sh''. In order to run the script, you will first have to create it. Open your text editor of choice and copy the highlighted code below into the new file. Save the file with the name: ''whisper_runall.sh''.
 +
 +**Note:** The PyTorch module needed to install Whisper has changed due to an update in the dependencies of Whisper. The module displayed in the screenshots is the previous version. Please make sure to **use the version of the module you find in the text**.
  
  
Line 11: Line 13:
 ---- ----
  
 +<code>
 +#!/bin/bash
 +#SBATCH --time=08:00:00
 +#SBATCH --gpus-per-node=1
 +#SBATCH --mem=16000
  
-''#!/bin/bash'' +module load PyTorch/2.1.2-foss-2023a-CUDA-12.1.1 
- +source $HOME/.envs/whisper/bin/activate 
-''#SBATCH %%--%%time=08:00:00'' +whisper $HOME/whisper_audio/* --model large-v2 --output_dir $HOME/whisper_output/ 
- +</code>
-''#SBATCH %%--%%gpus-per-node=1'' +
- +
-''#SBATCH %%--%%mem=16000'' +
- +
-\\ +
- +
-''module load PyTorch/1.12.1-foss-2022a-CUDA-11.7.0'' +
- +
-''source $HOME/.envs/whisper/bin/activate'' +
- +
-''whisper $HOME/whisper_audio/* %%--%%model large-v2 %%--%%output_dir $HOME/whisper_output/'' +
 ---- ----
  
Line 35: Line 30:
 The example below uses the ''vi'' text editor found in HPC to create the script. Follow the instructions in the figures to create the script using this specific text editor: The example below uses the ''vi'' text editor found in HPC to create the script. Follow the instructions in the figures to create the script using this specific text editor:
  
-  * Type or copy ''vi whisper_runall.sh'' into the terminal, then press "enter". This will create a new and empty file that you can edit. The terminal is going to change what is displayed when ''vi'' starts. {{ :dcc:itsol:whisper:script_1.png?direct&800 | }} +  * Type or copy ''vi whisper_runall.sh'' into the terminal, then press "enter". This will create a new and empty file that you can edit. The terminal is going to change what is displayed when ''vi'' starts. {{ :dcc:itsol:whisper:script_1a.png?direct&600 | }} 
-  * Press the "i" key on your keyboard to enable editing of the file. Check the figure to know if the editor is in the correct mode. {{ :dcc:itsol:whisper:script_2.png?direct&800 | }} +  * Press the "i" key on your keyboard to enable editing of the file. Check the figure to know if the editor is in the correct mode. {{ :dcc:itsol:whisper:script_2a.png?direct&600 | }} 
-  * Click on the terminal with the mouse wheel to paste the content of the script into the file. If you see the message in the figure displayed, click "OK" to complete the pasting. The content of the script can be found at the top of this section. {{ :dcc:itsol:whisper:script_3.png?direct&800 | }}+  * Copy the content of the script from the top of this section, then click on the terminal with the mouse wheel to paste the content of the script into the file you just opened.
   * Double-check that the content of the script is correct. If it is, it should look exactly like in the picture below.   * Double-check that the content of the script is correct. If it is, it should look exactly like in the picture below.
-  * **Note**: The colors displayed are also important, because it means that the editor recognizes the words in the text as script commands. {{ :dcc:itsol:whisper:script_4.png?direct&800 | }} +  * **Note**: The colors displayed are also important, because they mean that the editor recognizes the words in the text as script commands. {{ :dcc:itsol:whisper:script_4a.png?direct&600 | }} 
-  * Finally, to save the file and exit from ''vi'', first press ''esc'' on your keyboard. Then type '':wq'' directly on your keyboard. The input should be displayed at the bottom of the terminal like shown in the figure. Press enter to commit the command. The punctuation is to let the editor know that a command is coming, the ''w'' stands for "write", while the ''q'' stands for "quit". {{ :dcc:itsol:whisper:script_5.png?direct&800 | }} +  * Finally, to save the file and exit from ''vi'', first press ''esc'' on your keyboard. Then type '':wq'' directly on your keyboard. The input should be displayed at the bottom of the terminal as shown in the figure. Press Enter to commit the command. The punctuation is to let the editor know that a command is coming; the ''w'' stands for "write", while the ''q'' stands for "quit". {{ :dcc:itsol:whisper:script_5a.png?direct&600 | }} 
-  * If you want to make sure that the script has been saved, type ''ls'' into the terminal, then press "enter". If the script has been saved, it shuld now appear in the list of files you have in your home directory. {{ :dcc:itsol:whisper:script_6.png?direct&800 | }}+  * If you want to make sure that the script has been saved, type ''ls'' into the terminal, then press "enter". If the script has been saved, it should now appear in the list of files you have in your home directory. {{ :dcc:itsol:whisper:script_6a.png?direct&600 | }}
  
 ==== Content of the batch script ==== ==== Content of the batch script ====
Line 62: Line 57:
   * ''#SBATCH %%--%%gpus-per-node=1''   * ''#SBATCH %%--%%gpus-per-node=1''
  
-This line tells the cluster that the script is asking for 1 GPU to be allocated to this job. For Whisper, 1 GPU is more than enough to run the transcription, please do not modify this parameter.+This line tells the cluster that the script is asking for 1 GPU to be allocated to this job. For Whisper, 1 GPU is more than enough to run the transcription. Please do not modify this parameter.
  
   * ''#SBATCH %%--%%mem=16000''   * ''#SBATCH %%--%%mem=16000''
  
-This line specifies the amount of Memory/RAM asked for this job. In the default case, the script asks for 16GB of RAM to be allocated.+This line specifies the amount of Memory/RAM required for this job. In the default case, the script asks for 16GB of RAM to be allocated.
  
 \\ \\
Line 72: Line 67:
 The next two lines make sure that the virtual environment and the dependencies that Whisper needs to run are correctly loaded: The next two lines make sure that the virtual environment and the dependencies that Whisper needs to run are correctly loaded:
  
-  * ''module load PyTorch/1.12.1-foss-2022a-CUDA-11.7.0''+  * ''module load PyTorch/2.1.2-foss-2023a-CUDA-12.1.1''
  
-This line loads the program packages that Whisper needs to run. Please be sure to not modify it, otherwise the script is not going to load the correct dependencies.+This line loads the program packages that Whisper needs to run. Please be sure not to modify it; otherwise, the script is not going to load the correct dependencies.
  
   * ''source $HOME/.envs/whisper/bin/activate''   * ''source $HOME/.envs/whisper/bin/activate''
Line 98: Line 93:
 === Forced English === === Forced English ===
  
-This script forces Whisper to transcribe the audio into English. Use this script if the automatic language detection results in the wrong language (i.e. a strong English accent being recognized as Welsh, instead of English). The same concept works for other supported languages, for example Dutch. To change which language is forced, simply substitute the string ''English'' with the desired language behind the ''%%--%% language'' command.+This script forces Whisper to transcribe the audio into English. Use this script if the automatic language detection results in the wrong language (i.e., a strong English accent being recognized as Welsh, instead of English). The same concept works for other supported languages, for example, Dutch. To change which language is forced, simply substitute the string ''English'' with the desired language behind the ''%%--%% language'' command.
  
-When you save the script, you can call it ''whisper_forcedEnglish.sh''. If you forced a different language, we advise you to label it accordingly. To execute it, simply type into the terminal ''sbatch whisper_forcedEnglish.sh'' and follow the same steps as the general script (see [[dcc:itsol:whisper:running|here]]).+When you save the script, you can call it ''whisper_forcedEnglish.sh''. If you force a different language, we advise you to label it accordingly. To execute it, simply type into the terminal ''sbatch whisper_forcedEnglish.sh'' and follow the same steps as the general script (see [[dcc:itsol:whisper:running|here]]).
  
 ++++ Click to display the script | ++++ Click to display the script |
 +<code>
 +#!/bin/bash
 +#SBATCH --time=08:00:00
 +#SBATCH --gpus-per-node=1
 +#SBATCH --mem=16000
  
-''#!/bin/bash'' +module load PyTorch/2.1.2-foss-2023a-CUDA-12.1.1 
- +source $HOME/.envs/whisper/bin/activate 
-''#SBATCH %%--%%time=08:00:00'' +whisper $HOME/whisper_audio/* --model large-v2 --language English --output_dir $HOME/whisper_output/ 
- +</code>
-''#SBATCH %%--%%gpus-per-node=1'' +
- +
-''#SBATCH %%--%%mem=16000'' +
- +
-\\ +
- +
-''module load PyTorch/1.12.1-foss-2022a-CUDA-11.7.0'' +
- +
-''source $HOME/.envs/whisper/bin/activate'' +
- +
-''whisper $HOME/whisper_audio/* %%--%%model large-v2 %%--%%language English %%--%%output_dir $HOME/whisper_output/'' +
 ++++ ++++
 \\ \\
 === Translate instead of transcribe === === Translate instead of transcribe ===
  
-Whisper is also capable of translating any X language into English. To let the program know that you wish to see a translation instead of a transcription, you need to specify which ''%%--%% task'' the program needs to perform. The script below is already edited to perform a translation. Please keep in mind that the transcript will **only** be translated then, and **the original text will not be displayed in the output files**. If you need to have the original as a means of comparison, you can either first run the general script on the audio, or you can run a forced language script (see above) before you run the translation.+Whisper is also capable of translating any X language into English. To let the program know that you wish to see a translation instead of a transcription, you need to specify which ''%%--%% task'' the program needs to perform. The script below is already edited to perform a translation. Please keep in mind that the transcript will **only** be translated then, and **the original text will not be displayed in the output files**. If you need to have the original as a means of comparison, you can either first run the general script on the audio or you can run a forced language script (see above) before you run the translation.
  
-**Note**: Regardless of whether you run the transcription or the translation first, the file names of the output files will be the exact same. In order for the second operation (translation or transcription) to not overwrite the first, you need to rename the output files before you run the second operation. In this way, the output of your first operation will remain untouched by the second operation.+When you save the script, you can call it ''whisper_translate.sh''. To execute it, simply type into the terminal ''sbatch whisper_translate.sh'' and follow the same steps as the general script (see [[dcc:itsol:whisper:running|here]]). 
 + 
 +**Note**: Regardless of whether you run the transcription or the translation first, the file names of the output files will be the exact same. For the second operation (translation or transcription) not to overwrite the first, you need to rename the output files before you run the second operation. In this way, the output of your first operation will remain untouched by the second operation.
  
 ++++ Click to display the script | ++++ Click to display the script |
 +<code>
 +#!/bin/bash
 +#SBATCH --time=08:00:00
 +#SBATCH --gpus-per-node=1
 +#SBATCH --mem=16000
  
-''#!/bin/bash'' +module load PyTorch/2.1.2-foss-2023a-CUDA-12.1.1 
- +source $HOME/.envs/whisper/bin/activate 
-''#SBATCH %%--%%time=08:00:00'' +whisper $HOME/whisper_audio/* --model large-v2 --task translate --output_dir $HOME/whisper_output/ 
- +</code>
-''#SBATCH %%--%%gpus-per-node=1'' +
- +
-''#SBATCH %%--%%mem=16000'' +
- +
-\\ +
- +
-''module load PyTorch/1.12.1-foss-2022a-CUDA-11.7.0'' +
- +
-''source $HOME/.envs/whisper/bin/activate'' +
- +
-''whisper $HOME/whisper_audio/* %%--%%model large-v2 %%--%%task translate %%--%%output_dir $HOME/whisper_output/'' +
 ++++  ++++