cgeb2001's DokuWiki!

User Tools

Site Tools

Trace:
using_perun

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
using_perun [2017/06/02 08:52] – created cgeb2001using_perun [2021/09/08 13:29] (current) – 134.190.232.139
Line 1: Line 1:
-**Accessing and Using Perun**+====== Accessing and Using Perun ====== 
  
 for an account contact  for an account contact 
Line 9: Line 10:
 (username is perun, password is Perun_) as well as other useful pages about using perun. (username is perun, password is Perun_) as well as other useful pages about using perun.
  
-When text appears smaller and in the font courier this indicates that you should type it either on the command line or into a text editor as part of a script.+When text appears in bold this indicates that you should type it either on the command line or into a text editor as part of a script.
  
 Open a terminal on the Mac. Open a terminal on the Mac.
 Login to perun using Login to perun using
-ssh -l username perun.biochem.dal.ca+ 
 + 
 +**ssh -l username perun.biochem.dal.ca** 
 + 
 You will be asked for your password You will be asked for your password
 Then, always Then, always
-qrsh+ 
 + 
 +**qrsh** 
 + 
 You may or may not be asked for your password. If you don't enter qrsh you will be in a limited part of the system, unable to access all the programs. You may or may not be asked for your password. If you don't enter qrsh you will be in a limited part of the system, unable to access all the programs.
  
 You will always start in your home directory. You will always start in your home directory.
-You will notice on the cursor line that the absolute pathname of you location is present and should look something like this+You will notice on the cursor line that the absolute pathname of your location is present and should look something like this
 /home/username@perun11 /home/username@perun11
 /home/username is the absolute pathname while @perun11 indicates that you have been assigned to node 11 /home/username is the absolute pathname while @perun11 indicates that you have been assigned to node 11
Line 26: Line 35:
 Every person who has an account on perun has a home directory. To see the other Every person who has an account on perun has a home directory. To see the other
 home directories home directories
-cd ..+ 
 + 
 +**cd ..** 
 + 
 then then
-ls+ 
 + 
 +**ls** 
 + 
 You should see the names of the other perun account holders. Unless they have modified their home directory permissions you cannot enter or access the contents of their home directories (unless you are a superuser). To go back to your home directory simply type You should see the names of the other perun account holders. Unless they have modified their home directory permissions you cannot enter or access the contents of their home directories (unless you are a superuser). To go back to your home directory simply type
-cd+ 
 + 
 +**cd** 
 + 
 This will work regardless of where you are in the computer system. This will work regardless of where you are in the computer system.
  
Line 37: Line 58:
 Therefore, if you want to work with large files (like the ones generated by NGS machines) and/or produce large result files you should do this in /scratch1/username or /scratch2/username. The storage capacity for scratch1 is 21 Tb and 64 Tb for scratch2.   One thing to keep in mind is that the home directory is backed up three times a week while the scratch directories are not backed up. The cluster is stable and there is little chance that any data in the scratch areas will be lost but to be on the safe side you should keep or copy your important files in your home directory and/or on your local computer. Therefore, if you want to work with large files (like the ones generated by NGS machines) and/or produce large result files you should do this in /scratch1/username or /scratch2/username. The storage capacity for scratch1 is 21 Tb and 64 Tb for scratch2.   One thing to keep in mind is that the home directory is backed up three times a week while the scratch directories are not backed up. The cluster is stable and there is little chance that any data in the scratch areas will be lost but to be on the safe side you should keep or copy your important files in your home directory and/or on your local computer.
 To see the capacity and how much is currently being used on the system To see the capacity and how much is currently being used on the system
-df -h+ 
 + 
 +**df -h** 
  
 Running Scripts and Processes - Command line vs the Queue Running Scripts and Processes - Command line vs the Queue
 Perun contains lots of scripts and programs for your use. There are two main ways to run scripts/programs on perun. You can simply run the process from the command line Perun contains lots of scripts and programs for your use. There are two main ways to run scripts/programs on perun. You can simply run the process from the command line
-./myscript.pl datafile1 datafile2+ 
 + 
 +**./myscript.pl datafile1 datafile2** 
 + 
 and the script/program will run right away. This is what you are doing when you issue a UNIX command like ls or cd. You are running a little script called ls or cd that does something very simple and very fast. The other way to run a script/program is by putting the appropriate commands and options in a little script and then submitting the script to a queue. The queue is a perun program that decides where and when to run your script/program depending on what computing resources are available. This means, depending on how many users are using the system and what jobs they are running, that your script/program might not run right away. Once there are CPUs available the queue will then allow your script/program to run. and the script/program will run right away. This is what you are doing when you issue a UNIX command like ls or cd. You are running a little script called ls or cd that does something very simple and very fast. The other way to run a script/program is by putting the appropriate commands and options in a little script and then submitting the script to a queue. The queue is a perun program that decides where and when to run your script/program depending on what computing resources are available. This means, depending on how many users are using the system and what jobs they are running, that your script/program might not run right away. Once there are CPUs available the queue will then allow your script/program to run.
 So, why would you submit your script/program to the queue rather than run it directly? So, why would you submit your script/program to the queue rather than run it directly?
-1. Perun has 15 nodes with multiple CPUs per node. When you run something on the command line directly it is only using the computing power of the head node which is used by everyone else to do simple tasks. Consequently you are not using the full potential of perun and you are slowing down everyone else's simple tasks like ls or cd.+1. Perun has 17 nodes with multiple CPUs per node. When you run something on the command line directly it is only using the computing power of the head node which is used by everyone else to do simple tasks. Consequently you are not using the full potential of perun and you are slowing down everyone else's simple tasks like ls or cd.
 2. You may not be getting all the appropriate environmental parameters necessary to run the script/program because it is expecting to be called through the queue. 2. You may not be getting all the appropriate environmental parameters necessary to run the script/program because it is expecting to be called through the queue.
 3. The script/program will run in the background without you having to have a terminal open or even logged in to perun. 3. The script/program will run in the background without you having to have a terminal open or even logged in to perun.
Line 56: Line 84:
 Either open a text editor on perun (pico, vi, emacs) or create the script on your local computer (and then copy and paste it into a pico editor on perun or scp it to perun). Either open a text editor on perun (pico, vi, emacs) or create the script on your local computer (and then copy and paste it into a pico editor on perun or scp it to perun).
 Every submission script should start with the line Every submission script should start with the line
-#!/bin/sh + 
-at the very top. This tells the UNIX system what type of shell to use (this one is the bourne shell). You could use other UNIX shells like csh or tcsh if you wanted to.+ 
 +**#!/bin/bash** 
 + 
 + 
 +at the very top. This tells the UNIX system what type of shell to use (this one is the Bourne-Again shell). You could use other UNIX shells like csh or tcsh if you wanted to.
 The next two lines should be The next two lines should be
-#$ -S /bin/sh+ 
 + 
 +**#$ -S /bin/bash** 
 + 
 and and
-. /etc/profile+ 
 + 
 +**. /etc/profile** 
 + 
 to ensure that  the correct and current environmental variables are inherited to ensure that  the correct and current environmental variables are inherited
  
  
 Then you can have options beneath these first lines. Some of the more common ones are Then you can have options beneath these first lines. Some of the more common ones are
- #$ -cwd (this means run the process in the current working directory ie where you submitted the script) + 
-#$ -o logfilename_of_your_choice (this creates a log file that contains any output that your script/program might generate that is not directly related to the results, so things like error messages, updates of where the process is, etc) + 
-#$ -pe threaded number (how many CPUs you want to assign to the task - the default is 1 which is typically adequate for most jobs - there are 204 slots)+** #$ -cwd** 
 + 
 + (this means run the process in the current working directory ie where you submitted the script) 
 + 
 + 
 +**#$ -o logfilename_of_your_choice**  
 + 
 +(this creates a log file that contains any output that your script/program might generate that is not directly related to the results, so things like error messages, updates of where the process is, etc) 
 + 
 + 
 +**#$ -pe threaded number** 
 + 
 + (how many CPUs you want to assign to the task - the default is 1 which is typically adequate for most jobs)
  
 After the options you would then have the name of the program you are calling along with any program parameters and/or options. When you run the program from the command line you often just need the name of the program (and any options or parameters). However, when you call a program from a submission script it is generally a good idea to use the full path of the program like so After the options you would then have the name of the program you are calling along with any program parameters and/or options. When you run the program from the command line you often just need the name of the program (and any options or parameters). However, when you call a program from a submission script it is generally a good idea to use the full path of the program like so
-/opt/perun/bin/blastn blah blah blah+**/opt/perun/bin/blastn blah blah blah**
 (See below for how to find the pathname of programs). (See below for how to find the pathname of programs).
  
 Submitting the script Submitting the script
 Once the submission script is finished save it. To submit the script Once the submission script is finished save it. To submit the script
-qsub name_of_script+ 
 + 
 +**qsub name_of_script**
  
  
Line 83: Line 137:
 There are special queues for large computationally heavy  job like large assemblies or large trees. To use these special queues you use a modified qsub command There are special queues for large computationally heavy  job like large assemblies or large trees. To use these special queues you use a modified qsub command
  
-qsub -q 144G-batch name_of_script (nodes with 144 GB RAM, 24 slots per node (1)) +**qsub -q 144G-batch name_of_script** (nodes with 144 GB RAM, 24 slots per node (1)) 
-qsub -q 16G-batch name_of_script (nodes with 16 GB RAM, 8 slots per node (12)) +**qsub -q 16G-batch name_of_script** (nodes with 16 GB RAM, 8 slots per node (12)) 
-qsub -q 256G0batch name_of _script  (nodes with 256 GB RAM, 48 slots per node (2))+**qsub -q 256G-batch name_of _script**  (nodes with 256 GB RAM, 48 slots per node (2))
  
  
Line 93: Line 147:
 Monitoring your run Monitoring your run
 To see if you job is running To see if you job is running
-qstat+ 
 + 
 +**qstat** 
 + 
 This will list all the jobs currently being run on perun as well as who is logged in. To just see your jobs This will list all the jobs currently being run on perun as well as who is logged in. To just see your jobs
-qstat -u username+ 
 +you would normally do qstat -u username but for some reason that doesn't work on perun so do 
 + 
 +**qstat | grep username** 
 + 
 +or  
 + 
 +**watch qstat** (control c to exit). 
 + 
 + 
 + 
 The qstat columns give you information about the processes that are being run. The first column is the job id. This can be used to delete/stop a job The qstat columns give you information about the processes that are being run. The first column is the job id. This can be used to delete/stop a job
-qdel job_id+ 
 + 
 +**qdel job_id** 
 + 
 The third column is the name of the process or script being run. If it says QRLOGIN this is simply the terminal (and the qrsh command you issued at the beginning of the session).  The third column is the name of the process or script being run. If it says QRLOGIN this is simply the terminal (and the qrsh command you issued at the beginning of the session). 
 The fourth column is the username of who is running the script/process. The fourth column is the username of who is running the script/process.
 The fifth column indicates whether the process is running or not. An r means it is actively running while qw means it is waiting in the queue for a slot to be free. If you see Eqw (this should not happen very often) it means your process is waiting in an error state and probably should be deleted. To see what the error is The fifth column indicates whether the process is running or not. An r means it is actively running while qw means it is waiting in the queue for a slot to be free. If you see Eqw (this should not happen very often) it means your process is waiting in an error state and probably should be deleted. To see what the error is
-qstat -j job_id  | grep error+ 
 + 
 +**qstat -j job_id  | grep error** 
  
 If you do ls in the directory where you submitted the script myscript.x to the queue you will see that there is a file called something like this - myscript.x.e798764.  This is the error file associated with your submission and one is created every single time you submit something regardless of whether there are any errors. Usually this error file will be empty if the script/program runs successfully. However, sometimes it contains useful information generated by the script/program that has nothing to do with errors so once your process finishes you should check this file. The number after the e is the job_id of the process. If you do ls in the directory where you submitted the script myscript.x to the queue you will see that there is a file called something like this - myscript.x.e798764.  This is the error file associated with your submission and one is created every single time you submit something regardless of whether there are any errors. Usually this error file will be empty if the script/program runs successfully. However, sometimes it contains useful information generated by the script/program that has nothing to do with errors so once your process finishes you should check this file. The number after the e is the job_id of the process.
  
-You can have multiple perun sessions/terminals open at any one time. You can also remain logged in indefinitely. If you wish to exit you can close the terminal or else enter  +You can have multiple perun sessions/terminals open at any one time. You can also remain logged in indefinitely. If you wish to exit you can close the terminal or else enter. 
-exit+ 
 +**qalter -q [node you want, eg 256G-batch] [job ID number]** 
 + 
 +The command above can change the job's hard queue after submission (without killing it). 
 + 
 + 
 +**exit** 
 + 
  The first exit will terminate the qrsh environment you created so you need to do another  The first exit will terminate the qrsh environment you created so you need to do another
 exit exit
Line 113: Line 197:
  
 Finding the path of an installed program Finding the path of an installed program
-Most programs are installed in common, standard places on UNIX systems. On perun many of the programs/packages are installed in /opt/perun/ or /opt/perun/bin/ +Most programs are installed in common, standard places on UNIX systems. On perun many of the programs/packages are installed in /opt/perun/ or /opt/perun/bin/ or /scratch2/software/ 
-cd /opt/perun/ + 
-ls + 
-cd bin +**cd /opt/perun/** 
-ls+ 
 + 
 +**ls** 
 + 
 + 
 +//cd bin// 
 + 
 + 
 +**ls** 
 + 
 to see the names of the programs to see the names of the programs
  
 If you know the precise name of the program you can find its path by If you know the precise name of the program you can find its path by
-which velvetg+ 
 + 
 +**which velvetg** 
 + 
 or  or 
-whereis velvetg+ 
 + 
 +**whereis velvetg**
  
  
Line 146: Line 246:
  eg blastn -help  eg blastn -help
 4. If all else fails you can try and read the program code (if it is in perl or python as opposed to  C or C++) and try and figure out what the authors want. 4. If all else fails you can try and read the program code (if it is in perl or python as opposed to  C or C++) and try and figure out what the authors want.
- 
using_perun.1496404346.txt.gz · Last modified: by cgeb2001