Exercise
Convert existing script to a batch script and run it using the sbatch
command from Slurm
Set up
- Locate the file
generate_bad_reads_summary.sh
(which was created in the Loops and Automation lesson)Hint: This file is likely in your
badreads
directory.cd ~/unix_lesson/badreads
- Duplicate this file and call the duplicate file
sbatch_generate_bad_reads_summary.sh
. (Hint: use thecp
command)cp generate_bad_reads_summary.sh sbatch_generate_bad_reads_summary.sh
- Create a new directory inside the
badreads
directory calledsbatch_output
.mkdir sbatch_output
Update the new shell script
Using vim open up the file sbatch_generate_bad_reads_summary.sh
to make some edits. Once inside the file, do the following:
Hint: Remember to change to insert mode in vim before beginning to edit!
- Update these two lines of code below, such that the
grep
output in both cases is redirected to the~/unix_lesson/badreads/sbatch_output/
directory. Do not change the names of the output files.grep -B1 -A2 NNNNNNNNNN $filename > ~/unix_lesson/badreads/${samplename}_badreads.fq grep -cH NNNNNNNNNN $filename >> ~/unix_lesson/badreads/badreads.count.summary
- Add SLURM/
sbatch
directives at the top of the script requesting the following resources:- Use partition
priority
(-p
) - Request 5 minutes (
-t
) - Request 100MB of memory (
--mem
) - Request a single core (
-c
) - Specify the output file name (
-o
) - Specify the error file name (
-e
)
- Use partition
- Save the file and exit vim.
Run the new shell script to start a new job on O2
- Run the new script using the
sbatch
command
Check the job/run
- Use
sacct
to check the status of your job submission - Check the contents of your current directory -
- Are there any new files with names ending in
.out
and.err
? - What are the contents of these two files?
- Are there any new files with names ending in
- Check the contents of
~/unix_lesson/badreads/sbatch_output/
, are the expected files there?
This lesson has been developed by members of the teaching team at the Harvard Chan Bioinformatics Core (HBC). These are open access materials distributed under the terms of the Creative Commons Attribution license (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.