Introduction to Shell and Cluster computing (FAS-RC’s compute cluster)

Learning Objectives

• Understand the necessity for, and use of, the command line interface (bash/shell).

Installations

Mac users: No installation requirements.
Windows users: GitBash

Notes

• These materials focus on the use of local computational resources at Harvard, which are only accessible to Harvard affiliates
• Non-Harvard folks can download the data and set up to work on their local clusters (with the help of local system administrators)

Cluster access

To run through the code in the lessons below, you will need a FAS-RC cluster account. Once you have an account, please do the following.

1. Log in using ssh username@login.rc.fas.harvard.edu and enter your password (replace username with your username).
2. Once you are on the login node, use salloc -p test -t 0-2:30 --mem 8G to get on a compute node or as specified in the lesson.
3. Proceed only once your command prompt does not have the word login in it.
4. If you log out between lessons (using the exit command twice), please follow points 1. and 2. above to log back in and get on a compute node when you restart with the self learning.

Lessons

Part I

1. Introduction to Shell
2. Wildcards and shortcuts in Shell
3. Examining and creating files
4. Searching and redirection
5. Shell scripts and variables in Shell

• Solution to exercises in above lessons

Part II

1. Loops and automation
2. Permissions and Environment Variables

• Solution to exercises in above lessons

Part III

1. Introduction to High-performance computing
2. Introduction to the FAS-RC cluster

---

Building on this workshop

• More fun with bash commands
• Advanced bash commands (aliases, copying files, and symlinks)

---

Resources

Cheat sheets:

• http://fosswire.com/post/2007/08/unixlinux-command-cheat-sheet/
• https://github.com/swcarpentry/boot-camps/blob/master/shell/shell_cheatsheet.md
• tldr_ : Simplified version of the man pages (online and searchable)

Online tutorials:

• Explain Shell
• Introduction to the Command Line for Genomics
• BASH Programming - Introduction HOW-TO
• Bioinformatics from the Command Line

General help:

• Google it! - if you don’t know how to do something, try Googling it, other people have probably had the same question.
• Learn by doing! There’s no real other way to learn this than by trying it out.
  • Use vim on your laptop
  • Move around the directory structure on your laptop using the Terminal/Shell counts
  • Open folders and files using the command open
  • Automate something you don’t really need to automate
• Use man bash to get more information about bash (bourne-again shell)

---

These materials have been developed by members of the teaching team at the Harvard Chan Bioinformatics Core (HBC). These are open access materials distributed under the terms of the Creative Commons Attribution license (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.