Workshop Schedule

Day 1

Time Topic Instructor
9:30 - 10:10 Workshop introduction Will
10:10 - 11:40 Introduction to Shell Jihe
11:40 - 12:00 Overview of self-learning materials and homework submission Will

Before the next class:

I. Please study the contents and work through all the code within the following lessons:

  1. Wildcards and shortcuts in Shell
    Click here for a preview of this lesson
    Perhaps you are interested in only listing the files that have a .txt extension or you want to navigate to your home directory quickly. There are many shortcuts in Shell that will help you do these types of tasks.

    This lesson will cover:
    - Utilizing wildcards for selecting multiple files
    - Implementing shortcuts for moving around the Shell quickly

  2. Examining and creating files
    Click here for a preview of this lesson
    Now that you can navigate around the Shell environment, you are likely interested to know how to view and edit your files.

    This lesson will cover:
    - Viewing your files
    - Editing your files using vim

  3. Searching and redirection
    Click here for a preview of this lesson
    You will encounter large files that need a search function to find the information you are looking for. You might also be interested in writing the output of that search to a file or use it as the input to another function.

    This lesson will cover:
    - Searching files using grep
    - Writing the output of a command to a file
    - Redirecting the output of a command to an additional command

  4. Shell scripts and variables in Shell
    Click here for a preview of this lesson
    Being able to reproduce your work is critically important for good science and writing your commands to a Shell script is a simple way to do this. Additionally, when writing shell scripts you will find it helpful to use variables to hold information, such as paths, in order to avoid typos and improve the readability of your script.

    This lesson will cover:
    - Creating Shell scripts
    - Assigning variables in Shell

NOTE: To run through the code above, you will need to be logged into O2 and working on a compute node (i.e. your command prompt should have the word compute in it).

  1. Log in using ssh rc_trainingXX@o2.hms.harvard.edu and enter your password (replace the “XX” in the username with the number you were assigned in class).
  2. Once you are on the login node, use srun --pty -p interactive -t 0-2:30 --mem 1G /bin/bash to get on a compute node.
  3. Proceed once your command prompt has the word compute in it.
  4. If you log out between lessons (using the exit command twice), please follow points 1. and 2. above to log back in and get on a compute node when you restart with the self learning.

II. Complete the exercises:

Questions?

Day 2

Time Topic Instructor
09:30 - 10:45 Self-learning lessons review Will
10:45 - 12:00 Loops and automation Jihe

Before the next class:

I. Please study the contents and work through all the code within the following lessons:

  1. Permissions and Environment Variables

    Click here for a preview of this lesson
    When using a multi-user system like the O2 cluster, you may want to limit access to your work. Permissions exist to clearly delineate who has the ability to read, write and execute your files.

    Also, when working in a UNIX system, there are a core set of default variables that control the behavior of your command-line. One of the most important of these is the $PATH variable, which tells the system where to look for commands that you give it.

    This lesson will cover:
    - Interepting and modifying existing permissions
    - Querying environmental variables
    - Reading and appending to the $PATH variable

  2. Introduction to High-performance computing

    Click here for a preview of this lesson
    Now that you had a chance to explore the O2 cluster, let's focus on the components of this system, how it is different than your personal computer and the advantages that it offers in terms of parallelization.

    This lesson will cover:
    - Differentiating a high-performance computing cluster like O2 from your personal computer
    - Discuss the large parallelization advantage that O2 has over a personal computer

NOTE: To run through the code above, you will need to be logged into O2 and working on a compute node (i.e. your command prompt should have the word compute in it). For login instructions, please see above.

II. Complete the exercises:

Questions?

Day 3

Time Topic Instructor
09:30 - 10:00 Self-learning lessons review Jihe
10:00 - 11:00 Introduction to the O2 cluster Will
11:00 - 11:30 Exercise (answer key) Jihe
11:30 - 11:45 Introduction to the O2 cluster - data storage Jihe
11:45 - 12:00 Wrap up Will

Dataset

Introduction to Shell: Dataset

Answer keys

Advanced bash commands

If you are interested in learning some more advanced tools for working on the command-line, we encourage you to walk-through the materials linked below:

Resources

Cheat sheets:

Online tutorials:

General help:

These materials have been developed by members of the teaching team at the Harvard Chan Bioinformatics Core (HBC). These are open access materials distributed under the terms of the Creative Commons Attribution license (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.