Getting started on the cluster can feel overwhelming. With a few simple tweaks to your workflow, your experience can be greatly enhanced. If you're still frustrated by something after reading this, please stop by our office or open a support ticket. We'd love to help.

Autocomplete

When typing at the command line, hit the tab key. If nothing happens hit it again.

The tab key is a power tool that helps you navigate directories, find files, and find programs you want to run. If a command can be found in your PATH or you need to navigate to a file location, hit the tab key! It will autocomplete if it finds a singular match. If there are several, hit tab a second time and it will give you a list of options. This saves you a lot of typing, and helps you find things.

Always Logging In Frustration

It can get really frustrating to enter your credentials every time you want a new terminal, VS Code connection, to run a copy command, etc. during a single work session. The great news is you don't have to!

SSH Multiplexing

If you're frequently using commands like ssh or scp to connect to the supercomputer, ssh multiplexing reduces the frequency of logins. Read these instructions to get it set up.

Open OnDemand

Open OnDemand is frequently used for GUI applications and as a Desktop. If you're a Windows user, it helps with the amount of times you need to sign in when using VS Code. If you want several terminals, you can open them all up on the Desktop without having to reauthenticate (or set up ssh multiplexing).

Altering Code

VS Code

Many of our users connect from a local instance of VS Code to the supercomputer. It's great for development. Check out these instructions to get started.

Quick Changes

Sometimes you just need to make a quick change and don't want to open up an IDE. That's where command line editors come into play. Vim and Nano are the most common. Nano is easiest to learn, but vim is more powerful. It is highly recommended to learn vim. If you want to learn how to use the text editor vim (or vi), please run vimtutor after connecting to our systems via ssh, or play Vim Adventures.

Command History

When your terminal closes normally, by default it saves up to the last 1000 commands you ran. There are many ways to take advantage of this.

I Want to Rerun a Command

If the command is in you history, you can arrow up or down to get to it. No need to keep retyping it!

I Keep Forgetting What Command I Need to Run

I Remember Part of the Command

Try using reverse-i-search. From a terminal, hit Ctrl+R and start typing what you remember. The program will find the most recent command with that string. If you need to go back further, hit Ctrl+R again. Hit enter if you want to run it, right or left arrow if you want to edit the command before running it, or Ctrl+C to bail out of the search.

I Know Roughly When I Ran the Command

Running history will print out the most recent commands you've run. Often this gets quite long. less is a powerful tool to add scrolling and search functionality. We can send the output of history to less by running history | less. From there, you can scroll through your history or search it by hitting "\" and typing your query. Hit q to quit. Run man less for more functionality. Since history has timestamps, you can search through those.

I Want to Remember This Command (Long Term or Short Term)

Sometimes you might want to make comments to yourself about commands. Anything followed by a "#" is not interpreted as code, but still saved in your history for later. This is a great way to take notes during trainings.

Sometimes you've written a command, but realized you need to run something else first. Starting the line with "#" enables you to save it as a comment, run your next command, and then arrow back up to the commented one. You just need to remove the "#" to run it now instead of retyping it all.

Frequently Rerunning Commands

On Start Up

If you find yourself running the same commands at the beginning of each session, it's quite easy to automate that. These commands can include, but aren't limited to setting environment variables, adding aliases, or starting up your environment. When it comes to starting up your environment, there's several built in ways for the module system or conda to remember some start up functionality, but you could still do the following methods.

When you have an interactive login bash shell, your ~/.bash_profile file is run. Your ~/.bashrc is run when you create a non-login shell. You can learn more about the differences between the two files here, but most of our users simplify their workflow by having their ~/.bash_profile call their ~./bashrc which will give a similar environment regardless of the shell type. This can be accomplished by running

 echo "[[ -f ~/.bashrc ]] && source ~/.bashrc" >> ~/.bash_profile

Now you can create your ~/.bashrc file and fill it with useful commands to you.

One Time

Some times you find yourself running the same commands one after another, but it's not enough to warrant writing a script to automate it. For example, you might want to create a build directory while using cmake and start the build process. Instead of running

mkdir bld
cd bld
cmake ..
make

and having to retype or arrow up to all those commands in the correct order, you can put them all on one line like so

mkdir bld; cd bld; cmake ..; make

It's important to note that if one commands fails or doesn't execute properly, bash will continue running the other commands. You can now arrow up to get the full line back, but it must be used with caution.

It's Too Long

Commands with lots of parameters and file paths can be a pain to type or even arrow back up to. Nobody wants to have to type something like

cmd param1 param2 param3 param4 /super/long/file/path/to/what/you/need /maybe/there/is/another/input/file /maybe/you/want/to/save/it/here

frequently. One option is to save it as an alias. If it's saved in you history, running !cmd will execute the last command run that starts with 'cmd'.

Environment Frustration

Have you watched your code execute flawlessly in one location only to have it crash with a bunch of missing package or versioning issues in another? Or maybe you can't get it to run, but someone else in your lab has no problems? Believe it or not, but the technical term for this experience is dependency hell. Luckily, there's ways to get out of this.

Mamba (Conda) Environments

Traditionally, this was only a Python solution, but these types of environments now include software across multiple languages.

Conda keeps track of what environment packages are installed and makes sure that they work well together. Mamba is the faster version of Conda and therefore our recommended solution. Check out this page to get started.

Modules

We've seen a lot of software and it's likely we already have what you need installed. Run module avail to see it all. If there's something you need, let us know by stopping by or opening a ticket.

I Don't Want my Session to Die!

Sometimes your laptop is about to die and you don't want to lose your terminal. Sometimes it's going to take 3 hours to copy something and you don't want to babysit your computer. Thankfully, you don't have to! tmux can alleviate much of your pain by putting your processes on a server on our machines. Just sure to check which login node (run hostname) you created it on so that you can get back to it later.

Confusion on the Architecture

The clusters can be quite daunting to wrap your head around if you've never worked with them before. How do you know what resources are available in different locations? If you're trying to visualize what's going on, check out these two images.

If you just need to know the hardware specs for each cluster, our compute resources page details it all.

I'm Still Frustrated...

The ORC office has a great support team that wants to make your life easier! Come stop by our office or open a ticket for a low-stress-custom-tailored new user training, a quick chat, or just some empathy. Odds are good that we've also seen or experienced what you're going through and have another trick up our sleeve to make life easier.