Power outages occurred in the Crabtree Technology Building 11:47 AM - 12:12 PM and again 4:33 PM - 4:47 PM, on Wednesday, August 21, 2019. This affected a portion of our compute cluster (predominantly m8g and m8f nodes). Jobs running on those nodes will be lost. Those nodes should now be available to schedule again. Last Updated Thursday, Aug 22 01:16 pm 2019

Transferring Files

This page covers three ways to transfer files between your machine and the supercomputer:

  • SCP: a command line tool for quickly copying a file or two
  • Local mounting: mount remote storage so that you can browse files on the supercomputer as if they were on your own hard drive
  • Dedicated GUIs: graphical programs designed specifically for moving files between local and remote storage

All the methods described run over ssh are are thus encrypted. Instructions for Windows, macOS, and Linux are included.

If you are interested in transferring files on the supercomputer to and from cloud storage, we recommend using rclone.

SCP

scp is useful when you know exactly where your file is and simply want to copy it to/from the supercomputer. To do so, open a terminal/command prompt and enter something like (replacing username with your username, of course):

# From supercomputer to local computer:
scp username@ssh.rc.byu.edu:/fslhome/username/myfile.txt Documents/
# From local computer to supercomputer:
scp Documents/myfile.txt username@ssh.rc.byu.edu:/fslhome/username/

Linux and macOS computers will likely have scp out of the box; Windows users can download PuTTY and use pscp in the place of scp, or use scp within the Windows Subsystem for Linux.

Local Mounting

If you want to browse files on the supercomputer using your file manager, you can use SSHFS (macOS/Linux) or SFTP Net Drive (Windows). This allows you to transparently interact with the files there as if they were on your local machine--you could open a remote text file with Notepad, or save the csv file you're working on in Excel directly to the supercomputer.

macOS/Linux

sshfs allows you to mount supercomputer storage locally from the command line. For Linux, install sshfs with your package manager if it isn't included; for macOS, download FUSE and SSHFS here. To mount the supercomputer storage, open a terminal, choose an empty directory (called sc for the sake of this example), and use the following:

# For Linux:
sudo sshfs -o allow_other,follow_symlinks \
        username@ssh.rc.byu.edu:/fslhome/username sc
# For macOS:
sudo sshfs -o allow_other,follow_symlinks,defer_permissions \
        username@ssh.rc.byu.edu:/fslhome/username sc

After entering your password and verification code, you should be able to navigate to sc in Finder/Nautilus/Dolphin/Thunar/etc. and view files on the supercomputer.

To unmount, use sudo umount sc.

Windows

Windows users can use SFTP Net Drive to mount supercomputer storage as a Windows drive. Download and install it, open it, then fill out the needed information:

  • Profile: create a new one
  • Server: ssh.rc.byu.edu
  • Username: your username
  • Authentication: Keyboard-interactive
  • Drive Letter: the default ("Last Available") works

Upon pressing "Connect" you will be asked for your password and verification code, and File Explorer will open to your supercomputer storage.

As of the writing of this article, one can't use sshfs within the Windows Subsystem for Linux and SSHFS-Win still doesn't work with two-factor authentication, so SFTP Net Drive seems to be the only simple way to mount remote storage on Windows.

Dedicated GUIs

Most programs that are specifically designed for transferring files between local and remote storage work somewhat like Norton Commander, with one column for local storage and one column for remote storage to facilitate moving files between them. FileZilla works on all the major operating systems; to get it to behave more nicely, follow this guide (replacing addresses and usernames appropriately). If you are using Windows, WinSCP is a bit more intuitive and easy to set up than FileZilla. Cyberduck is also commonly used, but SSHFS or SFTP Net Drive in combination with your file manager achieves the same effect.