There are several ways to move files between your machine, cloud storage, and the supercomputer. The following is provided as a guide on how to do so, but note that we only support use of the programs that we provide or that are centrally installed on the supercomputer (namely Globus, rclone
, and sftp
). Support for other software listed here is outside of our purview--refer to said software's documentation with questions.
Globus is ideal if moving large data sets, such as millions of files, or very large files. Globus file transfer jobs can be initiated from either a cloud-based GUI or the command line. Globus supports moving files between various supercomputing facilities, national labs, and personal computers. Globus also supports sharing datasets with individuals inside and outside of BYU.
Rclone is our recommended tool for transferring files between the supercomputer and cloud storage.
SCP / SFTP
is useful when you know where your files are and simply want to copy them to/from the supercomputer. To do so, open a terminal/command prompt and enter something like (replacing username
with your username, of course):# From supercomputer to local computer:
scp email@example.com:/home/netid/myfile.txt Documents/
# From local computer to supercomputer:
scp Documents/myfile.txt firstname.lastname@example.org:/home/netid/
The interface for sftp
is similar to FTP programs, but the data is transferred encrypted over ssh. See the manual for information about various commands.
SCP / SFTP Graphical Interfaces
Most programs that are specifically designed for transferring files between local and remote storage work somewhat like Norton Commander, with one column for local storage and one column for remote storage to facilitate moving files between them. FileZilla works on all the major operating systems; to get it to behave more nicely, follow this guide (replacing addresses and usernames appropriately). If you are using Windows, WinSCP is a bit more intuitive and easy to set up than FileZilla. Cyberduck is also commonly used, but SSHFS or SFTP Net Drive in combination with your file manager achieves the same effect.
If you want to browse files on the supercomputer using your file manager, you can use SSHFS for Linux or SFTP Net Drive for Windows; ExpanDrive and Mountain Duck work on both Windows and MacOS (ExpanDrive also works on Linux), but are paid. This allows you to transparently interact with the files there as if they were on your local machine--you could open a remote text file with Notepad, or save the csv file you're working on in Excel directly to the supercomputer.
Most graphical file browsers on Linux allow you to navigate to paths like sftp://email@example.com/home/netid
. Sometimes you may use ssh:
(KDE option, but sftp:
works) instead of sftp:
, but that's an uncommon need. You may need to type Ctrl
to open up the URL bar where you can type the path.
allows you to mount supercomputer storage locally from the command line. Install sshfs with your package manager if it isn't included with your distro by default. To mount the supercomputer storage, open a terminal, choose an empty directory (called sc
for the sake of this example), and use the following:sshfs -o follow_symlinks firstname.lastname@example.org:/home/netid sc
After entering your password and verification code, you should be able to navigate to sc
in Finder/Nautilus/Dolphin/Thunar/etc. and view files on the supercomputer.
To unmount, use fusermount -u sc
Profile: create a new one
Username: your username
Drive Letter: the default ("Last Available") works
Upon pressing "Connect" you will be asked for your password and verification code, and File Explorer will open to your supercomputer storage.
As of the writing of this article, one can't use sshfs
within the Windows Subsystem for Linux and SSHFS-Win still doesn't work with two-factor authentication. ExpanDrive and Mountain Duck both work but are not free.