Group File Sharing

The purpose of a group is to share files with multiple users. Groups work well for researchers who share common datasets and/or development code. Groups also serve as a mechanism for faculty to easily access research data produced by student researchers, and retain it after students have graduated and left the university.

Group Filesharing Overview

• Groups allow multiple users to view/edit/write files to a common directory
• The group owner controls which users have access to group files.
• A filesharing group's storage quota is similar to a user's personal storage quota. This includes:
  • 2TB of high-speed, backed up storage in the group home directory.
  • 20TB of high-speed storage in the group nobackup/autodelete/ directory. Files here are deleted after 12 weeks of no use.
  • 20TB of slow storage in the group nobackup/archive/ directory. Files here are not autodeleted.
• Some coordination must occur between users to prevent data corruption.
• Groups can be associated with a course to automatically grant students read or write access.
• Groups can require members to have certain privileges.
• Web-Only users are not allowed to create, manage, or own filesharing groups

Be aware that within a group folder, multiple users can access, read, and write to the same files simultaneously. If more than one process is writing to the same file at the same time, the file will become corrupted and unrecoverable except through a restore from backup, if the backup exists.

Group Directories

Suppose user freddie is a member of groups grp_namd and grp_qscat. These would be the associated directories:

For convenience, links will be put in freddie's home directory (/home/freddie/fsl_groups) for easy access to group directories:

Requesting Group Filespace

Create a group. Enter a name for the group and submit the form. Note that grp_ is prepended to group names. Initial propagation of group folders and permissions across all Office of Research Computing systems could take 15 minutes. You must log out of existing ssh sessions then log back in. Use the groups command after re-logging in to check if the group is ready to use. Repeat the process until it is available.

Managing Group Members

Once the group is created, click on File Sharing Groups in the left Navigation Column. You will see a list of all groups you are a member of. Click on the group to be modified. Follow the on screen directions to add or remove users from your group.

Group Privileges

Group owners can require members to have certain privileges. At this moment, we support 2 factor authentication (2fa) and ITAR clearance. The 2fa privilege is automatically granted when someone enrolls in 2fa. To request ITAR clearance, please open a support ticket with the name of the individual you are requesting ITAR clearance for.

Tips

• To set the output of your jobs to be accessible by other users in your group, it is recommended that you submit your scripts with something like sg grp_MYGROUP "sbatch myscript.sh".
• You can see your quota and consumption by running the orcquota command on any interactive node.

Caveats

Running your jobs under a different group can cause some interesting debugging problems. ptrace functionality may not work correctly:

$ gdb ./prog -p 1234
...
ptrace: Operation not permitted.

$ strace -p 1234
attach: ptrace(PTRACE_ATTACH, ...): Operation not permitted

This is because ptrace expects the UID and primary GID to match that of the target process. To debug a job running as group grp_MYGROUP, run newgrp grp_MYGROUP. This will place you in a new shell with grp_MYGROUP as your primary group (compare id output before and after). gdb and strace should then work with -p.