Grace Hopper

Hardware Specifications

There are two GH200 Grace Hopper nodes. The specifications for each node are:

• Processor Family: NVIDIA GH200 Grace Hopper Superchip
• Number of Processors: 1
• Processor Type: 72 core NVIDIA Grace ARM Neoverse V2
• GPU: 1 x NVIDIA H100
• Internal Interconnect: NVIDIA NVLink-C2C 900GB/s
• System Memory: 480 GB LPDDRX
• GPU Memory: 96 GB HBM3
• Memory accessible from GPU: All 576 GB of GPU and system memory is accessible from the GPU
• InfiniBand: HDR200 (operating at 100 Gb/s due to upstream switches)

CPU Architecture: It's very different

These nodes use ARM, a different CPU architecture than the x86/x86_64 used by AMD and Intel. This means that code needs to be compiled for ARM. Scripts should generally still work.

This particular ARM architecture is usually labeled as aarch64 in Linux.

Batch jobs

To use these nodes, your job submissions must include:

• Either -C arm or --constraint=arm
• A GPU request using an argument like --gpus gh200:1

Walltime limit: 1 day, subject to change.

Operating System

The OS will be our Red Hat Enterprise Linux 9 image.

Support

We are installing software on the ARM system as requested. If you need certain tools or software, please let us know.

Code Compilation

In the future, we will add information here about recommendations for which compilers and flags to use.

GH200 vs H100 vs H200

NVIDIA's naming system is confusing. "Grace" (the "G" in "GH200") refers to the CPU generation. "Hopper" (the "H" in "GH200") is the GPU generation. The numbers tacked onto the end appear to be specific to the combination of CPU and GPU. In this case, the GH200 contains an H100 GPU along with a 72-core Grace CPU, which is an NVIDIA ARM CPU. The GH200 does not contain an H200 GPU, which is a different product. Bigger numbers do not always mean better things, such as with the H800 which is a crippled version of the H100.