This page contains details on how to build the Hypre package on various clusters for both CPU and GPU implementations.
Building Hypre for a Linux system
The code developers have only attempted to installed Hypre on Linux based systems. Before you begin installation, ensure that you have a suitable installation of MPI. The Makefiles for LTP-PIC default to searching for a version of OpenMPI, therefore we recommend installing this flavour of MPI. To install OpenMPI in Ubuntu for example you could execute the following command:
sudo apt install openmpi-bin
After navigating to the directory where you would like to install Hypre generally the following instructions will work:
Download the latest version of Hypre:
git clone https://github.com/hypre-space/hypre.git
Navigate to hypre/src and configure the install with the following settings:
./configure
Install this configuration via:
make clean
make
Building Hypre for Stellar
The PPPL partition of the Stellar cluster at Princeton University has 296 nodes with Quad Cascade Lake, Platinum 8268, 2.9 GHz processors connected by Infiniband. There are four CPUs per node with 24 cores per CPU, for a total of 96 cores per node, and 8 GB of RAM are available per core
Hypre can be built only for the CPU, and we generally rely on the Intel suite of compilers to best target the available architecture.
To begin, load the following modules:
module load intel/2021.1.2
module load openmpi/intel-2021.1/4.1.2
Download the latest version of Hypre:
git clone https://github.com/hypre-space/hypre.git
Navigate to hypre/src and configure the install with the following settings:
./configure
Install this configuration via:
make clean
make
Building Hypre for Traverse
The Traverse cluster at Princeton University has 46 IBM POWER9 nodes with four NVIDIA V100 GPUs per node. Hypre can be built to run on the NVIDIA GPUs, however depending on the problem size, LTP-PIC may run faster when relying on the CPU build of Hypre.
LTP-PIC relies on the OpenACC programming standard to accelerate the code on GPUs. The NVIDIA-HPC compilers are best suited for this standard, and therefore where possible we build Hypre using these compilers.
Building Hypre for CPUs on Traverse
Current as of 03/08/2022 (DD/MM/YYY) for the modules listed below.
Load the following modules,
module load nvhpc/22.5
module load openmpi/nvhpc-22.5/4.1.3/64
Download the latest version of Hypre:
git clone https://github.com/hypre-space/hypre.git
Hypre has not yet been updated to be aware of the new naming of the NVIDIA compilers, therefore we must explicitly set the compilers during configuration. Navigate to hypre/src and configure the install with the following settings:
./configure CC=mpicc CXX=mpic++ FC=mpifort
Install this configuration via:
make clean
make
Building Hypre for GPUs on Traverse
Current as of 03/08/2022 (DD/MM/YYY) for the modules listed below.
Load the following modules,
module load nvhpc/22.5
module load openmpi/nvhpc-22.5/4.1.3/64
Download the latest version of Hypre:
git clone https://github.com/hypre-space/hypre.git
Hypre has not yet been updated to be aware of the new naming of the NVIDIA compilers, therefore we must explicitly set the compilers during configuration. Navigate to hypre/src and configure the install with the following settings:
./configure CC=mpicc CXX=mpic++ FC=mpifort HYPRE_CUDA_SM=70 CUDA_HOME=/usr/local/cuda-11.7 --with-cuda
Install this configuration via:
make clean
make
This step generally takes much longer than when compiling only for CPU.
Building Hypre for Perlmutter
Perlmutter is a supercomputer constituting part of the DOE's National Energy Research Scientific Computing Center (NERSC). The system has two partitions, partition 1 comprises of heterogeneous CPU+GPU nodes and partition 2 comprises of only CPU nodes. Our instructions demonstrate compilation with the recommended Cray compilers, however other compiler options may also work.
Building Hypre for CPUs on Perlmutter
Installation instructions are very similar to the generic instructions above, however we must use the correct commands at the configuration step to target Perlmutter's compiler wrappers.
To begin, swap the following module to load the Cray compilers:
module swap PrgEnv-gnu/8.3.3 PrgEnv-cray/8.3.3
NOTE: If you want to run the CPU installation of Hypre in combination with GPUs for the rest of the code we recommend compiling a CPU version of Hypre using the Nvidia compiler. To do so replace PrgEnv-cray/8.3.3 with PrgEnv-nvidia/8.3.3 in the corresponding step above.
Download the latest version of Hypre:
git clone https://github.com/hypre-space/hypre.git
Navigate to hypre/src and configure the install with the following settings, make sure to include the compiler wrapper links:
./configure CC=cc CXX=CC FC=ftn
Install this configuration via:
make clean
make
Building Hypre for GPUs on Perlmutter
To begin, swap the following module to load the NVIDIA compilers:
module swap PrgEnv-gnu/8.3.3 PrgEnv-nvidia/8.3.3
Download the latest version of Hypre:
git clone https://github.com/hypre-space/hypre.git
Navigate to hypre/src and configure the install with the following settings,
./configure CC=cc CXX=CC FC=ftn HYPRE_CUDA_SM=80 --with-cuda
Install this configuration via:
make clean
make
Building Hypre for CPUs on Antya
Current as of 15/07/2024 (DD/MM/YYY) for the modules listed below.
Load the following modules,
module load intel-2019
Download the latest version of Hypre:
git clone https://github.com/hypre-space/hypre.git
Navigate to hypre/src and configure the install with the following settings:
./configure --disable-fortran
Install this configuration via:
make clean
make
Building Hypre for GPUs on Antya
Current as of 01/08/2024 (DD/MM/YYY) for the modules listed below.
Load the following modules,
module load nvhpc/20.7
Download the latest version of Hypre:
git clone https://github.com/hypre-space/hypre.git
Navigate to hypre/src and configure the install with the following settings:
./configure --disable-fortran
Install this configuration via:
make clean
make