Compiling Gromacs 4.6.x on ARCHER Phase 1

Gromacs compilation consists of two steps:

• Compiling the serial tools (for the postprocessing nodes).
• Compiling the parallel MD simulation program (for the comute nodes).

Build the serial tools for the PP nodes

Log into a PP node

You must build on the PP nodes to ensure the correct versions of executables are built:

ssh espp2

Set up your environment for the serial build

Switch to the GNU programming environment:

module swap PrgEnv-cray PrgEnv-gnu

and load the FFTW 3 and CMake modules:

module add fftw
module add cmake

Module list for centrally installed executables:

Currently Loaded Modulefiles:
  1) modules/3.2.6.7                       14) dmapp/6.0.1-1.0500.7263.9.31.ari
  2) eswrap/1.0.20-1.010200.643.0          15) pmi/4.0.1-1.0000.9753.86.2.ari
  3) switch/1.0-1.0500.41328.1.120.ari     16) ugni/5.0-1.0500.0.3.306.ari
  4) craype-network-aries                  17) udreg/2.3.2-1.0500.6756.2.10.ari
  5) PrgEnv-gnu/5.0.41                     18) cray-libsci/12.1.2
  6) atp/1.7.0                             19) gcc/4.8.2
  7) rca/1.0.0-2.0500.41336.1.120.ari      20) craype/2.01
  8) alps/5.0.3-2.0500.8095.1.1.ari        21) pbs/12.1.400.132424
  9) dvs/2.3_0.9.0-1.0500.1522.1.180       22) craype-ivybridge
 10) csa/3.0.0-1_2.0500.41366.1.129.ari    23) cray-mpich/6.1.1
 11) job/1.5.5-0.1_2.0500.41368.1.92.ari   24) packages-archer
 12) xpmem/0.1-2.0500.41356.1.11.ari       25) cmake/2.8.12
 13) gni-headers/3.0-1.0500.7161.11.4.ari

Configure the single-precision build

Create a build directory in the source tree

cd gromacs-4.6.5
mkdir build
cd build

Set the environment variables for the CMake build. (Note, for at least 4.6.3 and 4.6.5, FLAGS=-ffast-math results in errors and test failures.) Note the use of the native compiler commands below.

export CXX=g++
export CC=gcc
export CMAKE_PREFIX_PATH=/home/y07/y07/cse/fftw/3.3.3-phase1-noavx-gcc-pp
export FLAGS="-static -O3 -ftree-vectorize -funroll-loops"

Use CMake to configure the build:

cmake ../ -DCMAKE_SKIP_RPATH=YES -DBUILD_SHARED_LIBS=OFF \
          -DGMX_MPI=OFF -DGMX_OPENMP=ON -DGMX_GPU=OFF -DGMX_X11=OFF -DGMX_DOUBLE=OFF \
          -DCMAKE_C_FLAGS="$FLAGS" -DCMAKE_CXX_FLAGS="$FLAGS" \
          -DCMAKE_INSTALL_PREFIX=/usr/local/packages/gmx/4.6.5-phase1

Build the serial, single-precision code

make install

Configure and build the serial, double-precision code

Create a build directory in the source tree

cd gromacs-4.6.5
mkdir build_d
cd build_d

Set the environment variables for the CMake build. (Note, for at least 4.6.3 and 4.6.5, FLAGS=-ffast-math results in errors and test failures.) Note the use of the native compiler commands below.

export CXX=g++
export CC=gcc
export CMAKE_PREFIX_PATH=/home/y07/y07/cse/fftw/3.3.3-phase1-noavx-gcc-pp
export FLAGS="-static -O3 -ftree-vectorize -funroll-loops"

Use CMake to configure the build:

cmake ../ -DCMAKE_SKIP_RPATH=YES -DBUILD_SHARED_LIBS=OFF \
          -DGMX_MPI=OFF -DGMX_OPENMP=ON -DGMX_GPU=OFF -DGMX_X11=OFF -DGMX_DOUBLE=ON \
          -DCMAKE_C_FLAGS="$FLAGS" -DCMAKE_CXX_FLAGS="$FLAGS" \
          -DCMAKE_INSTALL_PREFIX=/usr/local/packages/gmx/4.6.5-phase1

Build the serial, double-precision code

make install

Build the parallel simulation code for the compute nodes

Set up your environment for the parallel build

Switch to the GNU programming environment:

module swap PrgEnv-cray PrgEnv-gnu

and load the FFTW 3 and CMake modules:

module add fftw
module add cmake

Module list for centrally installed parallel executables:

Currently Loaded Modulefiles:
  1) modules/3.2.6.7                       14) dmapp/6.0.1-1.0500.7263.9.31.ari
  2) eswrap/1.0.20-1.010200.643.0          15) pmi/4.0.1-1.0000.9753.86.2.ari
  3) switch/1.0-1.0500.41328.1.120.ari     16) ugni/5.0-1.0500.0.3.306.ari
  4) craype-network-aries                  17) udreg/2.3.2-1.0500.6756.2.10.ari
  5) PrgEnv-gnu/5.0.41                     18) cray-libsci/12.1.2
  6) atp/1.7.0                             19) gcc/4.8.2
  7) rca/1.0.0-2.0500.41336.1.120.ari      20) craype/2.01
  8) alps/5.0.3-2.0500.8095.1.1.ari        21) pbs/12.1.400.132424
  9) dvs/2.3_0.9.0-1.0500.1522.1.180       22) craype-ivybridge
 10) csa/3.0.0-1_2.0500.41366.1.129.ari    23) cray-mpich/6.1.1
 11) job/1.5.5-0.1_2.0500.41368.1.92.ari   24) packages-archer
 12) xpmem/0.1-2.0500.41356.1.11.ari       25) cmake/2.8.12
 13) gni-headers/3.0-1.0500.7161.11.4.ari

Configure and build the parallel, single-precision build

Create a build directory in the source tree

cd gromacs-4.6.5
mkdir build_mpi
cd build_mpi

Set the environment variables for the CMake build. (Note, for at least 4.6.3 and 4.6.5, FLAGS=-ffast-math results in errors and test failures.) Note the use of the compiler wrapper scripts below.

export CXX=CC
export CC=cc
export CMAKE_PREFIX_PATH=/home/y07/y07/cse/fftw/3.3.3-phase1-noavx-gcc
export FLAGS="-static -O3 -ftree-vectorize -funroll-loops"

Use CMake to configure the build and then build and install:

cmake ../ -DCMAKE_SKIP_RPATH=YES -DBUILD_SHARED_LIBS=OFF \
          -DGMX_MPI=ON -DGMX_OPENMP=ON -DGMX_GPU=OFF -DGMX_X11=OFF -DGMX_DOUBLE=OFF \
          -DCMAKE_C_FLAGS="$FLAGS" -DCMAKE_CXX_FLAGS="$FLAGS" \
          -DCMAKE_INSTALL_PREFIX=/usr/local/packages/gmx/4.6.5-phase1

make install-mdrun

Configure and build the parallel, double-precision build

Create a build directory in the source tree

cd gromacs-4.6.5
mkdir build_mpi_d
cd build_mpi_d

Set the environment variables for the CMake build. (Note, for at least 4.6.3 and 4.6.5, FLAGS=-ffast-math results in errors and test failures.) Note the use of the compiler wrapper scripts below.

export CXX=CC
export CC=cc
export CMAKE_PREFIX_PATH=/home/y07/y07/cse/fftw/3.3.3-phase1-noavx-gcc
export FLAGS="-static -O3 -ftree-vectorize -funroll-loops"

Use CMake to configure the build and then build and install:

cmake ../ -DCMAKE_SKIP_RPATH=YES -DBUILD_SHARED_LIBS=OFF \
          -DGMX_MPI=ON -DGMX_OPENMP=ON -DGMX_GPU=OFF -DGMX_X11=OFF -DGMX_DOUBLE=ON \
          -DCMAKE_C_FLAGS="$FLAGS" -DCMAKE_CXX_FLAGS="$FLAGS" \
          -DCMAKE_INSTALL_PREFIX=/usr/local/packages/gmx/4.6.5-phase1

make install-mdrun