Compiling OpenFOAM 2.1.1 on ARCHER

This page outlines procedures for building OpenFOAM 2.1.1 on ARCHER. We have not built this version of OpenFOAM on ARCHER, the default module version (installed version on ARCHER) is OpenFOAM 2.2.2. However, we have supported users building this version of OpenFOAM for ARCHER and the instructions below are based on their experiences.

We use the Intel compiler for building this version of OpenFOAM on ARCHER.

It is necessary to modify a number of files. Firstly alter etc/bashrc. Set WM_MPLIN=MPICH2 and WM_COMPILER=icc. Also ensure that FOAM_INST_DIR and WM_PROJECT_INST_DIR point to the correct install directories.

Modify etc/config/settings.sh, replace 64) with the following:

        64)
            WM_ARCH=linux64
            export WM_COMPILER_LIB_ARCH=64
            export WM_CC='cc'
            export WM_CXX='CC'
            export WM_CFLAGS='-fPIC'
            export WM_CXXFLAGS='-fPIC'

And before MPICH) add the following:

MPICH2)
    export FOAM_MPI=mpich2
    export FOAM_MPI_LIBBIN=$FOAM_LIBBIN/mpich2
    export MPI_ARCH_PATH=$MPICH_DIR
    ;;

Edit the OpenFOAM file src/parallel/decompose/ptscotchDecomp/ptscotchDecomp.C, change:

extern "C"
{
#include <stdio.h>
#include "mpi.h"
#include "ptscotch.h"
}

to:

#include <mpi.h>

extern "C"
{
#include <stdio.h>
#include "ptscotch.h"
}

Next alter files in wmake/rules/linux64Icc, changing this line in file c to:

cc          = cc

Change this line in file c++ to:

CC          = CC -std=c++0x

Changed these lines in file c++Opt to:

c++DBUG     =
c++OPT      = -O2 -no-prec-div

Finally, ensure that the correct compilers are specified in ThirdParty-2.1.1/etc/wmakeFiles/scotch/Makefile.inc.i686_pc_linux2.shlib-OpenFOAM-64. It should be sufficient to ensure the following lines are as below:

....
AR              = cc
ARFLAGS         = -shared -o
....
CCP             = cc
CCD             = cc
....

Now OpenFOAM should be ready to compile. It can take a number of hours so it is best to build it using the serial nodes on ARCHER. You can use a batch script like the one below to do this:

#!/bin/bash --login
#
#PBS -l select=serial=true:ncpus=1
#PBS -l walltime=12:00:00
#PBS -A ##INSERT APPROPRIATE BUDGET CODE HERE

# Make sure any symbolic links are resolved to absolute path
export PBS_O_WORKDIR=$(readlink -f $PBS_O_WORKDIR)               

# Change to the directory that the job was submitted from
cd $PBS_O_WORKDIR

export WM_64=ON

export CRAYPE_LINK_TYPE=dynamic
module swap PrgEnv-cray PrgEnv-intel
module load zlib

cd ##INSERT OPENFOAM DIRECTORY HERE
source etc/bashrc

./Allwmake lib >& make.out

It is necessary to update the budget code and the directory in the above script. If you have any problems with this build, or there are any errors in the output of the OpenFOAM build (the above script will store the build output in make.out).