You are here

OpenFOAM, OpenFOAM+ and OpenFOAM Extend

Introduction

OpenFOAM is a toolkit for creation of computational fluid dynamics (CFD) applications.  It includes solver libraries and pre- and post-processing utilities.

Variants of OpenFOAM from the OpenFOAM+ and Extend projects are also available on some systems. See the Software Versions page.

Like other jobs on WestGrid systems, OpenFOAM jobs are run by submitting an appropriate script for batch scheduling using the qsub command. See documentation on running batch jobs for more information.

Brief instructions for running the pre-built OpenFOAM applications on various WestGrid systems are shown below.  See the Software Versions page for a list of the specific versions installed.

Running OpenFOAM, OpenFOAM+ and OpenFOAM Extend on Bugaboo

To set up the environment for using OpenFOAM on Bugaboo, load the openfoam module:

module load openfoam

To set up the environment for using foam-extend, use

module load openfoam/foam-extend

To set up the environment for OpenFOAM+, use

module load openfoam/1606

Running OpenFOAM on Hermes/Nestor

To set up the environment for using OpenFOAM on Hermes/Nestor, load the openfoam module for the default version:

module load openfoam

Alternatively you can use module avail to find the names of version-specific modules, such as:

module load openfoam/2.1.1

Running OpenFOAM on Jasper

To set up the environment for using OpenFOAM on Jasper, use:

module load application/OpenFOAM/2.1.1
source $OPENFOAM_SETUP

Running OpenFOAM and OpenFOAM Extend on Lattice and Parallel

Several versions of OpenFOAM and OpenFOAM Extend are available on Lattice and Parallel, but, it is recommended that you use the latest version in each case.

OpenFOAM 4.1

OpenFOAM 4.1 from http://openfoam.org/ is available on Lattice and Parallel.  It was built with the parameters WM_LABEL_SIZE=64 WM_COMPILER_TYPE=ThirdParty FOAMY_HEX_MESH=yes, where the ThirdParty compiler refers to GCC 4.8.5. The accompanying ParaView 5.0.1 is available on Parallel when run with vglrun as described at on the WestGrid remote visualization page, but, no attempt has been made to get the paraFoam script to work.

Here is a sample script that was used to test OpenFOAM with one of the supplied tutorial cases (damBreakFine), which used interFoam, modified to use the scotch decomposition option.

#!/bin/bash
#PBS -S /bin/bash

# 2016-11-29 DSP - Example batch job script for a parallel OpenFOAM run on Parallel or Lattice

cd $PBS_O_WORKDIR

# Set up the environment

BUILD_OPTIONS="WM_LABEL_SIZE=64 WM_COMPILER_TYPE=ThirdParty FOAMY_HEX_MESH=yes"
export FOAM_INST_DIR=/global/software/openfoam
source $FOAM_INST_DIR/OpenFOAM-4.1/etc/bashrc ${BUILD_OPTIONS}

export FOAM_RUN=$PBS_O_WORKDIR

# If using a custom application, define a directory similar to this:
#export FOAM_USER_APPBIN=/global/scratch/phillips/software/openfoam/bin

echo "Working in $PWD"

echo "Node file: $PBS_NODEFILE :"
echo "---------------------"
cat $PBS_NODEFILE
echo "---------------------"

CORES=`/bin/awk 'END {print NR}' $PBS_NODEFILE`
echo "Running on $CORES cores."

echo "Make a new decomposeParDict file"
DATE=`date`

cat > system/decomposeParDict <<EOF
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    location    "system";
    object      decomposeParDict;
}

// decomposeParDict created at ${DATE}.

numberOfSubdomains $CORES;

method          scotch;

EOF

echo "Forcing new decomposition"

decomposePar -force

echo "Using mpirun: `which mpirun`"
FOAM=`which interFoam`

echo "About to run $FOAM at `date`"
mpiexec -n $CORES interFoam -parallel > interFoam_${CORES}cores_${PBS_JOBID}.out
echo "Finished interFoam at `date`"

echo "Running reconstructPar at `date`."
reconstructPar
echo "Finished reconstructPar at `date`."
echo "Manually delete processor directories if reconstruction succeeded. "

 

OpenFOAM 2.3.0

OpenFOAM 2.3.0 was built on Lattice and Parallel without CGAL or ParaView support.

#!/bin/bash
#PBS -S /bin/bash

# 2014-09-26 DSP - Example batch job script for a parallel OpenFOAM run on Parallel or Lattice

cd $PBS_O_WORKDIR

# Set up the environment

export FOAM_INST_DIR=/global/software/openfoam
source $FOAM_INST_DIR/OpenFOAM-2.3.0/etc/bashrc

export FOAM_RUN=$PBS_O_WORKDIR

# If using a custom application, define a directory similar to this:
#export FOAM_USER_APPBIN=/global/scratch/phillips/software/openfoam/bin

echo "Working in $PWD"

echo "Node file: $PBS_NODEFILE :"
echo "---------------------"
cat $PBS_NODEFILE
echo "---------------------"

CORES=`/bin/awk 'END {print NR}' $PBS_NODEFILE`
echo "Running on $CORES cores."

echo "Make a new decomposeParDict file"
cat > system/decomposeParDict <<EOF
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    location    "system";
    object      decomposeParDict;
}

numberOfSubdomains $CORES;

method          scotch;

EOF

echo "Forcing new decomposition"

decomposePar -force

echo "Using mpirun: `which mpirun`"
FOAM=`which rhoCentralFoam`
echo "About to run $FOAM at `date`"
#mpiexec rhoCentralFoam -parallel  > rhoCentralFoam_${PBS_JOBID}.out
mpirun --hostfile $PBS_NODEFILE -np $CORES rhoCentralFoam -parallel > rhoCentralFoam_${PBS_JOBID}.out
echo "Finished at `date`"

 

OpenFOAM Extend 3.1

OpenFOAM Extend 3.1 was built on Lattice and Parallel without CGAL or ParaView support. A batch job script similar to the one shown for OpenFOAM 2.3.0 should be appropriate, except replacing the environment setup lines with

export FOAM_INST_DIR=/global/software/openfoam
source $FOAM_INST_DIR/foam-extend-3.1_20140710/etc/bashrc

OpenFOAM 2.2.1 (not recommended)

Unless you have a particularly good reason for using OpenFOAM 2.2.1, please use a more recent version instead. OpenFOAM 2.2.1 was built on Lattice and Parallel in July 2013, before a system upgrade that occurred in October 2013.  This build requires use of an Open MPI version older than the current default on those systems.  So, the example job script given here includes a line

module load openmpi/old

Here is an example script.

#!/bin/bash
#PBS -S /bin/bash

# 2013-08-02 DSP - Example batch job script for a parallel OpenFOAM run on Parallel or Lattice
# 2014-01-06 DSP - Added "load module openmpi/old", required until OpenFOAM is recompiled.

cd $PBS_O_WORKDIR

# Set up the environment

export FOAM_INST_DIR=/global/software/openfoam
export foamDotFile=$FOAM_INST_DIR/OpenFOAM-2.2.1/etc/bashrc
[ -f $foamDotFile ] && . $foamDotFile
module load intel/12

# Running an application compiled before the October 2013 system
# upgrade on Lattice and Parallel requires using an old version of OpenMPI
module load openmpi/old

export FOAM_RUN=$PBS_O_WORKDIR

# If using a custom application, define a directory similar to this:
#export FOAM_USER_APPBIN=/global/scratch/phillips/software/openfoam/bin

echo "Working in $PWD"

echo "Node file: $PBS_NODEFILE :"
echo "---------------------"
cat $PBS_NODEFILE
echo "---------------------"

CORES=`/bin/awk 'END {print NR}' $PBS_NODEFILE`
echo "Running on $CORES cores."

echo "Make a new decomposeParDict file"
cat > system/decomposeParDict <<EOF
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    location    "system";
    object      decomposeParDict;
}

numberOfSubdomains $CORES;

method          scotch;

EOF

echo "Forcing new decomposition"

decomposePar -force

echo "Using mpirun: `which mpirun`"
FOAM=`which rhoCentralFoam`
echo "About to run $FOAM at `date`"
#mpiexec rhoCentralFoam -parallel  > sky_parallel_${PBS_JOBID}.out
mpirun --hostfile $PBS_NODEFILE -np $CORES rhoCentralFoam -parallel > sky_parallel_${PBS_JOBID}.out
echo "Finished at `date`"

 

Running OpenFOAM on Orcinus

To set up the environment for using OpenFOAM on Orcinus, load the openfoam module:

module load openfoam

Note, as of this writing (2014-12-22), module show openfoam cautions that the installation is not ready for production and does not include the ParaView visualization support.

Updates:
2014-01-06 - Added Lattice/Parallel batch job script.
2014-09-26 - Added Lattice/Parallel batch job script for OpenFOAM 2.3.0.
2014-11-01 - Added OpenFOAM Extend Project references.
2014-12-22 - Added Orcinus section.
2016-09-26 - Added OpenFOAM+ references.
2016-11-29 - Added Lattice/Parallel batch job script for OpenFOAM 4.1.