104. Building gromacs with fftw3 and openmpi on ROCKS 5.4.3/CentOS

This guide was heavily modified on 13/03/2012 to remove the need for sudo/root privileges.

Not all flavours of linux are equal. I've always been a Debian man, but have recently become a user of a ROCKS based HPC cluster on a different continent. To make sure that I don't screw things up I'm currently trying to work out how to reliably compile common computational packages under ROCKS 5.4.3, which is CentOS based.

If you installed the bio roll from the beginning you'll have openmpi in /opt/openmpi (rocks_openmpi.x86_64 package), and fftw in /opt/rocks/lib and /opts/rocks/include (fftw.x86_64 package)

If you only installed the basic rolls, you won't have either. Now, you can either download the bio roll and install from there, or you can install the regular openmpi package and compile fftw yourself. In fact, you'll need to do the latter if you want double-precision gromacs anyway.

My goal is to avoid having to use sudo or root at all. I've rewritten this guide a couple of times, so there may be weird annoying errors that I've missed.

Installing openmpi:
If you don't have openmpi in /opt, then you can install it from the base roll
sudo yum install openmpi

---

fftw3:
You can skip this step IF
1. you have fftw files in /opt/rocks/lib and /opt/rocks/include
AND
2. you only want single precision

Otherwise:

wget http://www.fftw.org/fftw-3.3.1.tar.gz
tar -xvf fftw-3.3.1.tar.gz
cd fftw-3.3.1


Then use --prefix to tell make where to install the files:

Single precision fftw3 libraries:
make distclean
./configure --enable-float --enable-mpi --enable-threads --with-pic --prefix=/export/home/me/.fftwsingle
make
make install

Double-precision fftw3 libraries:
make distclean
./configure --disable-float --enable-mpi --enable-threads --with-pic --prefix=/export/home/me/.fftwdouble
make 
make install

---

gromacs:

First download and extract:

cd ~/tmp
wget ftp://ftp.gromacs.org/pub/gromacs/gromacs-4.5.5.tar.gz

tar -xvf gromacs-4.5.5.tar.gz
cd gromacs-4.5.5/

Before building you need to define where the openmpi libs are i.e.
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/openmpi/lib
OR
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib64/openmpi/1.4-gcc/lib

We now have three permutations of possible builds:
1. We use the single precision fftw libs in /opt/rocks/lib and /opt/rocks/include
export LDFLAGS=-L/opt/rocks/lib
export CPPFLAGS=-I/opt/rocks/include
./configure --enable-mpi --enable-float --with-fft=fftw3 --program-suffix=_spmpi --prefix=/export/home/me/gromacs
make
make install

2. We use the single precision fftw libs in /export/home/me/.fftwsingle

export LDFLAGS=-L/export/home/me/.fftwsingle/lib
export CPPFLAGS=-I/export/home/me/.fftwsingle/include
./configure --enable-mpi --enable-float --with-fft=fftw3 --program-suffix=_spmpi --prefix=/export/home/me/gromacs
make
make install

3. We use the double precision fftw libs in /export/home/me/.fftwdouble

export LDFLAGS=-L/export/home/me/.fftwdouble/lib
export CPPFLAGS=-I/export/home/me/.fftwdouble/include
./configure --enable-mpi --disable-float --with-fft=fftw3 --program-suffix=_ddmpi --prefix=/export/home/me/gromacs
make
make install

---

Running

You will now have single and double-precision binaries, e.g.
grompp_spmpi and grompp_ddmpi

Make sure that you define/have defined LD_LIBRARY_PATH in /etc/profile or ~/.bashrc and included the paths to your mpi libs and your fftw libs, e.g.:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/openmpi/lib:/export/home/me/.fftwsingle:/export/home/me/.fftwdouble

Actually, it doesn't seem necessary to include the fftw path.

You may also want to include your gromacs bins in your path:
export PATH=$PATH:/export/home/me/gromacs/bin

Dynamic load-balancing seems to be disabled by default, so to use multiple cores run using e.g.
mpirun -n 4 mdrun_spmpi -s inp.tpr -o out.trr etc.

DONE

---

---

Troubleshooting

---

Error:

checking size of off_t... configure: error: in `/export/home/me/tmp/gromacs-4.5.5':
configure: error: cannot compute sizeof (off_t)
See `config.log' for more details

config.log:

./conftest: error while loading shared libraries: libmpi.so.0: cannot open shared object file: No such file or directory

Solution:
Set LD_LIBRARY_PATH to your openmpi libs e.g.
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/openmpi/lib

Error:

/usr/local/lib/libfftw3f.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
make[3]: *** [libmd.la] Error 1

Solution:
Compile fftw3 using the --with-pic switch:
./configure --enable-float --enable-mpi --enable-threads --with-pic 

---

---