I willingly admit that I am no expert on this blas library stuff, so I may well be doing something obviously wrong.
1. Compiling and installing openblas
sudo mkdir /opt/openblas
sudo chown ${USER} /opt/openblas
mkdir ~/tmp
cd ~/tmp
download from here http://github.com/xianyi/OpenBLAS/tarball/v0.1.1
tar xvf xianyi-OpenBLAS-v0.1.1-0-g5b7f443.tar.gz
make all BINARY=64 CC=/usr/bin/gcc FC=/usr/bin/gfortran USE_THREAD=0 INTERFACE64=1 1> make.log 2>make.err
make PREFIX=/opt/openblas install
cp lib*.* /opt/openblas/lib
cd /opt/openblas/lib
rm libopenblas.so.0
ln -s libopenblas.so libopenblas.so.0
do ls /opt/openblas/lib and note what cpu specific file you have such as libopenblas_barcelona-r0.1.1.so or libopenblas_nehalem-r0.1.1.a
edit /etc/ld.so.conf and add
/opt/openblas/lib
2. Compiling nwchem
sudo apt-get install build-essential gfortran python2.7-dev
mkdir /opt/nwchem/
sudo chown ${USER} /opt/nwchem
cd /opt/nwchem
wget http://www.nwchem-sw.org/images/Nwchem-6.0.tar.gz
tar -xvf Nwchem-6.0.tar.gz
cd nwchem-6.0/
For python support, edit src/config/makefile.h and add -lz -lssl to line 1962 (see here: http://verahill.blogspot.com.au/2012/04/adding-python-support-to-nwchem-under.html)
Next, continue
export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export PYTHONHOME=/usr
export PYTHONVERSION=2.7
export USE_MPI=y
export USE_MPIF=y
export MPI_LOC=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export BLASOPT="-L/opt/openblas/lib -lopenblas -lopenblas_barcelona-r0.1.1"
export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openmpi/lib
export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"
cd $NWCHEM_TOP/src
make clean
make nwchem_config
make FC=gfortran
You might find that ecce still won't properly execute nwchem jobs (you don't get an error -- it just doesn't work) -- edit your /etc/ld.so.conf and add a line saying
/opt/openblas/lib
Time to look into benchmarking...
So far I've got the following benchmarks (in seconds) for nwchem when optimising benzene with 6-311++g** using mpirun -n 3.
without -lpthread
_____Internal blas_____openblas_____ATLAS_____acml_____
Ta 249.8 277.7 335.9 ------
Be 408.2 421.7 568.4
B 546.4 579.9 760.2
Noticing a mention regarding -lpthread online I compiled with that as part of the mpilibs, and lo and behold:
with -lpthread
_____Internal blas_____openblas_____ATLAS_____acml_____
Ta 201.7 198.9 233.9 ------
Be 371.1 314.4 390.3
B 496.6 416.8 521.9
* All run using mpirun -n 3. Be=phenom II X6. B= Athlon II X3. Ta= i7-1600 X4. Intel MKL cost money, acml are free.
This doesn't mean that one library is superior to the other -- it just means that under the conditions I employed and in the presence of any mistake I may have made, these are my observations. Your mileage will vary.
tar -xvf Nwchem-6.0.tar.gz
cd nwchem-6.0/
For python support, edit src/config/makefile.h and add -lz -lssl to line 1962 (see here: http://verahill.blogspot.com.au/2012/04/adding-python-support-to-nwchem-under.html)
Next, continue
export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export PYTHONHOME=/usr
export PYTHONVERSION=2.7
export USE_MPI=y
export USE_MPIF=y
export MPI_LOC=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export BLASOPT="-L/opt/openblas/lib -lopenblas -lopenblas_barcelona-r0.1.1"
export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openmpi/lib
export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"
cd $NWCHEM_TOP/src
make clean
make nwchem_config
make FC=gfortran
Now edit your ~/.bashrc and add
export PATH=$PATH:/opt/nwchem/nwchem-6.0/bin/LINUX64and you should be done.
export LD_LIBRARY_PATH=/opt/openblas/lib:$LD_LIBRARY_PATH
You might find that ecce still won't properly execute nwchem jobs (you don't get an error -- it just doesn't work) -- edit your /etc/ld.so.conf and add a line saying
/opt/openblas/lib
Then do sudo ldconfig and it will work
Time to look into benchmarking...
So far I've got the following benchmarks (in seconds) for nwchem when optimising benzene with 6-311++g** using mpirun -n 3.
without -lpthread
_____Internal blas_____openblas_____ATLAS_____acml_____
Ta 249.8 277.7 335.9 ------
Be 408.2 421.7 568.4
B 546.4 579.9 760.2
Noticing a mention regarding -lpthread online I compiled with that as part of the mpilibs, and lo and behold:
with -lpthread
_____Internal blas_____openblas_____ATLAS_____acml_____
Ta 201.7 198.9 233.9 ------
Be 371.1 314.4 390.3
B 496.6 416.8 521.9
* All run using mpirun -n 3. Be=phenom II X6. B= Athlon II X3. Ta= i7-1600 X4. Intel MKL cost money, acml are free.
This doesn't mean that one library is superior to the other -- it just means that under the conditions I employed and in the presence of any mistake I may have made, these are my observations. Your mileage will vary.
