140. MD = ECCE + NWChem: 1. Prepare

Update 19 June 2013:
A much better written and complete guide to getting started with MD in ECCE+NWChem is found here: http://saccharides.blogspot.tw/2013/06/ecce-md-calculation.html

Original post:
Here's a multi-part description of how to set up a minimal MD simulation using ECCE/NWChem. To make things real easy we'll do an example with a fully described and parametrised system.

Part 2:
http://verahill.blogspot.com.au/2012/05/md-ecce-nwchem-2-md-optimize.html

Part 3:
http://verahill.blogspot.com.au/2012/05/md-ecce-nwchem-3-energy.html

Part 4:
http://verahill.blogspot.com.au/2012/05/md-ecce-nwchem-4dynamics.html


How to install and configure ecce is described elsewhere on this blog.

Start here
Start Ecce, select Organizer

 Rightclick in the organiser and create a new MD Study

 Rightclick on the new project, and create a MD Prepare

 This is what you should have now. Double click on the Prepare icon, then click on the Editor icon

 You get this menu. Click on the Builder icon in the bottom left corner.

Click on the Import from Structure Library icon (hidden behind the dropped down menu in the screenshot). Go to RNA bases, monomers and select Cytidine. Also, check the Atom table and Coordinates options in the Tools menu.

 Hit ctrl+s to save, and close the window.

Back in the organiser set the size of the box for the system

 And click on solvate. You'll see what commands are being added to your input files.

 You might not be able to launch at this point, with ecce complaining about being unable to find the pdb. For me, opening the builder and clicking on Center in the Coordinates box on the right (make it show by checking the right box in Tools menu in the Builder)

If all goes well you'll be able to launch your job, which will finish very fast.

 And here's what we have at this point.

Part 2 will do the next step -- MD Optimize

139. compiling nwchem with custom ATLAS on debian

I'm not yet sure, but there seems to be something weird with this build. Use the openblas build here preferentially:  http://verahill.blogspot.com.au/2012/05/nwchem-with-openblas.html

i.e. do not use this build without testing it against builds without external blas/atlas and against other libraries.

I am not an expert

The atlas build is pretty much like what was described here:
http://verahill.blogspot.com.au/2012/05/compile-atlas-blas-on-debian-testing.html

---

Build atlas
Check you cpu governor:

cat /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor 

ondemand

Next, set it to performance for all cores (example for three-core AMD)
sudo cpufreq-set -g performance
sudo cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
sudo cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor

Prepare the install directory:
sudo mkdir /opt/ATLAS
sudo chown ${USER} /opt/ATLAS

Prepare the compile directory
mkdir ~/tmp
cd ~/tmp
 wget http://downloads.sourceforge.net/project/math-atlas/Developer%20%28unstable%29/3.9.72/atlas3.9.72.tar.bz2
tar xvf atlas3.9.72.tar.bz2
cd ATLAS/
mkdir build/
vim Make.top 
change -V on line 6 to -v

cd build
.././configure --prefix=/opt/ATLAS

There's any level of detail in what you can pass to configure. -A AMD64 would be one option. Look at the bottom for more info.

make
make install
cp lib/lib* /opt/ATLAS/lib

Once it's all done, set the cpu governor back if you so desire.
sudo cpufreq-set -g ondemand
sudo cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
sudo cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor

---

nwchem

For python support you need to edit src/config/makefile.h (see http://verahill.blogspot.com.au/2012/04/adding-python-support-to-nwchem-under.html)

export LARGE_FILES=TRUE

export TCGRSH=/usr/bin/ssh

export NWCHEM_TOP=/opt/nwchem/nwchem-6.0

export NWCHEM_TARGET=LINUX64

export NWCHEM_MODULES="all python"

export PYTHONHOME=/usr

export PYTHONVERSION=2.7

export BLASOPT="-L/opt/ATLAS/lib -latlas -lblas -llapack"

export USE_MPI=y

export USE_MPIF=y

export MPI_LOC=/usr/lib/openmpi/lib

export MPI_INCLUDE=/usr/lib/openmpi/include

export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openmpi/lib

export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"

cd $NWCHEM_TOP/src

make clean

make nwchem_config

make FC=gfortran

Note: don't use HAS_BLAS=y. It gave me no end of grief.

Also, IF you binary executes without a hitch at this point it means that you're libs are not being loaded.  You need to put

export LD_LIBRARY_PATH=/opt/ATLAS/lib:$LD_LIBRARY_PATH

in your ~/.bashrc

Building takes a while, but should go ok. Make sure you have python2.7-dev installed, and if you're using a different python version, make sure to change the options above.

---

Here's also an example ~/.nwchemrc file for no particular reason at all -- if you've got ecce installed you might want to make sure yours is correct.

nwchem_basis_library /opt/nwchem/nwchem-6.0/src/basis/libraries/
nwchem_nwpw_library /opt/nwchem/nwchem-6.0/src/nwps/libraryps/
ffield amber
amber_1 /opt/nwchem/nwchem-6.0/src/data/amber_s/
amber_2 /opt/nwchem/nwchem-6.0/src/data/amber_x/
spce /opt/nwchem/nwchem-6.0/src/data/solvents/spce.rst
charmm_s /opt/nwchem/nwchem-6.0/src/data/charmm_s/
charmm_x /opt/nwchem/nwchem-6.0/src/data/charmm_x/

---

You can get a list over compile options for ATLAS using
make xprint_enums ; ./xprint_enums

after you've done .././configure

See http://sourceforge.net/tracker/index.php?func=detail&aid=3021404&group_id=23725&atid=379483

138. Compile ATLAS blas on debian testing

I first tried the latest stable and gave up due to "ERROR 639 DURING CACHE EDGE DETECTION!!" errors. Hence, I used unstable.

Start here

mkdir /opt/ATLAS
chown ${USER}  /opt/ATLAS

mkdir ~/tmp
cd ~/tmp
 wget http://downloads.sourceforge.net/project/math-atlas/Developer%20%28unstable%29/3.9.72/atlas3.9.72.tar.bz2

tar xvf atlas3.9.72.tar.bz2

cd ATLAS/


Edit ATLAS/Make.top 
change the V on line 6 to lowercase i.e. from

- $(ICC) -V 2>&1  >> bin/INSTALL_LOG/ERROR.LOG

to

- $(ICC) -v 2>&1  >> bin/INSTALL_LOG/ERROR.LOG

mkdir build/
cd build/
sudo apt-get install cpufrequtils

Since I don't like computers overheating I normally throttle my cpu. To unset throttling on a AMD64 (phenom II) six core system, before compile:
/usr/bin/cpufreq-selector -g performance
sudo cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor

sudo cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor

sudo cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor

sudo cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu4/cpufreq/scaling_governor

sudo cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu5/cpufreq/scaling_governor

Alternatively, you can disable throttle checking using the -Si cputhrchk 0 configure switch. If you don't, and throttling is on, configure will fail.

OK, continuing with the build, in your build/ directory:

.././configure --prefix=/opt/ATLAS -A AMD64

Before you continue, do
make xprint_enums ; ./xprint_enums
and study the output -- you might find a better ARCH fit than e.g. AMD64.  If so, run the configure command above again. There is a lot of other options which you can configure as well. Obviously, nothing prevents you from experimenting and re-compiling if something goes wrong. You can also run without any -A switch at all.

Continuing:
make
make install
cp lib/lib* /opt/ATLAS/lib

Your libs should be in /opt/ATLAS/lib -- when you compiled something to include the new libs, use
"-L/opt/ATLAS/lib"