647. Linda/G16 and ecce + slurm

I had the opportunity to test G16 with Linda on my cluster. Suprisingly, I observed a decent reduction in computational time for geometry optimisation, and a fair reduction for vibrational analysis.

Computational time -- geometry optimisation
1x6 cores: 100%
2x6 cores: 56% (2 nodes)
3x6 cores: 44% (3 nodes)

Computational time -- normal mode computation
1x6 cores: 100%
2x6 cores: 63% (2 nodes)
3x6 cores: 57% (3 nodes)

In no case was the speed up good enough that it actually makes long-term sense to use Linda (need less than 50% for 2 nodes and 33% or less for 3 nodes), but if you're itching for quick results, it might be worth it.

Either way, as part of this I wrote a CONFIG.Linda for ECCE and SLURM:

Gaussian-16: /opt/gaussian/g16a/g16/g16
perlPath: /usr/bin/
qmgrPath: /usr/bin/
xappsPath: /usr/bin/

Slurm {
#!/bin/csh
#SBATCH -p linda
#SBATCH --time=$walltime
#SBATCH --output=slurm.out
#SBATCH --job-name=$submitFile
#SBATCH --nodes=$nodes
#SBATCH --cpus-per-task=$ppn
}

Gaussian-16FilesToRemove{ core }

Gaussian-16Command{
set path = ( /opt/nbo6/bin $path ) 

set nnodes = `scontrol show job $SLURM_JOBID|grep NodeList|sed 's/=/\t/'|gawk '{print $2}'|tail -n 1`
sed -i "2s/^/%LindaWorkers=$nnodes\n/" g16.g16in
sed -i '3s/^/%UseSSH\n/' g16.g16in
sed -i "s/nprocshared.*/nprocshared=$ppn./" g16.g16in

setenv GAUSS_SCRDIR /home/me/scratch
setenv GAUSS_EXEDIR /opt/gaussian/g16a/g16
setenv GAUSS_ARCHDIR /opt/gaussian/g16a/g16/arch
setenv GAUSS_BSDDIR /opt/gaussian/g16a/g16/bsd
setenv GAUSS_LEXEDIR /opt/gaussian/g16a/g16/linda-exe
/opt/gaussian/g16a/g16/g16< $infile > $outfile 
}

The slurm partition linda has all the nodes that has g16.

646. NWChem 6.6 on debian stretch (9)

I've got plenty of posts on how to compile nwchem on this blog. However, as I'm still running debian jessie on my work computers and cluster I have failed to appreciate quite how different things work in debian stretch.

This includes the compilation of ECCE (although owing to updates meant to rectify that at https://github.com/FriendsofECCE/ECCE this isn't necessarily something you'd notice) and NWChem.

Anyway, here's how to build nwchem with openmpi support under debian stretch:

---

1. Download source code and patches

sudo mkdir /opt/nwchem -p
sudo chown $USER:$USER /opt/nwchem
cd /opt/nwchem
mkdir patches6.6
cd patches6.6
for i in Tddft_mxvec20.patch.gz Tools_lib64.patch.gz Config_libs66.patch.gz Cosmo_meminit.patch.gz Sym_abelian.patch.gz Xccvs98.patch.gz Dplot_tolrho.patch.gz Driver_smalleig.patch.gz Ga_argv.patch.gz Raman_displ.patch.gz Ga_defs.patch.gz Zgesvd.patch.gz Cosmo_dftprint.patch.gz Txs_gcc6.patch.gz Gcc6_optfix.patch.gz Util_gnumakefile.patch.gz Util_getppn.patch.gz Gcc6_macs_optfix.patch.gz Xatom_vdw.patch.gz Notdir_fc.patch.gz Hfmke.patch.gz Cdfti2nw66.patch.gz ; do wget http://www.nwchem-sw.org/download.php?f=$i ; done
gunzip *.gz
cat *.patch > consolidated.patch
cd ..
wget http://www.nwchem-sw.org/download.php?f=Nwchem-6.6.revision27746-src.2015-10-20.tar.gz -O Nwchem-6.6.revision27746-src.2015-10-20.tar.gz
tar xvf Nwchem-6.6.revision27746-src.2015-10-20.tar.gz

Apply patches:

cd /opt/nwchem/nwchem-6.6/
cp /opt/nwchem/patches6.6/consolidated.patch .
patch -p0 < consolidated.patch

In addition, you can do the following to print more information so that GabEdit properly shows MOs:

cd nwchem-6.6/
grep -rl "do klo = 0, min(n-1,9), 2"|xargs -I {} sed -i 's/do klo = 0, min(n-1,9), 2/do klo = 0, min(n-1,199), 2/g' {}
grep -rl " 0.15d0,"|grep -v "atomdata"|xargs -I {} sed -i 's/0.15d0/0.01d0/g' {}

---

2. Install dependencies and compile.
 How exactly it is done depends on your compiler and math libs. I've used red for examples with INTEL mkl, blue for AMD acml and orange for the free openblas. ifort is shown in purple and gfortran like this.You should only use one compiler + one math lib.
Hint: also make sure your ld.conf.so.d files contain paths to your math libs.

cd /opt/nwchem/nwchem-6.6
sudo apt-get install build-essential gfortran python2.7-dev libopenmpi-dev openmpi-bin

export NWCHEM_TOP=`pwd`
export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all"
export PYTHONVERSION=2.7
export PYTHONHOME=/usr

export BLASOPT="-L/opt/intel/composer_xe_2013.4.183/mkl/lib/intel64/ -lmkl_core -lmkl_sequential -lmkl_intel_ilp64"
export LIBRARY_PATH="$LIBRARY_PATH:/opt/intel/composer_xe_2013.4.183/mkl/lib/intel64/"

export BLASOPT="-L/opt/acml/acml5.3.1/gfortran64_int64/lib -lacml"
export LIBRARY_PATH="$LIBRARY_PATH:/opt/acml/acml5.3.1/gfortran64_int64/lib"

export BLASOPT="-L/opt/openblas/lib -lopenblas"
export LIBRARY_PATH="$LIBRARY_PATH:/opt/openblas/lib"


export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export ARMCI_NETWORK=SOCKETS

cd $NWCHEM_TOP/src

make clean
make nwchem_config
make FC=ifort 1> make.log 2>make.err
make FC=gfortran 1> make.log 2>make.err

cd $NWCHEM_TOP/contrib
export FC=ifort
export FC=gfortran
./getmem.nwchem

---

3. Run
mpirun -n 8 /opt/nwchem-6.6/bin/LINUX64/nwchem input.nw > output.nw

645. Moodle on Debian

I want to play around with moodle on my own desktop so I can experiment before making any major commits to a course website.

I first had a look at https://docs.moodle.org/33/en/Installing_Moodle_on_Debian_based_distributions, but, realising that moodle is in sid, decided to build my own .deb package.

This was done on debian jessie (8.9), which has at this point been overtaken by stretch as stable. I'm thus three releases behind sid.

1. Add the sid /src/ repo to sources.list
deb-src http://ftp.au.debian.org/debian/ sid main non-free contrib

2. Get source and build
I don't know exactly what dependencies you'll need. Error messages will tell you.

sudo apt-get update
sudo apt-get install build-essential dpkg-dev mysql-server mysql-client php5-xmlrpc php5-intl
sudo service mysql start
mkdir ~/tmp/moodle -p
cd ~/tmp/moodle
sudo apt-get source moodle

Edit debian/control (necessary if you're on Jessie/8.9 -- newer debian versions may not need this):

14 Depends: ${misc:Depends},
 15  php5, php5-mysql | php5-pgsql, php5-gd, php5-curl, php5-cli,
 16  apache2 | httpd, ucf, postgresql-client | default-mysql-client | virtual-mysql-client,
 17  dbconfig-common, libphp-phpmailer, libphp-adodb, php-htmlpurifier, php-tcpdf,
 18  php-cas (>= 1.3.3-1), libjs-jquery-migrate-1
 19 Pre-Depends: debconf (>= 0.5) | debconf-2.0

Then continue in the terminal:

cd ~/tmp/moodle-2.7.19+dfsg
sudo dpkg-buildpackage -us -uc
cd ..
sudo dpkg -i moodle_2.7.19+dfsg-2_all.deb
sudo apt-get -f install
sudo cp /etc/moodle/apache.conf /etc/apache2/sites-available/moodle.conf
sudo ln -s /etc/apache2/sites-available/moodle.conf /etc/apache2/sites-enabled/moodle.conf
sudo chmod o+r /etc/apache2/sites-available/moodle.conf
sudo chmod go+w /var/lib/moodle
sudo service apache2 reload

Navigate to http://localhost/moodle and follow the prompts