14 February 2018

647. Linda/G16 and ecce + slurm

I had the opportunity to test G16 with Linda on my cluster. Suprisingly, I observed a decent reduction in computational time for geometry optimisation, and a fair reduction for vibrational analysis.

Computational time -- geometry optimisation
1x6 cores: 100%
2x6 cores: 56% (2 nodes)
3x6 cores: 44% (3 nodes)

Computational time -- normal mode computation
1x6 cores: 100%
2x6 cores: 63% (2 nodes)
3x6 cores: 57% (3 nodes)

In no case was the speed up good enough that it actually makes long-term sense to use Linda (need less than 50% for 2 nodes and 33% or less for 3 nodes), but if you're itching for quick results, it might be worth it.

Either way, as part of this I wrote a CONFIG.Linda for ECCE and SLURM:
Gaussian-16: /opt/gaussian/g16a/g16/g16 perlPath: /usr/bin/ qmgrPath: /usr/bin/ xappsPath: /usr/bin/ Slurm { #!/bin/csh #SBATCH -p linda #SBATCH --time=$walltime #SBATCH --output=slurm.out #SBATCH --job-name=$submitFile #SBATCH --nodes=$nodes #SBATCH --cpus-per-task=$ppn } Gaussian-16FilesToRemove{ core } Gaussian-16Command{ set path = ( /opt/nbo6/bin $path ) set nnodes = `scontrol show job $SLURM_JOBID|grep NodeList|sed 's/=/\t/'|gawk '{print $2}'|tail -n 1` sed -i "2s/^/%LindaWorkers=$nnodes\n/" g16.g16in sed -i '3s/^/%UseSSH\n/' g16.g16in sed -i "s/nprocshared.*/nprocshared=$ppn./" g16.g16in setenv GAUSS_SCRDIR /home/me/scratch setenv GAUSS_EXEDIR /opt/gaussian/g16a/g16 setenv GAUSS_ARCHDIR /opt/gaussian/g16a/g16/arch setenv GAUSS_BSDDIR /opt/gaussian/g16a/g16/bsd setenv GAUSS_LEXEDIR /opt/gaussian/g16a/g16/linda-exe /opt/gaussian/g16a/g16/g16< $infile > $outfile }
The slurm partition linda has all the nodes that has g16.

26 September 2017

646. NWChem 6.6 on debian stretch (9)

I've got plenty of posts on how to compile nwchem on this blog. However, as I'm still running debian jessie on my work computers and cluster I have failed to appreciate quite how different things work in debian stretch.

This includes the compilation of ECCE (although owing to updates meant to rectify that at https://github.com/FriendsofECCE/ECCE this isn't necessarily something you'd notice) and NWChem.

Anyway, here's how to build nwchem with openmpi support under debian stretch:


1. Download source code and patches
sudo mkdir /opt/nwchem -p sudo chown $USER:$USER /opt/nwchem cd /opt/nwchem mkdir patches6.6 cd patches6.6 for i in Tddft_mxvec20.patch.gz Tools_lib64.patch.gz Config_libs66.patch.gz Cosmo_meminit.patch.gz Sym_abelian.patch.gz Xccvs98.patch.gz Dplot_tolrho.patch.gz Driver_smalleig.patch.gz Ga_argv.patch.gz Raman_displ.patch.gz Ga_defs.patch.gz Zgesvd.patch.gz Cosmo_dftprint.patch.gz Txs_gcc6.patch.gz Gcc6_optfix.patch.gz Util_gnumakefile.patch.gz Util_getppn.patch.gz Gcc6_macs_optfix.patch.gz Xatom_vdw.patch.gz Notdir_fc.patch.gz Hfmke.patch.gz Cdfti2nw66.patch.gz ; do wget http://www.nwchem-sw.org/download.php?f=$i ; done gunzip *.gz cat *.patch > consolidated.patch cd .. wget http://www.nwchem-sw.org/download.php?f=Nwchem-6.6.revision27746-src.2015-10-20.tar.gz -O Nwchem-6.6.revision27746-src.2015-10-20.tar.gz tar xvf Nwchem-6.6.revision27746-src.2015-10-20.tar.gz

Apply patches:
cd /opt/nwchem/nwchem-6.6/ cp /opt/nwchem/patches6.6/consolidated.patch . patch -p0 < consolidated.patch
In addition, you can do the following to print more information so that GabEdit properly shows MOs:
cd nwchem-6.6/ grep -rl "do klo = 0, min(n-1,9), 2"|xargs -I {} sed -i 's/do klo = 0, min(n-1,9), 2/do klo = 0, min(n-1,199), 2/g' {} grep -rl " 0.15d0,"|grep -v "atomdata"|xargs -I {} sed -i 's/0.15d0/0.01d0/g' {}

2. Install dependencies and compile.
 How exactly it is done depends on your compiler and math libs. I've used red for examples with INTEL mkl, blue for AMD acml and orange for the free openblas. ifort is shown in purple and gfortran like this.You should only use one compiler + one math lib.
Hint: also make sure your ld.conf.so.d files contain paths to your math libs.
cd /opt/nwchem/nwchem-6.6 sudo apt-get install build-essential gfortran python2.7-dev libopenmpi-dev openmpi-bin export NWCHEM_TOP=`pwd` export LARGE_FILES=TRUE export TCGRSH=/usr/bin/ssh export NWCHEM_TOP=`pwd` export NWCHEM_TARGET=LINUX64 export NWCHEM_MODULES="all" export PYTHONVERSION=2.7 export PYTHONHOME=/usr export BLASOPT="-L/opt/intel/composer_xe_2013.4.183/mkl/lib/intel64/ -lmkl_core -lmkl_sequential -lmkl_intel_ilp64" export LIBRARY_PATH="$LIBRARY_PATH:/opt/intel/composer_xe_2013.4.183/mkl/lib/intel64/" export BLASOPT="-L/opt/acml/acml5.3.1/gfortran64_int64/lib -lacml" export LIBRARY_PATH="$LIBRARY_PATH:/opt/acml/acml5.3.1/gfortran64_int64/lib" export BLASOPT="-L/opt/openblas/lib -lopenblas" export LIBRARY_PATH="$LIBRARY_PATH:/opt/openblas/lib" export USE_MPI=y export USE_MPIF=y export USE_MPIF4=y export ARMCI_NETWORK=SOCKETS cd $NWCHEM_TOP/src make clean make nwchem_config make FC=ifort 1> make.log 2>make.err make FC=gfortran 1> make.log 2>make.err cd $NWCHEM_TOP/contrib export FC=ifort export FC=gfortran ./getmem.nwchem

3. Run
mpirun -n 8 /opt/nwchem-6.6/bin/LINUX64/nwchem input.nw > output.nw

23 August 2017

645. Moodle on Debian

I want to play around with moodle on my own desktop so I can experiment before making any major commits to a course website.

I first had a look at https://docs.moodle.org/33/en/Installing_Moodle_on_Debian_based_distributions, but, realising that moodle is in sid, decided to build my own .deb package.

This was done on debian jessie (8.9), which has at this point been overtaken by stretch as stable. I'm thus three releases behind sid.

1. Add the sid /src/ repo to sources.list
deb-src http://ftp.au.debian.org/debian/ sid main non-free contrib

2. Get source and build
I don't know exactly what dependencies you'll need. Error messages will tell you.
sudo apt-get update sudo apt-get install build-essential dpkg-dev mysql-server mysql-client php5-xmlrpc php5-intl sudo service mysql start mkdir ~/tmp/moodle -p cd ~/tmp/moodle sudo apt-get source moodle
Edit debian/control (necessary if you're on Jessie/8.9 -- newer debian versions may not need this):
14 Depends: ${misc:Depends}, 15 php5, php5-mysql | php5-pgsql, php5-gd, php5-curl, php5-cli, 16 apache2 | httpd, ucf, postgresql-client | default-mysql-client | virtual-mysql-client, 17 dbconfig-common, libphp-phpmailer, libphp-adodb, php-htmlpurifier, php-tcpdf, 18 php-cas (>= 1.3.3-1), libjs-jquery-migrate-1 19 Pre-Depends: debconf (>= 0.5) | debconf-2.0
Then continue in the terminal:
cd ~/tmp/moodle-2.7.19+dfsg sudo dpkg-buildpackage -us -uc cd .. sudo dpkg -i moodle_2.7.19+dfsg-2_all.deb sudo apt-get -f install sudo cp /etc/moodle/apache.conf /etc/apache2/sites-available/moodle.conf sudo ln -s /etc/apache2/sites-available/moodle.conf /etc/apache2/sites-enabled/moodle.conf sudo chmod o+r /etc/apache2/sites-available/moodle.conf sudo chmod go+w /var/lib/moodle sudo service apache2 reload
Navigate to http://localhost/moodle and follow the prompts