12 June 2013

449. Nwchem 6.3 -- updated sources. Compiling on Debian

The previous post uses the sources from the 17th of May.


What's new?
Maybe it's just me, but I can't find any obvious location where they list the differences between the 28th of June and the 17th of June releases. In fact, I wouldn't have known that there was a new version out unless someone had specifically pointed it out to me.

Anyway, luckily there's diff. There were more changes than the ones I'm showing below, but most of them were minor ones such as .stamp files and a re-done pair of QA test output files.
  1 Only in nwchem-6.3-src.2013-05-28/QA/tests: rodft-cam
  2 diff -r nwchem-6.3-src.2013-05-17/src/nwdft/scf_dft/dft_canorg.F nwchem-6.3-src.2013-05-28/src/nwdft/scf_dft/dft_canorg.F
  3 6c6
  4 < c     $Id: dft_canorg.F 23846 2013-03-19 04:08:25Z edo $
  5 ---
  6 > c     $Id: dft_canorg.F 24271 2013-05-24 06:48:33Z niri $
  7 279c279,280
  8 <          if(iter.ge.nfock/2)iswitc = iswitc+1
  9 ---
 10 >          if(iter.ge.nfock/2) iswitc = iswitc+1
 11 >          if(abs(delta).lt.1d-6) iswitc = iswitc+2
 12 diff -r nwchem-6.3-src.2013-05-17/src/nwdft/scf_dft/dft_scf.F nwchem-6.3-src.2013-05-28/src/nwdft/scf_dft/dft_scf.F
 13 9c9
 14 < c     $Id: dft_scf.F 23988 2013-04-08 23:06:52Z d3y133 $
 15 ---
 16 > c     $Id: dft_scf.F 24269 2013-05-24 00:55:11Z edo $
 17 819c819
 18 <          iswitc = 1
 19 ---
 20 >          iswitc = 2
 21 diff -r nwchem-6.3-src.2013-05-17/src/nwdft/scf_dft_cg/dft_roks_fock.F nwchem-6.3-src.2013-05-28/src/nwdft/scf_dft_cg/dft_roks_fock.F
 22 6c6
 23 < * $Id: dft_roks_fock.F 23999 2013-04-10 18:23:02Z d3y133 $
 24 ---
 25 > * $Id: dft_roks_fock.F 24274 2013-05-24 07:34:16Z niri $
 26 102c102
 27 <       double precision edisp    ! [input] dispersion corrrection
 28 ---
 29 >       double precision Edisp    ! [input] dispersion correction
 30 107c107
 31 <       double precision errmax, ebq
 32 ---
 33 >       double precision errmax, Ebq
 34 124c124
 35 <       integer g_tmp(2)
 36 ---
 37 >       integer g_tmp(nset)
 38 326a327,335
 39 >         call ga_zero(g_tmp(1))
 40 >         if (nopen.gt.0) then
 41 >           g_tmp(2) = ga_create_atom_blocked(geom, basis,
 42 >      $                                      'dft_roks_fock: tmp2')
 43 >           g_tmp(3) = ga_create_atom_blocked(geom, basis,
 44 >      $                                      'dft_roks_fock: tmp3')
 45 >           call ga_zero(g_tmp(2))
 46 >           call ga_zero(g_tmp(3))
 47 >         endif
 48 330d338
 49 <         call ga_zero(g_tmp(1))
 50 339c347
 51 <      $     tol2e, oskel, iv_dens, g_tmp(1), .false., .false.)
 52 ---
 53 >      $     tol2e, oskel, iv_dens, g_tmp, .false., .false.)
 54 340a349,352
 55 >         if (nopen.gt.0) then
 56 >           call ga_dadd(1d0,iv_fock(2),1d0,g_tmp(2),iv_fock(2))
 57 >           call ga_dadd(1d0,iv_fock(3),1d0,g_tmp(3),iv_fock(3))
 58 >         endif
 59 344a357,360
 60 >         if (nopen.gt.0) then
 61 >           call ga_zero(g_tmp(2))
 62 >           call ga_zero(g_tmp(3))
 63 >         endif
 64 353c369
 65 <      $     tol2e, oskel, iv_dens, g_tmp(1), .false., .true.)
 66 ---
 67 >      $     tol2e, oskel, iv_dens, g_tmp, .false., .true.)
 68 354a371,374
 69 >         if (nopen.gt.0) then
 70 >           call ga_dadd(1d0,iv_fock(2),1d0,g_tmp(2),iv_fock(2))
 71 >           call ga_dadd(1d0,iv_fock(3),1d0,g_tmp(3),iv_fock(3))
 72 >         endif
 73 358a379,385
 74 >         if (nopen.gt.0) then
 75 >           if (.not. ga_destroy(g_tmp(2))) call errquit
 76 >      $               ('xc_getv: ga corrupt?',0, GA_ERR)
 77 >           if (.not. ga_destroy(g_tmp(3))) call errquit
 78 >      $               ('xc_getv: ga corrupt?',0, GA_ERR)
 79 >         endif
 80 > c
 81 383c410
 82 <       etwo = etwo_closed + etwo_open + edisp
 83 ---
 84 >       etwo = etwo_closed + etwo_open + Edisp
 85 diff -r nwchem-6.3-src.2013-05-17/src/tools/GNUmakefile nwchem-6.3-src.2013-05-28/src/tools/GNUmakefile
 86 342a343,346
 87 > ifeq ($(ARMCI_NETWORK),SOCKETS)
 88 >         MAYBE_ARMCI = --with-sockets
 89 > endif # SOCKETS
 90 >
 91 Only in nwchem-6.3-src.2013-05-28/src/util: util_ga_version.F
 92 diff -r nwchem-6.3-src.2013-05-17/src/util/util_nwchem_version.F nwchem-6.3-src.2013-05-28/src/util/util_nwchem_version.F
 93 4c4
 94 <       nwrev="24252"
 95 ---
 96 >       nwrev="24277"

The section in red is the Iswtch.patch in http://verahill.blogspot.com.au/2013/05/424-nwchem-63-on-debian-wheezy.html.

I'm guessing that this patch is also included: http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id840/rodft_and_range_separated-functi....html
I've indicated it in blue.

Purple is probably to make sure that SOCKETS gets implemented correctly -- there were issues with that before.

I f I were to guess I'd say that if you compiled nwchem as shown here: http://verahill.blogspot.com.au/2013/05/424-nwchem-63-on-debian-wheezy.html
AND if you are not going to use CAM with open-shell molecules, then you are not in a hurry to recompile.

The former issue is manifested in slow run times, while the second issue is manifested in crashing calculations, so neither should be able to fly under the radar.

Anyway

The GabEdit Patch
The following patch allows you to use GabEdit as a GUI to nwchem, and allows you to compile nwchem with python support.
Copy the following and paste it into a file, e.g. 6.3.patch, and put it into /opt/nwchem/nwchem-6.3-src.2013-05-28:

diff -rupN src.original/config/makefile.h src/config/makefile.h
--- src.original/config/makefile.h      2013-04-15 12:41:45.016853322 +1000
+++ src/config/makefile.h       2013-04-15 12:38:44.933319544 +1000
@@ -2039,7 +2039,7 @@ endif
 
      ifeq ($(BUILDING_PYTHON),python)
 #   EXTRA_LIBS += -ltk -ltcl -L/usr/X11R6/lib -lX11 -ldl
-     EXTRA_LIBS +=    -lnwcutil  -lpthread -lutil -ldl
+     EXTRA_LIBS +=    -lnwcutil  -lpthread -lutil -ldl -lssl -lz
   LDOPTIONS = -Wl,--export-dynamic 
      endif
 ifeq ($(NWCHEM_TARGET),CATAMOUNT)
diff -rupN src.original/ddscf/movecs_pr_anal.F src/ddscf/movecs_pr_anal.F
--- src.original/ddscf/movecs_pr_anal.F 2013-04-15 12:41:45.036852381 +1000
+++ src/ddscf/movecs_pr_anal.F  2013-04-15 12:23:28.100409225 +1000
@@ -195,7 +195,7 @@ c
  22         format(1x,2('  Bfn.  Coefficient  Atom+Function  ',5x))
             write(LuOut,23)
  23         format(1x,2(' ----- ------------  ---------------',5x))
-            do klo = 0, min(n-1,9), 2
+            do klo = 0, min(n-1,199), 2
                khi = min(klo+1,n-1)
                write(LuOut,2) (
      $              int_mb(k_list+k)+1, 
diff -rupN src.original/ddscf/rohf.F src/ddscf/rohf.F
--- src.original/ddscf/rohf.F   2013-04-15 12:41:45.036852381 +1000
+++ src/ddscf/rohf.F    2013-04-15 12:23:28.100409225 +1000
@@ -153,7 +153,7 @@ c
             ilo = 1
             ihi = nmo
          endif
-         call movecs_print_anal(basis, ilo, ihi, 0.15d0, g_movecs, 
+         call movecs_print_anal(basis, ilo, ihi, 0.01d0, g_movecs, 
      $        'ROHF Final Molecular Orbital Analysis', 
      $        .true., dbl_mb(k_eval), oadapt, int_mb(k_irs),
      $        .true., dbl_mb(k_occ))
diff -rupN src.original/ddscf/scf_vec_guess.F src/ddscf/scf_vec_guess.F
--- src.original/ddscf/scf_vec_guess.F  2013-04-15 12:41:45.036852381 +1000
+++ src/ddscf/scf_vec_guess.F   2013-04-15 12:23:28.100409225 +1000
@@ -511,19 +511,19 @@ c
          nprint = min(nclosed+nopen+30,nmo)
          if (scftype.eq.'RHF' .or. scftype.eq.'ROHF') then
             call movecs_print_anal(basis, 1,
-     &           nprint, 0.15d0, g_movecs, 
+     &           nprint, 0.01d0, g_movecs, 
      &           'ROHF Initial Molecular Orbital Analysis', 
      &           .true., dbl_mb(k_eval), oadapt, int_mb(k_irs),
      &           .true., dbl_mb(k_occ))
          else
             nprint = min(nalpha+20,nmo)
             call movecs_print_anal(basis, max(1,nbeta-20),
-     &           nprint, 0.15d0, g_movecs, 
+     &           nprint, 0.01d0, g_movecs, 
      &           'UHF Initial Alpha Molecular Orbital Analysis', 
      &           .true., dbl_mb(k_eval), oadapt, int_mb(k_irs),
      &           .true., dbl_mb(k_occ))
             call movecs_print_anal(basis, max(1,nbeta-20),
-     &           nprint, 0.15d0, g_movecs(2), 
+     &           nprint, 0.01d0, g_movecs(2), 
      &           'UHF Initial Beta Molecular Orbital Analysis', 
      &           .true., dbl_mb(k_eval+nbf), oadapt, int_mb(k_irs+nmo),
      &           .true., dbl_mb(k_occ+nbf))
diff -rupN src.original/ddscf/uhf.F src/ddscf/uhf.F
--- src.original/ddscf/uhf.F    2013-04-15 12:41:45.036852381 +1000
+++ src/ddscf/uhf.F     2013-04-15 12:23:28.096409414 +1000
@@ -144,11 +144,11 @@ C
          enddo
          ihi = max(ihi-1,1)
  9611    continue
-         call movecs_print_anal(basis, ilo, ihi, 0.15d0, g_movecs, 
+         call movecs_print_anal(basis, ilo, ihi, 0.01d0, g_movecs, 
      $        'UHF Final Alpha Molecular Orbital Analysis', 
      $        .true., dbl_mb(k_eval), oadapt, int_mb(k_irs),
      $        .true., dbl_mb(k_occ))
-         call movecs_print_anal(basis, ilo, ihi, 0.15d0, g_movecs(2), 
+         call movecs_print_anal(basis, ilo, ihi, 0.01d0, g_movecs(2), 
      $        'UHF Final Beta Molecular Orbital Analysis', 
      $        .true., dbl_mb(k_eval+nbf), oadapt, int_mb(k_irs+nmo),
      $        .true., dbl_mb(k_occ+nbf))
diff -rupN src.original/mcscf/mcscf.F src/mcscf/mcscf.F
--- src.original/mcscf/mcscf.F  2013-04-15 12:41:45.000854073 +1000
+++ src/mcscf/mcscf.F   2013-04-15 12:23:23.748613695 +1000
@@ -719,7 +719,7 @@ c
       if (util_print('final vectors analysis', print_default))
      $     call movecs_print_anal(basis, 
      $     max(1,nclosed-10), min(nbf,nclosed+nact+10),
-     $     0.15d0, g_movecs, 'Analysis of MCSCF natural orbitals',
+     $     0.01d0, g_movecs, 'Analysis of MCSCF natural orbitals',
      $     .true., dbl_mb(k_evals), .true., int_mb(k_sym), 
      $     .true., dbl_mb(k_occ))
 c     
diff -rupN src.original/nwdft/scf_dft/dft_mxspin_ovlp.F src/nwdft/scf_dft/dft_mxspin_ovlp.F
--- src.original/nwdft/scf_dft/dft_mxspin_ovlp.F        2013-04-15 12:41:45.604825677 +1000
+++ src/nwdft/scf_dft/dft_mxspin_ovlp.F 2013-04-15 12:23:28.228403211 +1000
@@ -184,14 +184,14 @@ c
       call ga_sync()
 c
       call movecs_print_anal(basis,int_mb(k_non),int_mb(k_non)
-     & ,0.15d0,g_alpha,'Alpha Orbitals without Beta Partners',
+     & ,0.01d0,g_alpha,'Alpha Orbitals without Beta Partners',
      &   .false., 0.0 ,.false., 0 , .false., 0 )
 c
       if (nct.GE.2) then
       do i = 2,nct
       ind = int_mb(k_non+i-1)
       call movecs_print_anal(basis,ind,ind
-     & ,0.15d0,g_alpha,' ',
+     & ,0.01d0,g_alpha,' ',
      &   .false., 0.0 ,.false., 0 , .false., 0 )
       enddo
       endif
@@ -350,7 +350,7 @@ c      endif
 c      endif
 c 9990 format(/,18x,'THERE ARE',i3,1x,'UN-PARTNERED ALPHA ORBITALS')
 c
-       call movecs_print_anal(basis, 1, nalp, 0.15d0, g_ualpha,
+       call movecs_print_anal(basis, 1, nalp, 0.01d0, g_ualpha,
      & 'Alpha Orb. w/o Beta Partners (after maxim. alpha/beta overlap)',
      &   .false., 0.0 ,.false., 0 , .false., 0 )
 c
diff -rupN src.original/nwdft/scf_dft/dft_scf.F src/nwdft/scf_dft/dft_scf.F
--- src.original/nwdft/scf_dft/dft_scf.F        2013-04-15 12:41:45.608825490 +1000
+++ src/nwdft/scf_dft/dft_scf.F 2013-04-15 12:23:28.228403211 +1000
@@ -1774,7 +1774,7 @@ c
             else
                blob='DFT Final Beta Molecular Orbital Analysis' 
             endif
-            call movecs_print_anal(ao_bas_han, ilo, ihi, 0.15d0, 
+            call movecs_print_anal(ao_bas_han, ilo, ihi, 0.01d0, 
      &           g_movecs(ispin), 
      &           blob, 
      &           .true., dbl_mb(k_eval(ispin)), oadapt, 
diff -rupN src.original/nwdft/scf_dft_cg/dft_cg_solve.F src/nwdft/scf_dft_cg/dft_cg_solve.F
--- src.original/nwdft/scf_dft_cg/dft_cg_solve.F        2013-04-15 12:41:45.612825303 +1000
+++ src/nwdft/scf_dft_cg/dft_cg_solve.F 2013-04-15 12:23:28.220403588 +1000
@@ -183,7 +183,7 @@ c
             blob = 'DFT Final Beta Molecular Orbital Analysis'
           endif
           call movecs_fix_phase(g_movecs(ispin))
-          call movecs_print_anal(basis, ilo, ihi, 0.15d0,
+          call movecs_print_anal(basis, ilo, ihi, 0.01d0,
      &         g_movecs(ispin),blob,
      &         .true., dbl_mb(k_eval+(ispin-1)*nbf),
      &         oadapt, int_mb(k_irs+(ispin-1)*nbf),




Compiling nwchem
First install ACML or OpenBlas:
http://verahill.blogspot.com.au/2013/05/423-openblas-on-debian-wheezy.html
http://verahill.blogspot.com.au/2013/05/422-set-up-acml-on-linux.html


sudo apt-get install build-essential gfortran python2.7-dev libopenmpi-dev openmpi-bin
sudo mkdir /opt/nwchem -p
sudo chown $USER:$USER /opt/nwchem
cd /opt/nwchem
wget http://www.nwchem-sw.org/download.php?f=Nwchem-6.3.revision1-src.2013-05-28.tar.gz -O Nwchem-6.3.revision1-src.2013-05-28.tar.gz
tar xvf Nwchem-6.3.revision1-src.2013-05-28.tar.gz
cd nwchem-6.3-src.2013-05-28/

Apply the Gabedit patch:
patch -p0 < 6.3.patch 
patching file src/config/makefile.h patching file src/ddscf/movecs_pr_anal.F patching file src/ddscf/rohf.F patching file src/ddscf/scf_vec_guess.F patching file src/ddscf/uhf.F patching file src/mcscf/mcscf.F patching file src/nwdft/scf_dft/dft_mxspin_ovlp.F patching file src/nwdft/scf_dft/dft_scf.F patching file src/nwdft/scf_dft_cg/dft_cg_solve.F

export NWCHEM_TOP=`pwd`
export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export PYTHONVERSION=2.7
export PYTHONHOME=/usr
export BLASOPT="-L/opt/acml/acml5.3.1/gfortran64_int64/lib -lacml"
#export BLASOPT="-L/opt/openblas/lib -lopenblas"

export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export LIBRARY_PATH="$LIBRARY_PATH:/usr/lib/openmpi/lib:/opt/acml/acml5.3.1/gfortran64_int64/lib"
#export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openmpi/lib:/opt/openblas/lib

export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"
export ARMCI_NETWORK=SOCKETS

cd $NWCHEM_TOP/src

make clean
make nwchem_config
make FC=gfortran 1> make.log 2>make.err

cd $NWCHEM_TOP/contrib
export FC=gfortran
./getmem.nwchem

Comment out the bold parts and uncomment the commented parts to compile with openblas instead of ACML e.g. if you're compiling on an intel machine instead of AMD.

Setting up your ~/.bashrc
As usual, do
echo 'export NWCHEM_EXECUTABLE=/opt/nwchem/nwchem-6.3-src.2013-05-28/bin/LINUX64/nwchem' >> ~/.bashrc
echo 'export NWCHEM_BASIS_LIBRARY=/opt/nwchem/nwchem-6.3-src.2013-05-28/src/basis/libraries/' >> ~/.bashrc
echo 'export PATH=$PATH:/opt/nwchem/nwchem-6.3-src.2013-05-28/bin/LINUX64' >> ~/.bashrc

Also, put a .nwchemrc in your home folder:
nwchem_basis_library /opt/nwchem/nwchem-6.3-src.2013-05-28/src/basis/libraries/
ffield amber
amber_1 /opt/nwchem/nwchem-6.3-src.2013-05-28/src/data/amber_s/
amber_2 /opt/nwchem/nwchem-6.3-src.2013-05-28/src/data/amber_x/
amber_3 /opt/nwchem/nwchem-6.3-src.2013-05-28/src/data/amber_q/
amber_4 /opt/nwchem/nwchem-6.3-src.2013-05-28/src/data/amber_u/
amber_5 /opt/nwchem/nwchem-6.3-src.2013-05-28/src/data/custom/
spce /opt/nwchem/nwchem-6.3-src.2013-05-28/src/data/solvents/spce.rst
charmm_s /opt/nwchem/nwchem-6.3-src.2013-05-28/src/data/charmm_s/
charmm_x /opt/nwchem/nwchem-6.3-src.2013-05-28/src/data/charmm_x/

You're done.

448. Metal-pi bonds in ECCE

It should be easy, since it's one of the implemented bond types in ECCE together with single, double, triple and pi-arene bonds, but it doesn't function quite as advertised (supposed to ignore nubs).

I think it might be a bug, but given the very few resources currently given to ECCE, it's better to develop work-arounds than to demand quick bug fixes.

A reasonable criticism of this post would be that it really doesn't matter in the long run though, as the bonds that we make here are eye-candy only -- DFT doesn't 'know' what bonds are in the sense of the ball-and-stick model. On the other hand, ECCE will complain loudly if there's a discrepancy between oxidation states, multiplicity and number of vacant sites.

In the end, if you have a crystal structure to start from, use that. If not, then this is a fairly simple way of yielding a visually pleasing and reasonably intuitive model.

Anyway, let's use Noyori's catalyst as an example:


Start ECCE, set up a new job and click on Builder.

Click on Import from Structure Library, select Alicycles, and click on Cyclooctane, then click in the main window.


Delete one hydrogen on each of the sp2 carbons, and change the bond type to double bond:

 Change the sp2 carbons to TrigonalPlanar, and Clean (click the broom which does MM (UFF) geometry optimization).

Insert an octahedral iridium atom -- you'll need two 'nubs' for each pi bond, and two nubs for the two phosphine ligands.

 Change the sp2 carbons back to tetrahedral so you get 'nubs' to bond to (not supposed to be necessary according to the manual, but it is). Make Metal-pi bonds.
 Make sure all the bonds (check the first one in particular) are Metal-Pi bonds, and Clean the structure (i.e. MM/UFF optimization)


 Insert two tetrahedral phosphorous atoms, and make single bonds
Click on Add H if you want PH3 ligands, and clean the structure (i.e. optimize geo with MM/UFF):


At this point the structure isn't quite perfectly square planar like we would like it to be. The easiest way to sort that out is to set the torsional angles between the H-C(=)-Ir-P to 0 (uncheck rotate group around bond) for each P, then Clean. The result isn't perfect, but still not too shabby:

447. Multiuser ECCE

A recent comment lead me to make this post:
[..] Can you advise a correct way of setting up ECCE so that everyone can have an individual account and run on the same machine?

It's a valid question. I haven't had any need for it since I'm running my own cluster, which I currently don't share with anyone. I also manage a multi-user cluster overseas, but I am the only one using ECCE, and so I'm fine running the ECCE server here.

Anyway. I built ECCE 6.4 as shown e.g. here and here.

Below I install the ECCE server as the user verahill, and then install the client software as the users lindqvst and me. In the former case I configure the server using a copied config_from_server script, and in the latter case I use config_with_server. Both methods worked fine. All users are on the same machine, but by editing apps/siteconfig/DataServers you can quite easily configure the ECCE client to connect to a remote server.


Server:

hostname
helium
sudo adduser verahill sudo adduser lindqvst ./install_ecce.v6.4.csh
Main ECCE installation menu =========================== 1) Help on main menu options 2) Prerequisite software check 3) Full install 4) Full upgrade 5) Application software install 6) Application software upgrade 7) Server install 8) Server upgrade IMPORTANT: If you are uncertain about any aspect of installing or running ECCE at your site, please refer to the detailed ECCE Installation and Administration Guide at http://ecce.pnl.gov/docs/installation/2864B-Installation.pdf Hit at prompts to accept the default value in brackets. Selection: [1] 7 Host name: [helium] Server installation directory: [/home/verahill/ecce-v6.4/server] Enter the path to the ECCE application software directory that will use this server (even if this directory is not accessible from the current machine) or [return] if you have not installed application software yet or don't want to update it: ECCE v6.4 will be installed using the settings: Installation type: [server install] Host name: [helium] Server installation directory: [/home/verahill/ecce-v6.4/server] Are these choices correct (yes/no/quit)? [yes] ECCE installation succeeded. *************************************************************** !! You MUST perform the following steps in order to use ECCE !! -- Transfer the script: /home/verahill/ecce-v6.4/server/ecce-admin/config_from_server to the machine where the application software is installed. Run this script as the same user as the application software installation. (This step can be skipped if you have not installed the application software and the server name is specified during that install) -- Start the ECCE server as 'verahill' by running: /home/verahill/ecce-v6.4/server/ecce-admin/start_ecce_server ***************************************************************
cp /home/verahill/ecce-v6.4/server/ecce-admin/config_from_server ~ chmod ugo+r config_from_server


First client
su lindqvst
cd ~
cp /home/verahill/config_from_server .
cp /home/verahill/install_ecce.v6.4.csh .
./install_ecce.v6.4.csh
Main ECCE installation menu =========================== 1) Help on main menu options 2) Prerequisite software check 3) Full install 4) Full upgrade 5) Application software install 6) Application software upgrade 7) Server install 8) Server upgrade IMPORTANT: If you are uncertain about any aspect of installing or running ECCE at your site, please refer to the detailed ECCE Installation and Administration Guide at http://ecce.pnl.gov/docs/installation/2864B-Installation.pdf Hit at prompts to accept the default value in brackets. Selection: [1] 5 Host name: [helium] Server installation directory: [/home/lindqvst/ecce-v6.4/apps] ECCE installation succeeded. *************************************************************** !! You MUST perform the following steps in order to use ECCE !! -- Configure the application software to use the desired ECCE server by running the script: /home/lindqvst/ecce-v6.4/apps/scripts/config_with_server as the same user as the application software installation. (This step can be skipped if you prefer to copy over and run the config_from_server script created during the server installation in the ecce-admin directory) -- To register machines to run computational codes, please see the installation and compute resource registration manuals at http://ecce.pnl.gov/using/installguide.shtml -- Before running ECCE each user must source an environment setup script. For csh/tcsh users add this to ~/.cshrc: if ( -e /home/lindqvst/ecce-v6.4/apps/scripts/runtime_setup ) then source /home/lindqvst/ecce-v6.4/apps/scripts/runtime_setup endif For sh/bash users, add this to ~/.profile or ~/.bashrc: if [ -e /home/lindqvst/ecce-v6.4/apps/scripts/runtime_setup.sh ]; then . /home/lindqvst/ecce-v6.4/apps/scripts/runtime_setup.sh fi ***************************************************************
Next, configure:
./config_from_server
Enter the ECCE application software installation home directory: /home/lindqvst/ecce-v6.4/apps ECCE application software home directory is /home/lindqvst/ecce-v6.4/apps Is this correct? [yes] Adding data server URL for helium to siteconfig/DataServers Adding URL for online help to siteconfig/site_runtime Adding message server URL for helium to siteconfig/jndi.properti
And then do
echo 'export ECCE_HOME=/home/lindqvst/ecce-v6.4/apps' >> ~/.bashrc
echo 'export PATH=${ECCE_HOME}/scripts:${ECCE_HOME}/scripts/parsers:${PATH}' >> ~/.bashrc


Second client:
I set up a third user, installed ECCE as shown above for lindqvst, but instead of running config_from_server
I did:
/home/me/tmp/ecce-v6.4/ecce-v6.4/apps/scripts/config_with_server
Enter the full host name of the machine where the ECCE server is installed: helium ECCE server host name is helium Is this correct? [yes]
And then do
echo 'export ECCE_HOME=/home/me/ecce-v6.4/apps' >> ~/.bashrc
echo 'export PATH=${ECCE_HOME}/scripts:${ECCE_HOME}/scripts/parsers:${PATH}' >> ~/.bashrc
source ~/.bashrc


Tying it all together:

Start the server: As user verahill, start the server.
su verahill
/home/verahill/ecce-v6.4/server/ecce-admin/start_ecce_server
/home/verahill/ecce-v6.4/server/httpd/bin/apachectl start: httpd started [1] 13951 INFO BrokerService - ActiveMQ 5.1.0 JMS Message Broker (localhost) is starting INFO BrokerService - ActiveMQ JMS Message Broker (localhost, ID:helium-39358-1370944321599-0:0) started
ps a|grep 13951
13951 pts/3 S 0:00 grep -v -e ACTIVEMQ -e Loading -e AMQ -e Kaha -e help -e Transport
You should be able to autostart ecce using something along the lines of the following in your /etc/rc.local:
su verahill '/home/verahill/ecce-v6.4/server/ecce-admin/start_ecce_server' &
As a user, start ecce:
su lindqvst
/home/lindqvst/ecce-v6.4/apps/scripts/ecce
Create a password

Create a job
Then log out and log in as another user, e.g. me

su me
ecce
me is in reality called something else
Note that as user me you can't see any of the files in lindqvst's folder -- nor access them without inputting the correct (ECCE) username and password.

And again, note that there are alternative ways of setting this up -- you could have everyone log in as the same linux user, but still retain different ECCE identities. I think what I show here is more in line with how most people would want to use ECCE though.

Also, note that you can quite easily be a client on a different computer too -- the key lies in editing the apps/siteconfig/DataServers file.

11 June 2013

446. B3LYP and WAH -- the confusion

Quite a while back I was looking at the WAH (Wilson-Amos-Handy) functional, and while it turned out to be a bit more complicated than I had hoped, it led me to type up a brief discussion about b3lyp in different computational packages.

The issue is that there are several different definitions, and that even if a paper is kind enough to provide the Becke 1993 communication as a reference, this is rarely the actual form of b3lyp used. In fact, the LYP part would speak directly against it. As someone whom isn't well-versed in the computational and theoretical arts I do think it would be nice if we could get to the point where we can get useful information by doing point-and-click computations, but as exchange-correlation functionals essentially are fudge-factors, this probably won't happen for some time.  In other words, 6-31G/B3LYP may be a winning combination for the computation of electronic energies of a limited range of (mostly organic) small molecules in the gas phase, it doesn't always yield anything useful about real-world systems.

NOTE: I wrote the original text as I was trying to figure out what WAH was -- and I thought at that point that it was a simple form of B3PW91 with tweaked prefactors. It isn't -- it's requires changes in the way the GIAOs are computed (I think). So don't focus on the WAH discussion.

Anyway, here's the story as a bench chemist (i.e. not a computational or theoretical chemist) understands it, in the context of trying to understand what the WAH exchange-correlation functional looks like.

The WAH functional is a hybrid exchange correlation functional which was developed to provide accurate NMR shift calculations.

Note that the WAH functional is correctly implemented in PQS -- see the manual.

But the story is really about B3LYP...


1. Definition of the WAH exchange-correlation functional:
The definition[1] consists of
We found that by using hybrid Kohn-Sham orbitals and eigenvalues with an adjusted 'exact-exchange' coefficient Cx, the NMR shielding parameters gave an accuracy approaching the best coupled-cluster calculations for molecules containing first and second row atoms. For B3LYP[2] we find Cx=0.05[..] give(s) best values (note that the coefficient of Local Density Exchange is (1-Cx) in the amended B3LYP. We name the resulting NMR values B3LYPGGA0.05[..].
This is the paper which is cited by the PQS manual. The definition is brief, but not unreasonable.

2 Definition of B3LYP
[The first-person account of the background to B3LYP is found here: http://www.ccl.net/chemistry/resources/messages/2002/05/22.008-dir/]

In the paper which is cited as a source of B3LYP, Becke defined[2] a functional as
EXC=EXCLSDA+a0 (ExHF-ExLSDA )+axΔ ExBecke88+acΔ EcPW91   (eq 1)
where EXC denotes exchange-correlation functional, Ex denotes exchange functional, Ec denotes correlation functional, Δ denotes non-local contribution, LSDA is the local spin-density approximation, Becke88[3] is the gradient-corrected LDA and PW91 is the Perdew-Wang 1991 gradient correction.[4]. The B3 in B3LYP refers to the three parameters it involves: a0=0.2, ax=0.72 and ac=0.81. Note that a0 is the same as Cx above. We'll refer to equation 1 as B3PW91.

LSDA is poorly defined but is normally taken to be the SVWN of the form
EXCLSDA=EXLSDA+ECLSDA  (eq 2)
=ExSlater+EcVWN   (eq 3)
although there's a slew of Vosko-Wilk-Nusair (VWN) functionals -- most sources suggest that Becke referred to VWN5, while my reading of the literature is a bit different (Becke states he uses the electron-gas parametrization in [4]). Either way, equation 1 now becomes
EXC=a0 ExHF+(1-a0 )ExSlater+EcVWN +axΔ ExBecke88+acΔ EcPW91 (eq 4)
This is implemented as the hybrid exchange-correlation functional acm in NWChem (the adibatic connection method) using VWN5. A very brief summary of Becke '93 vs Gaussian '92 (don't ask me about the chronology) is also available by Stephens et al. in J. Phys. Chem. 1994, 98(45), p. 11624.

2.1 Gaussian '92
You may at this point be forgiven for asking yourself why it is called B3LYP and not B3PW91. In 1991 Gaussian hadn't yet implemented PW91 (fair enough) and substituted it with the Lee-Yang-Parr (LYP) correlation functional (ΔEcLYP). Since it's difficult to separate the local component (and we want the non-local component as indicated by Δ), they wrote
Δ EcLYP=EcLYP-EcVWN (eq 5)
which turns equation 4 into
EXC=a0 ExHF+(1-a0)ExSlater+axΔ ExBecke88+acEcLYP +(1-ac)EcVWN (eq 6)
In addition, in the original Gaussian implementation VWN_1_RPA was used, which sources tell me is 100\% wrong when taking Becke's intentions into account.
EXC=a0 ExHF+(1-a0)ExSlater +axΔ ExBecke88+acEcLYP +(1-ac)EcVWN_1_RPA (eq 7)
To make matters worse, today Gaussian uses VWN_3 and it seems they know how to get the nonlocal component of LYP directly (see below). Equation 7 is what you use if you use B3LYP in most software packages (though not all -- e.g. Gamess US uses VWN5 instead of VWN_1_RPA) So in G09 it's now
EXC=a0 ExHF+(1-a0)ExSlater+axΔ ExBecke88+EcVWN_3+acEcLYP (eq 8)

2.2 PQS
PQS uses the old gaussian version (eq. 7) as the b3lyp functional, but it doesn't explicitly state which form -- b3lyp or b3pw91 -- is used for the WAH functional.


3 So what definition did WAH use?
All we really care about is reproducing the original paper by Handy et al. -- not whether Becke would approve or not. But here's where the problem of citing papers you may not have read becomes an issue.

Wilson, Amos and Handy cite the 1993 paper by Becke which defines the canonical version of B3LYP (i.e. B3PW91), which should settle it in favour of WAH being defined as shown in equation 4.

However, they used CADPACK, which implements it as in equation 7. Reading the CADPACK manual I can see what Handy et al. probably did: the way you define custom parameters for hybrid functionals in CADPACK is by doing
hybrid a0 ax ac
so that they during the development of their functional most likely typed in
b3lyp 0.05 0.72 0.81
which meant they probably used
EXC=0.05 ExHF+0.95 ExSlater+ +0.72 Δ ExBecke88+0.81 EcLYP +0.19 EcVWN_1_RPA (eq 9)

4 Implementing it in your package of choice
[NOTE: this will NOT set up WAH correctly -- I'm leaving it as it shows how to set up custom XCs in nwchem, G09 and Dalton]

4.1 NWCHEM
The canonical version of Becke's functional, B3PW91, is implemented as acm in NWChem and which is manually entered as
xc HFexch 0.2 slater 0.8 becke88 nonlocal 0.72 vwn_5 1 Perdew91 0.81
while the Gaussian '92 form is manually entered as
xc HFexch 0.2 slater 0.8 becke88 nonlocal 0.72 vwn_1_rpa 0.19 lyp 0.81
This means that the two possible forms of WAH are:
xc HFexch 0.05 slater 0.95 becke88 nonlocal 0.72 vwn_5 1 Perdew91 0.81
and
 xc HFexch 0.05 slater 0.95 becke88 nonlocal 0.72 vwn_1_rpa 0.19 lyp 0.81
We'll refer to them as B3PW910.05 and B3LYP0.05, respectively.

4.2 Gaussian 09
Gaussian is a lot less elegant. The canonical version of Becke's functional, B3PW91, is implemented as acm in Gaussian as B3PW91, which is manually entered as
BPW91 IOp(3/76=1000002000) IOp(3/77=0720008000) IOp(3/78=0810010000)
while the old Gaussian form is manually entered as
BLYP IOp(3/76=1000002000) IOp(3/77=0720008000) IOp(3/78=0810001900
This means that the two forms of WAH are:
BPW91 IOp(3/76=1000000500) IOp(3/77=0720009500) IOp(3/78=0810010000)
and
BLYP IOp(3/76=1000000500) IOp(3/77=0720009500) IOp(3/78=0810001900)
We'll refer to them as B3PW910.05 and B3LYP0.05, respectively.

4.3 Dalton
The notations are (more or less)
Combine HF=0.20 Slater=0.80 Becke=0.72 PW91c=0.81 VWN5=1
Combine HF=0.20 Slater=0.80 Becke=0.72 LYP=0.81 VWN=0.19
Combine HF=0.05 Slater=0.95 Becke=0.72 PW91c=0.81 VWN5=1
Combine HF=0.05 Slater=0.95 Becke=0.72 LYP=0.81 VWN=0.19
As an aside, I don't think anyone at this point would be surprised to learn that B3PW91 in Dalton and Gaussian are two completely different exchange-correlation functionals...

Performance
NOTE: the results don't quite make any sense to me anymore -- using the same XCs and basis sets and structures one would expect to get the exact same results for all three packages. I don't know why that wasn't the case, assuming that I implemented the XCs correctly, and assuming that the basis sets really are the same (which often they actually aren't...). So keep that in mind.

Original:
I used the same equilibrium structures as Handy (i.e. those of Cybulski), and used the aug-cc-pVTZ basis set with a fine DFT grid (not the same as Handy -- but should be as good. Also, I've done this with def2-qzvp and 6-31+g* as well). All calculations are for the gas phase. Handy's values are for the Huzinaga IV basis set in their GIAO paper[5], but it doesn't really matter much which basis set is chosen. The results are tabulated in table 1.

Note that Handy also saw chemical shifts for the oxygen in carbon monoxide which were around -80 ppm. Basically, I can reproduce everything except for his B3LYP0.05gga. I did a few test-runs with Huz-IV in Dalton and it was just as bad as the other methods.

Table 1: Tabulated calculated gas phase NMR shifts using different combinations of exchange-correlation functionals and the aug-cc-pvtz basis set. Note that Handy's experimental data are not in agreement with my sources, which are 1.0, -42.3 and 344 ppm for CO, CO and H2O, respectively.
Package b3lypa gb3lypb acmc b3pw91b b3lyp0.05 b3pw910.05 Handy
Expt.
Carbon in CO (ppm)
NWChem -10.32 -10.32 -8.80 -8.80 -8.08 -6.55 5.6 2.8
G09 -7.71 -7.71 -9.95 -9.95 -5.66 -7.74
Dalton -10.21 -10.21 N/A -12.65 -7.97 -10.38
Oxygen in CO (ppm)
NWChem -71.51 -71.51 -72.34 -72.34 -70.06 -70.93 -40 -36.7
G09 -78.82 -78.82 -77.89 -77.89 -77.57 -76.86
Dalton -71.50 -71.50 N/A -72.03 -70.04 -70.89
Oxygen in water
NWChem 328.45 328.45 329.28 329.28 329.79 330.48 327 358
G09 N/A 328.76 328.39 328.39 328.95 328.31
Dalton 328.50 328.50 N/A 328.02 329.85 329.07
aNote that G09 uses eq. 8 by default (gives -11.70, -77.63, and 328.34 ppm), but I forced it to use eq. 7. b These were my manual implementations of 'b3lyp' and 'acm' to make sure I got things right. c This is acm in NWChem and B3PW91 in Gaussian 09.


References:
[1] P. J. Wilson, R. D. Amos, H. N. C., Chem. Phys. Letters 1999, 321, 475-484.
[2] A. D. Becke, J. Chem. Phys. 1993, 98, 5648-5652.
[3] A. D. Becke, Phys. Rev. A 1988, 38, 3098-3100.
[4] J. P. Perdew, Y. Wang, Phys. Rev. B 1991, 45, 13244-13249.
[5] T. Helgaker, P. J. Wilson, R. D. Amos, N. C. Handy, J. Chem. Phys. 2000, 113, 2983-2989.

445. GAMESS US 2013 R1 on Debian (Wheezy) -- w/o GPU

Update 27/6/2013:
Please note that Kirill Berezovsky has published a series of posts on GAMESS US, including how to compile it for both CPU and GPU use. See
http://biochemicalmatters.blogspot.com.au/2013/06/gamess-us-frequently-asked-questions_26.html
http://biochemicalmatters.blogspot.ru/2013/06/gamess-us-frequently-asked-questions_1687.html
http://biochemicalmatters.blogspot.ru/2013/06/gamess-us-frequently-asked-questions_1447.html
http://biochemicalmatters.blogspot.com.au/2013/06/gamess-us-frequently-asked-questions.html


Original post:
A new version of GAMESS is out now (2013 R1): https://groups.google.com/forum/?fromgroups#!topic/gamess-announce/8j1esKifzEo

GPU support will be a later post.

0. Install a math library
You can use e.g. acml or atlas. See http://verahill.blogspot.com.au/2013/05/422-set-up-acml-on-linux.html for acml (which I've only had luck with on AMD machines).

To get the debian ATLAS libs do
sudo apt-get install libatlas3-base libatlas-dev

If you want to compile your own ATLAS libs, see e.g. http://verahill.blogspot.com.au/2012/09/rocks-543-atlas-and-gromacs-on-xeon.html or http://verahill.blogspot.com.au/2012/09/compile-atlas-gromacs-nwchem-on-amd-fx.html

You can also link to openblas, as shown in this post: http://verahill.blogspot.com.au/2012/09/compiling-and-testing-gamess-us-on.html


1. Get GAMESS US
Go to http://www.msg.chem.iastate.edu/GAMESS/download/register/
Check the tick boxes next to the architectures you intend to use GAMESS on. Fill out your email address, and hit Submit. You'll receive an email with instructions almost immediately.

The email will contain a URL to an archive with the source code, and a password for downloading it.
I'll presume that you downloaded the file, gamess-current.tar.gz, to ~/Downloads.


2. Untar and prepare
I'm presuming that you don't already have any /opt/gamess or /opt/gamess/gamess directories. In my earlier write-ups (e.g. this) I put everything in /opt/gamess, which isn't a good long-term strategy since you often want to keep earlier versions of computational software alongside newer ones.

sudo apt-get install build-essential gfortran openmpi-bin libopenmpi-dev libboost-all-dev
sudo mkdir /opt/gamess -p
sudo chown $USER:$USER /opt/gamess
cd /opt/gamess
cp ~/Downloads/gamess-current.tar.gz gamess-2013r1.tar.gz
tar xvf gamess-2013r1.tar.gz
mv gamess gamess-2013r1
cd gamess-2013r1/


3. Configure
NOTE: Even if you may have gfortran 4.7 (e.g. Wheezy) you should give the version as 4.6 during configure (see below).


./config
please enter your target machine name: linux64 GAMESS directory? [/opt/gamess/gamess-2013r1] GAMESS build directory? [/opt/gamess/gamess-2013r1] Version? [00] 13 Please enter your choice of FORTRAN: gfortran Please enter only the first decimal place, such as 4.1 or 4.6: 4.6
ACML:
Enter your choice of 'mkl' or 'atlas' or 'acml' or 'none': acml enter this full pathname: /opt/acml/acml5.3.1 Math library 'acml' will be taken from /opt/acml/acml5.3.1/gfortran64_int64/lib
Atlas:
I've compiled ATLAS myself in the past, but the libs have not worked with all programs. Here we use the debian libs instead.
Enter your choice of 'mkl' or 'atlas' or 'acml' or 'none': atlas Please enter the Atlas subdirectory on your system: /usr/lib/atlas-base Math library 'atlas' will be taken from /usr/lib/atlas-base
mpi:
communication library ('sockets' or 'mpi')? mpi Enter MPI library (impi, mvapich2, mpt, sockets): openmpi Please enter your openmpi's location: /usr/lib/openmpi

4. Build
Edit comp and change
1663 # -fno-whole-file suppresses argument's data type checking 1664 set OPT='-O2' 1665 if (".$GMS_DEBUG_FLAGS" != .) set OPT="$GMS_DEBUG_FLAGS"
to
1663 # -fno-whole-file suppresses argument's data type checking 1664 set OPT='-O0' 1665 if (".$GMS_DEBUG_FLAGS" != .) set OPT="$GMS_DEBUG_FLAGS"
or exam44.inp will fail.

cd ddi/
./compddi
cd ../
./compall

If you are building with openmpi, edit lked and change
958 case openmpi: 959 set MPILIBS="-L$GMS_MPI_PATH/lib64" 960 set MPILIBS="$MPILIBS -lmpi" 961 breaksw
to
958 case openmpi: 959 set MPILIBS="-L$GMS_MPI_PATH/lib" 960 set MPILIBS="$MPILIBS -lmpi" 961 breaksw
Make the proper symlinks if you are using ATLAS:
sudo ln -s /usr/lib/atlas-base/libatlas.so.3 /usr/lib/atlas-base/libatlas.so
sudo ln -s /usr/lib/atlas-base/libf77blas.so.3 /usr/lib/atlas-base/libf77blas.so

./lked gamess 13r1 


I use my own script called gmrun:
#!/bin/csh
set TARGET=mpi
set SCR=$HOME/scratch
set USERSCR=/scratch
set GMSPATH=/opt/gamess/gamess-2013r1
set JOB=$1
set VERNO=$2
set NCPUS=$3

if ( $JOB:r.inp == $JOB ) set JOB=$JOB:r
echo "Copying input file $JOB.inp to your run's scratch directory..."
cp $JOB.inp $SCR/$JOB.F05

setenv TRAJECT $USERSCR/$JOB.trj
setenv RESTART $USERSCR/$JOB.rst
setenv INPUT $SCR/$JOB.F05
setenv PUNCH $USERSCR/$JOB.dat
if ( -e $TRAJECT ) rm $TRAJECT
if ( -e  $PUNCH ) rm $PUNCH
if ( -e  $RESTART ) rm $RESTART
source $GMSPATH/gms-files.csh

setenv LD_LIBRARY_PATH /usr/lib/openmpi/lib:$LD_LIBRARY_PATH
mpiexec -n $NCPUS $GMSPATH/gamess.$VERNO.x|tee $JOB.out
cp $PUNCH .

To run, do e.g.
gmrun exam01.inp 13r1 2

10 June 2013

444.Building Wine 1.6-rc1 on Debian Wheezy using a chroot

Here's a generic way of building wine which works for 1.6-rc1 (and 1.5.28 and everything in between except 1.5.30).

The build instructions are recycled from my 1.5-series instructions -- the 1.5 series is the development version and Wine 1.6 is the new stable release, and so Wine 1.5.31 and 1.6-rc1 should be pretty similar. As a release candidate (rc) 1.6-rc1 isn't meant for production work. For that, use Wine 1.4 which is the current stable series.


Here's a link to the release announcement: http://www.winehq.org/announce/1.6-rc1

See here for information about 3D acceleration using libGL/U with Wine: http://verahill.blogspot.com.au/2013/05/429-briefly-wine-libglliubglu-blender.html

Getting started:
If you set up a e.g. chroot to build 1.5.28 or  before or 1.5.31 (1.5.30 had a bug which needed patching), you don't need to set up a new chroot to build 1.6-rc1. In that case, skip the set-up step below and instead re-enter your existing chroot like this:
sudo mount -o bind /proc wine32/proc
sudo cp /etc/resolv.conf wine32/etc/resolv.conf
sudo chroot wine32
su sandbox
cd ~/tmp

And skip to 'Building wine'.

Otherwise do this:
Setting up the Chroot
sudo apt-get install debootstrap
mkdir $HOME/tmp/architectures/wine32 -p
cd $HOME/tmp/architectures
sudo debootstrap --arch i386 wheezy $HOME/tmp/architectures/wine32 http://ftp.au.debian.org/debian/
sudo mount -o bind /proc wine32/proc
sudo cp /etc/resolv.conf wine32/etc/resolv.conf
sudo chroot wine32

You're now in the chroot:
apt-get update
apt-get install locales sudo vim
echo 'export LC_ALL="C"'>>/etc/bash.bashrc
echo 'export LANG="C"'>>/etc/bash.bashrc
echo '127.0.0.1 localhost beryllium' >> /etc/hosts
source /etc/bash.bashrc
adduser sandbox
usermod -g sudo sandbox
echo 'Defaults !tty_tickets' >> /etc/sudoers
su sandbox
cd ~/

Replace 'beryllium' with the name your host system (it's just to suppress error messages)

Building Wine
While still in the chroot, continue (the i386 is ok; don't worry about it -- you don't actually need it):

sudo apt-get install libx11-dev:i386 libfreetype6-dev:i386 libxcursor-dev:i386 libxi-dev:i386 libxxf86vm-dev:i386 libxrandr-dev:i386 libxinerama-dev:i386 libxcomposite-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libdbus-1-dev:i386 libgnutls-dev:i386 libncurses-dev:i386 libsane-dev:i386 libv4l-dev:i386 libgphoto2-2-dev:i386 liblcms-dev:i386 libgstreamer-plugins-base0.10-dev:i386 libcapi20-dev:i386 libcups2-dev:i386 libfontconfig-dev:i386 libgsm1-dev:i386 libtiff-dev:i386 libpng-dev:i386 libjpeg-dev:i386 libmpg123-dev:i386 libopenal-dev:i386 libldap-dev:i386 libxrender-dev:i386 libxml2-dev:i386 libxslt-dev:i386 libhal-dev:i386 gettext:i386 prelink:i386 bzip2:i386 bison:i386 flex:i386 oss4-dev:i386 checkinstall:i386 ocl-icd-libopencl1:i386 opencl-headers:i386 libasound2-dev:i386 build-essential
mkdir ~/tmp
cd ~/tmp
wget http://prdownloads.sourceforge.net/wine/wine-1.6-rc1.tar.bz2

tar xvf wine-1.6-rc1.tar.bz2
cd wine-1.6-rc1/
./configure
time make -j3
sudo checkinstall --install=no
checkinstall 1.6.2, Copyright 2009 Felipe Eduardo Sanchez Diaz Duran This software is released under the GNU GPL. The package documentation directory ./doc-pak does not exist. Should I create a default set of package docs? [y]: Preparing package documentation...OK Please write a description for the package. End your description with an empty line or EOF. >> wine 1.6-rc1 >> ***************************************** **** Debian package creation selected *** ***************************************** This package will be built according to these values: 0 - Maintainer: [ root@beryllium ] 1 - Summary: [ wine 1.6-rc1] 2 - Name: [ wine-1.6 ] 3 - Version: [ 1.6-rc1] 4 - Release: [ 1 ] 5 - License: [ GPL ] 6 - Group: [ checkinstall ] 7 - Architecture: [ i386 ] 8 - Source location: [ wine-1.6-rc1 ] 9 - Alternate source location: [ ] 10 - Requires: [ ] 11 - Provides: [ wine-1.6 ] 12 - Conflicts: [ ] 13 - Replaces: [ ]
Checkinstall takes a little while (In particular this step: 'Copying files to the temporary directory...').

Installing Wine

Exit the chroot
sandbox@beryllium:~/tmp/wine-1.6-rc1$ exit
exit
root@beryllium:/# exit
exit
me@beryllium:~/tmp/architectures$ 

On your host system
 Enable multiarch* and install ia32-libs, since you've built a proper 32 bit binary:

sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install ia32-libs

*At some point I think ia32-libs may be replaced by proper multiarch packages, but maybe not. So we're kind of doing both here.

 Copy the .deb package and install it
sudo cp wine32/home/sandbox/tmp/wine-1.6-rc1/wine_1.6-rc1-1_i386.deb .
sudo chown $USER wine_1.6-rc1-1_i386.deb
sudo dpkg -i wine_1.6-rc1-1_i386.deb

443. Briefly: Running the QA tests in NWChem

To make sure that everything is working properly and that you get the expected results from your nwchem binaries, you should run the QA tests that come with nwchem.

Here's how to do it with the nwchem 6.3 QA tests.

I'm presuming that you built nwchem with mpi as shown e.g. here: http://verahill.blogspot.com.au/2013/05/424-nwchem-63-on-debian-wheezy.html

In this particular case I'm using nwchem linked with openblas on an AMD Phenom II 1055T with 8 Gb RAM since it was the only node that was free.

0. Go to the QA directories.
In my case everything is housed in /opt/nwchem/nwchem-6.3-src.patched and the QA tests are in /opt/nwchem/nwchem-6.3-src.patched/QA

1. Run the tests
First set the environmental variables, then start the tests. The 6 in './doqmtests.mpi 6' is the number of threads i.e. processors to use in parallel.

export NWCHEM_TOP=/opt/nwchem/nwchem-6.3-src.patched
export NWCHEM_TARGET=LINUX64
./doqmtests.mpi 6 |tee doqmtests.mpi.log
====================================================== QM: Running all tests (including some really big ones) ====================================================== Running tests/h2o_opt/h2o_opt cleaning scratch copying input and verified output files running nwchem (/opt/nwchem/nwchem-6.3-src.patched/bin/LINUX64/nwchem) 26.3u 8.8s 0:07.13 492.8% (0t+0ds+0avg+49046max)k 0i+6199464o 18pf 0swaps verifying output ... OK Running tests/c2h4/c2h4 cleaning scratch copying input and verified output files running nwchem (/opt/nwchem/nwchem-6.3-src.patched/bin/LINUX64/nwchem) 55.9u 2.1s 0:10.92 532.3% (0t+0ds+0avg+59834max)k 0i+808848o 19pf 0swaps verifying output ... OK [..]

2. Verify

Once the runs are done, go through the log to find out which, if any, failed. In my case, I had
Running tests/autosym/autosym 
 
     cleaning scratch
     copying input and verified output files
     running nwchem (/opt/nwchem/nwchem-6.3-src.patched/bin/LINUX64/nwchem)
 
15.3u 2.5s 0:04.20 426.4% (0t+0ds+0avg+48842max)k 0i+1325784o 17pf 0swaps
     verifying output ... failed

Running tests/dft_s12gh/dft_s12gh 
 
     cleaning scratch
     copying input and verified output files
     running nwchem (/opt/nwchem/nwchem-6.3-src.patched/bin/LINUX64/nwchem)
 
619.4u 4.3s 1:45.33 592.1% (0t+0ds+0avg+76724max)k 2472i+1938952o 26pf 0swaps
     verifying output ... failed
 
Failed
 
Running tests/cosmo_trichloroethene/cosmo_trichloroethene 
 
     cleaning scratch
     copying input and verified output files
     running nwchem (/opt/nwchem/nwchem-6.3-src.patched/bin/LINUX64/nwchem)
 
113.6u 2.3s 0:19.57 592.8% (0t+0ds+0avg+63700max)k 64i+1149456o 20pf 0swaps
     verifying output ... failed

 Running tests/bsse_dft_trimer/bsse_dft_trimer 
 
     cleaning scratch
     copying input and verified output files
     running nwchem (/opt/nwchem/nwchem-6.3-src.patched/bin/LINUX64/nwchem)
 
228.2u 4.8s 0:40.16 580.5% (0t+0ds+0avg+53806max)k 0i+2716224o 21pf 0swaps
     verifying output ... failed
 
Failed

and so on. Not a good start.

3. Troubleshoot the failed tests
To find out whether the failures are significant, we first need to understand how the script is doing the testing.

In runtests.mpi.unix
369 # Now verify the output 370 371 echo -n " verifying output ... " 372 373 perl $NWPARSE $STUB.out >& /dev/null 374 if ($status) then 375 echo nwparse.pl failed on test output $STUB.out 376 set overall_status = 1 377 continue 378 endif 379 perl $NWPARSE $STUB.ok.out >& /dev/null 380 if ($status) then 381 echo nwparse.pl failed on verified output $STUB.ok.out 382 set overall_status = 1 383 continue 384 endif 385 386 diff -w $STUB.ok.out.nwparse $STUB.out.nwparse >& /dev/null 387 @ diff1status = $status 388 # 389 endif 390 # 391 392 if ($diff1status) then 393 echo "failed" 394 set overall_status = 1 395 continue 396 else

In my case autosym failed:

cd testoutputs
diff autosym.ok.out.nwparse autosym.out.nwparse 
45c45 < Effective nuclear repulsion energy (a.u.) 4265.6221 --- > Effective nuclear repulsion energy (a.u.) 4265.6222
It seems to be a rounding error. As far as I know the precision at which the data is stored is significantly higher than at which it is reported, so this doesn't necessarily need to be a problem (it's still not a good thing though). Note that everything else, such as the thermochemical parameters, are identical.

Continuing:
diff dft_s12gh.ok.out.nwparse dft_s12gh.out.nwparse 
52c52 < The Zero-Point Energy (Kcal/mol) = 21.82496 --- > The Zero-Point Energy (Kcal/mol) = 21.82497 128c128 < H 0.0123 0.0030 0.0000 --- > H 0.0122 0.0030 0.0000
Same thing.

Here's a list over the tests that failed for me (-> indicates that the execution failed -- more details below; * indicates that it is expected to fail):

autosym
dft_s12gh
cosmo_trichloroethene
bsse_dft_trimer
cosmo_h3co
cosmo_h3co_gp
h2o_diag_to_cg_ub3lyp
* oh2
dft_cr2
dft_x
dft_ozone
hess_nh3_ub3lyp
pspw_SiC
paw
-> tddft_h2o_mxvc20
-> tddft_h2o_uhf_mxvc20
tce_cr_eom_t_ch_rohf
hi_zora_sf
o2_zora_so
lys_qmmm
ethane_qmmm
qmmm_opt0
prop_ch3f
ch3f-lc-wpbe
ch3f-lc-wpbeh
ch3radical_rot
ch3radical_unrot
cho_bp_props
-> prop_cg_nh3_b3lyp
acr-camb3lyp-cdfit
acr-camb3lyp-direct
acr_lcblyp
o2_bnl
fh_m06 ???
disp_dimer_ch4
disp_dimer_ch4_cgmin
mep-test
sif_sodft
h2o_raman_3
h2o_raman_4
tropt-ch3nh2
h3_dirdyvtst
h2o_hcons
etf_hcons
cho_bp_props
-> dntmc_h2o_nh3
5h2o_core
co_core
talc
neb-fch3cl
neb-isobutene
nwxc_pspw_1he
nwxc_pspw_1ne
nwxc_pspw_4n
nwxc_pspw_4p
nwxc_pspw_new_1he
nwxc_pspw_new_3he
nwxc_pspw_new_1ne
nwxc_pspw_new_4n
nwxc_pspw_new_1ar
nwxc_pspw_new_4p
nwxc_pspw_new_1kr
nwxc_pspw_new_4as
nwxc_pspw_new_1xe
nwxc_pspw_new_4sb
hess_nh3_dimer
pbo_nesc1e
h2o_selci
hess_biph
-> ch4_zts
-> ch4cl_zts

All the jobs without a '->' or '*' failed due to rounding errors. To quickly go through them I put the list of failed jobs in a file, and then did
cat fails |xargs -I {} diff testoutputs/{}.ok.out.nwparse testoutputs/{}.out.nwparse|less


The jobs that failed outright are listed below:

-> tddft_h2o_mxvc20
tddft_diagon: negative excitation energy 0 ------------------------------------------------------------------------ This type of error is most commonly associated with calculations not reaching convergence criteria
-> tddft_h2o_uhf_mxvc20
Last System Error Message from Task 5:: Numerical result out of range tddft_diagon: negative excitation energy 0
-> prop_cg_nh3_b3lyp
task hessian incompatible with cgmin 0 ------------------------------------------------------------------------ A feature requested has not yet been implemented
-> dntmc_h2o_nh3
********** Destroying SubGroups *********** ******************************************* deleting cloned rtdb deleting cloned rtdb Closing subgroup Closing subgroup 1:1:ga_pgroup_destroy_:Attempt to destroy process group with attached GAs:: 2 (rank:1 hostname:boron pid:1941):ARMCI DASSERT fail. ../../ga-5-2/armci/src/common/armci.c:ARMCI_Error():208 cond:0 2:2:ga_pgroup_destroy_:Attempt to destroy process group with attached GAs:: 2 (rank:2 hostname:boron pid:1942):ARMCI DASSERT fail. ../../ga-5-2/armci/src/common/armci.c:ARMCI_Error():208 cond:0 Last System Error Message from Task 1:: No such file or directory Last System Error Message from Task 2:: No such file or directory
-> ch4_zts
scf string failed 0 ------------------------------------------------------------------------ This type of error is most commonly associated with calculations not reaching convergence criteria
-> ch4cl_zts
scf string failed 0 ------------------------------------------------------------------------ This type of error is most commonly associated with calculations not reaching convergence criteria

It's time to go back and compare with 1. nwchem-6.3/acml and 2. nwchem-6.1.1/openblas and 3. a different processor architecture...

The question is how serious this is. In most cases I think the rounding errors are ok, but errors do accumulate, and especially when large and small numbers are multiplied they can become significant.

08 June 2013

442. Down? www.nwchem-sw.org

Epilogue: looks like it was announced (indirectly) here: http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id856/#post_3167

Update 10/6/2013: the site is back online.

Original post:
www.nwchem-sw.org is currently (Fri 7 pm 7 June 2013 PDT; Sat 12 pm 8 June AEST)  rejecting all connections with

"The specified URL cannot be found"

It's been doing it for a couple of hours now. It doesn't matter what your target URL is -- all connections are rejected.

07 June 2013

441. Links to two good articles on windows, piracy and linux.

I don't like the idea of simply posting links to other blogs, but in this case I'll make an exception due to the quality of the articles.

These two articles from 2008 discuss the lack of adoption of linux based on the notion that windows is de facto free (gratis) to most people:
http://autotelic.com/windows_is_free
http://autotelic.com/no_really_-_windows_is_free

I find them well-written and well-reasoned.

And they address the basis of one of the more disparaging (although to some extent true) remarks by people who don't see what the fuss over linux is all about: That we can't get people to use Linux even by giving it away for free.

Anyway.From a more personal POV:

* Having lived in China I definitely agree with the idea that piracy of windows is one biggest problems when it comes to the adoption of free software in the developing world. Not only windows of course -- I could even buy SPSS, Origin and Matlab in my local computer software street stall or my local DVD store (all pirated, of course).

* Having lived in the developing world I also agree that Windows is free in the sense that you can hardly buy a computer without getting a copy of a recent windows version included (whether you want it or not).

* Working at a university -- and having worked at five in total -- I also agree that it's easy enough to get free access to most pieces of proprietary software and since the distinction between Home and Work is a bit fluid in academia, for all intents and purposes I have free access to Office, Windows, Photoshop, SPSS etc.

* And finally, having bought my first computer in as a teenager in 1993 (a 1.8 MHz 386SX, 2 Mb RAM, 28 Mb HDD -- second hand) I also grew up swapping floppies with windows (3.11 FTW!), DOS (the box had 5.0, but got DOS 6 from a class mate), and various pieces of free/shareware that we ordered via mailorder...(or bbs -- but anything over 100 kb took forever). I don't think teens of today look at things much differently from how we did back then.


06 June 2013

440. Briefly: Upgrading Arch -- consolidation of /sbin, /bin and /usr/sbin

Anyone running Arch and attempting to upgrade recently will have run into this:

(83/83) checking for file conflicts  [##########] 100%
error: failed to commit transaction (conflicting files)
filesystem: /bin exists in filesystem
filesystem: /sbin exists in filesystem
filesystem: /usr/sbin exists in filesystem
Errors occurred, no packages were upgraded.


There's a solution:
http://news.softpedia.com/news/Read-This-Article-Before-Updating-Your-Arch-Linux-358104.shtml

What I'm doing in this post is simply testing that solution. The intellectual contribution from me is very small.

1. Non-official packages
pacman -Qqo /bin /sbin /usr/sbin | pacman -Qm -
bootchart2-git 20130314-1
Not too bad.
sudo pacman -R bootchart2-git

2. Unofficial repositories
cat /etc/pacman.conf | grep '^\['
[options] [core] [extra] [community] [multilib

OK. Let's see whether there are any packages around:

paclist options | awk ' { print $1 } ' | pacman -Ql - | grep ' /s\?bin/\| /usr/sbin/'
apache /usr/sbin/ apache /usr/sbin/ab apache /usr/sbin/apachectl apache /usr/sbin/apxs [..] wpa_supplicant /usr/sbin/wpa_supplicant zvbi /usr/sbin/ zvbi /usr/sbin/zvbid
paclist core | awk ' { print $1 } ' | pacman -Ql - | grep ' /s\?bin/\| /usr/sbin/' paclist extra | awk ' { print $1 } ' | pacman -Ql - | grep ' /s\?bin/\| /usr/sbin/' paclist community | awk ' { print $1 } ' | pacman -Ql - | grep ' /s\?bin/\| /usr/sbin/' paclist multilib | awk ' { print $1 } ' | pacman -Ql - | grep ' /s\?bin/\| /usr/sbin/'

I think 'options' would qualify as an Official repository, so we are fine.
3. Orphan packages
find /bin /sbin /usr/sbin -exec pacman -Qo -- {} + >/dev/null

comes up blank. Good.

4. Upgrade
sudo pacman -Syu --ignore filesystem,bash
[..] (27/81) upgrading libgdm [################] 100% (28/81) upgrading gdm [################] 100% warning: directory permissions differ on /var/log/gdm/ filesystem: 711 package: 1770 (29/81) upgrading libx11 [..] (67/81) upgrading pacman-mirrorlist [################] 100% warning: /etc/pacman.d/mirrorlist installed as /etc/pacman.d/mirrorlist.pacnew (68/81) upgrading pm-utils [..] (80/81) upgrading wicd-gtk [################] 100% (81/81) upgrading zvbi [################] 100%

So far, so good.

sudo pacman -S bash
resolving dependencies... looking for inter-conflicts... Packages (1): bash-4.2.045-4 Total Installed Size: 3.51 MiB Net Upgrade Size: -0.10 MiB :: Proceed with installation? [Y/n] :: Proceed with installation? [Y/n] (1/1) checking keys in keyring [###################] 100% (1/1) checking package integrity [###################] 100% (1/1) loading package files [###################] 100% (1/1) checking for file conflicts [###################] 100% (1/1) checking available disk space [###################] 100% (1/1) upgrading bash
sudo pacman -Su
:: Starting full system upgrade... resolving dependencies... looking for inter-conflicts... Packages (1): filesystem-2013.05-2 Total Installed Size: 0.01 MiB Net Upgrade Size: -0.30 MiB :: Proceed with installation? [Y/n] (1/1) checking keys in keyring [###########] 100% (1/1) checking package integrity [###########] 100% (1/1) loading package files [###########] 100% (1/1) checking for file conflicts [###########] 100% (1/1) checking available disk space [###########] 100% (1/1) upgrading filesystem [###########] 100%

So far so done!

05 June 2013

439. Calculate frequencies from a hessian file from NWChem: example in Octave (matlab)

I wanted to calculate normal modes (frequencies) for specific atoms in a calculation, and so I had to write my own code.

This Octave code calculates frequencies for the first N atoms, where N is given in the input.mass file.

Background
The format that NWChem uses for the Hessian is that of a flat, triangular matrix i.e. a triangular matrix such as
1  
2 3 
4 5 6
is represented as
1
2
3
4
5
6

The Hessian is symmetric around the diagonal, so the full Hessian matrix is
1 2 4
2 3 5
4 5 6

The Hessian is independent of the masses of the atom pairs, while the frequencies are heavily dependent on the masses (isotope effects are quite visible for light elements).

To get the mass-weighted matrix we divide by the square root of the product of the masses (H * /(sqrt(m1*m2))). Note that the matrix reported in the nwchem output ("MASS-WEIGHTED NUCLEAR HESSIAN (Hartree/Bohr/Bohr/Kamu)") is multiplied by 1,000.

Once you have the mass-weighted hessian you need to calculate the eigenvalues, sort them and convert them to cm-1 using a scaling factor.

That's it.

The code:
See below for example input.mass and input.hess

%% prepare
clear;
format long

%%Calculate conversion factor from H/B/B/amu to cm-1

%% csi=299792458; %speed of light, m/s 
%% t2au=2.418884326505E-17; % seconds per a.u.
%% Better to do it by hand to avoid rounding errors:
cau=(2.99792458 * 2.418884326505)*1E-9; %c in metres per t(a.u.)

%% 1 electron (au)=9.10938291E-31 kg
%% 1 amu = 1.66053892E-27 kg
%% Better to do by hand to avoid rounding errors:
amu2au=(1.66053892/9.10938291)*1E4;% 1 amu in a.u. (via kgs)
%% For clarity
cmtom=1/100; %m per cm
%% And finally we get our scaling factor:
scaling=cmtom*(1/(2*pi*cau*sqrt(amu2au))); %( m/cm * 1/((m/au) * au) = m/cm * 1/m = 1/cm)


%%read masses
% The mass file contains the masses of the atoms
% The first line is the number of atoms in the file
% The remaining lines are the atom masses in the same order
% as the atoms are given in the nwchem input
protomasses=fopen("input.mass");
natoms=str2num(fgetl(protomasses));
for i = 1:natoms
 mass(end+1)=str2num(fgetl(protomasses));
end
fclose(protomasses);

%% Read and construct hessian from flat hessian in .hess file
%% The .hess file provided by nwchem is flat (i.e. one
%% dimensional) and is the triangular form (i.e half) of 
%% the full hessian. We use fgetl/str2num so that we can deal 
%% with instances of scientific notation in the hessian file.
%% While we"re at it we construct the mass-weighted force matrix too.
protohessian=fopen("input.hess"); 
hessian=zeros(3*natoms);
massweighted=zeros(3*natoms);

for i = 1:3*natoms
 for j=1:i
  hessian(i,j)=str2num(fgetl(protohessian));
  massweighted(i,j)=hessian(i,j)/sqrt( mass(ceil(i/3))*mass(ceil(j/3)));
 end
end

for i=1:3*natoms
 for j=1:i
  hessian(j,i)=hessian(i,j);
  massweighted(j,i)=massweighted(i,j);
 end
end

%% Diagonalize and compute frequencies in cm^{-1}
eigen=sort(eig(massweighted));
freqs=sqrt(eigen).*scaling;

%% Make imaginary frequencies negative and store them 
%% in a new array
for n=1:size(freqs,1)
 if imag(freqs(n))==0
  frequencies(end+1,1)=real(freqs(n));
 else
  frequencies(end+1,1)=-imag(freqs(n));
 end
end

%% Echo frequencies to stdout
printf("%10.4f \n",frequencies)
%% Save frequencies as well to modes.outs
outfile=fopen("normal.out","w");
fprintf(outfile,"%i \n",natoms);
fprintf(outfile,"%10.10f \n",frequencies);
fclose(outfile);
%save 'modes.out' -ascii  frequencies

input.mass (for water):
3
1.5994910D+01
1.0078250D+00
1.0078250D+00

input.hess (this one has imaginary frequencies as well):
     6.6177469151D-01
    -5.8658669668D-12
    -1.0013075598D-05
     1.0754299967D-09
     4.5060920407D-10
     3.6644723357D-01
    -3.3088202114D-01
     2.1099357839D-10
     1.6617441386D-01
     3.6163164885D-01
     2.5270659061D-12
     4.0920019206D-06
     3.2209366184D-11
     1.6382988861D-11
     8.3427731090D-07
     2.3904755566D-01
    -2.2311539742D-10
    -1.8322029567D-01
    -2.0261099118D-01
     1.1292349908D-10
     1.7796238990D-01
    -3.3088202212D-01
    -2.4469194991D-10
    -1.6617441477D-01
    -3.0749615389D-02
     1.2368245322D-10
    -3.6436594678D-02
     3.6163164980D-01
     2.5272503844D-12
     4.0920029550D-06
     3.2022391582D-11
     1.6289326095D-11
    -4.9229359909D-06
    -1.5407535297D-11
    -1.8816632580D-11
     8.3427660670D-07
    -2.3904755666D-01
    -2.2750774181D-10
    -1.8322029575D-01
     3.6436523006D-02
     1.1512005611D-10
     5.2580053385D-03
     2.0261099171D-01
     1.1238793371D-10
     1.7796238961D-01

Output:
 
  -11.0036 
   -1.6327 
    3.1676 
    3.9298 
    7.5811 
   12.2862 
 1619.0207 
 3616.0904 
 3781.1341

c.f.
 ----------------------------------------------------------------------------
 Normal Eigenvalue ||                 Infra Red Intensities
  Mode   [cm**-1]  || [atomic units] [(debye/angs)**2] [(KM/mol)] [arbitrary]
 ------ ---------- || -------------- ----------------- ---------- -----------
    1      -11.004 ||    0.426523           9.840       415.796      59.477
    2       -1.633 ||    0.000029           0.001         0.028       0.004
    3        3.168 ||    0.000003           0.000         0.003       0.000
    4        3.930 ||    0.000700           0.016         0.682       0.098
    5        7.581 ||    0.134394           3.101       131.014      18.741
    6       12.286 ||    0.000000           0.000         0.000       0.000
    7     1619.021 ||    0.070174           1.619        68.409       9.786
    8     3616.091 ||    0.004517           0.104         4.404       0.630
    9     3781.135 ||    0.009065           0.209         8.837       1.264
 ----------------------------------------------------------------------------

03 June 2013

438. Very briefly: Freeing up RAM (sort of)

After having played around with two virtual machines simultaneous which at some point caused me to use/reserve all of my ram plus a small amount of swap (419 Mb), my desktop has been a bit slower. I don't see any real evidence that it's swapping to and from disk, but the system is much less responsive, e.g. when switching applications.

Note that when you RAM usage goes up, not all of the RAM is actually used in an active sense -- some may be reserved. Another factor to take into account is that you actually DO want to use as much RAM as possible as long as it improves performance, and this is done via caching. What's shown here is simply how to clean that cache. 

So is cleaning that cache a good thing? Well, sometimes. Empirically, it seems like dropping the caches can help if you've been doing something that caused a lot of ram to be used, but which is no longer running. Such as something heavily graphical (e.g. using VMD and rendering something) or a virtual machine.

See e.g. here for a discussion with comments: http://catalin-festila.blogspot.com.au/2011/06/myth-of-dropcaches.html

Anyway.

At the beginning this is how it looked (free -m):
total used free shared buffers cached Mem: 7991 7488 503 0 1097 379 -/+ buffers/cache: 6011 1980 Swap: 15257 419 14838
To free up the ram cache, do

me@beryllium:~$ sudo su
[sudo] password for me:
root@beryllium:/home/me# sync
root@beryllium:/home/me# echo 3 > /proc/sys/vm/drop_caches

And when we're done it looks like this:
total used free shared buffers cached Mem: 7991 3503 4488 0 47 147 -/+ buffers/cache: 3308 4683 Swap: 15257 419 14838
Our swap usage hasn't changed, but the apparent free RAM has increased significantly. And my computer feels snappier.

Often the need to free up RAM is precipitated by the presence of memory leaks though, since these are often manifested by the slow increase in the amount of RAM a program is using. Older versions of gnome-shell (including the one presently used in Wheezy) are known culprits, and ECCE has a tendency to eat up RAM like there's no tomorrow when running for too long (still working on getting hard numbers for it).


02 June 2013

437. system-config-firewall on debian

Please, read this first:
system-config-firewall is a python based Red Hat tool.

The best way to manage your firewall is by configuring iptables by hand. It gives you minimal rule sets which are clear and easy to read.

If you absolutely need a GUI, then try included debian tools such as gufw, firestarter etc. but note that they can only configure a small subset of what iptables are capable of. They also often given rule sets that can be difficult to read.

If you are coming to debian from rhel/fedora+clones and are missing some of the redhat tools, note that you are probably better of adapting the recommended work flow of the distro you are using.

Having said that, e.g. system-config-samba works fine on debian so far (but again, configuring samba by hand is not that difficult).

The real work was done by the person who did the patching (can't find a specific name -- just Ubuntu Contributions)

This was done on Debian Jessie. I may have missed some of the pre-requisite dependencies. Post comments and I shall add.

Anyway, look at this as an exercise, nothing more.

sudo apt-get install checkinstall python-slip-dbus python-tksnack build-essential gfortran debhelper python-newt selinux-utils intltool python-support
mkdir ~/tmp
cd ~/tmp
mkdir sysfirewall
cd sysfirewall/
wget https://launchpad.net/~ubuntu-contribs/+archive/contrib/+files/system-config-firewall_1.2.29.orig.tar.gz
wget https://launchpad.net/~ubuntu-contribs/+archive/contrib/+files/system-config-firewall_1.2.29-2.diff.gz
tar xvf system-config-firewall_1.2.29.orig.tar.gz
gunzip system-config-firewall_1.2.29-2.diff.gz
sed -i 's/python2.6/python2.7/g' system-config-firewall_1.2.29-2.diff
patch -p0 < system-config-firewall_1.2.29-2.diff
cd system-config-firewall-1.2.29/
dpkg-buildpackage -uc -us
sudo dpkg -i ../*.deb
sudo apt-get -f install
system-config-firewall



You can see what your rules are and if they have taken effect by doing

sudo iptables -L
Chain INPUT (policy ACCEPT) target prot opt source destination ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED ACCEPT icmp -- anywhere anywhere ACCEPT all -- anywhere anywhere ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh REJECT all -- anywhere anywhere reject-with icmp-host-prohibited Chain FORWARD (policy ACCEPT) target prot opt source destination REJECT all -- anywhere anywhere reject-with icmp-host-prohibited Chain OUTPUT (policy ACCEPT) target prot opt source destination

You can also use your pre- and hand-configured rule set and put it in /etc/default/iptables and /etc/default/ip6tables, then edit with system-config-firewall (for whatever reason).

01 June 2013

436. Miramath on debian

For no particular reason (well, because of this: http://forums.debian.net/viewtopic.php?f=10&t=104566) here's a very brief how-to to get miramath up and running. There may well be more packages required and which I had pre-installed. The pre-built miramath binaries don't work on debian jessie, but since this is python...well..Anyway:

First compile and install PyQwt-Polar. It takes quite a while. Note that there's a conflict between PyQwt-Polar and python-qwt5-qt4 which may potentially be serious. A side-effect of this is that you can't build a proper package, since dpkg would detect this conflict (hence the make install). What this means is that PyQwt-Polar will over-write files installed by the python-qwt5-qt4 files i.e. A potentially Bad Thing.  Consider setting it up in a chroot.

There's another reason why I wouldn't recommend miramath for serious work at the moment -- it is in pre-alpha and the degree to which is it maintained isn't entirely clear to me. For scientific work, use octave and maxima, which are well-maintained and have been around for ages.

The flip-side of this is obviously that there's no harm in checking out a new and interesting piece of software, and presumably no-one is preventing you from contributing. Anyway. The main reason for checking this out was the debian forum post referenced above.

I did this on Debian Jessie, but it should work on Wheezy as well.

sudo apt-get install python-sip-dev python-qt4-dev python-scipy python-qt4-gl python-sympy python-ply python-qwt5-qt4 python-qwt3d-qt4 python-guiqwt python-dev libqwt-dev libqwt5-qt4
mkdir ~/tmp/
cd ~/tmp
wget http://ufpr.dl.sourceforge.net/project/miramath/PyQwt-Polar/PyQwt-Polar-5.2.0.tar.bz2
tar xvf PyQwt-Polar-5.2.0.tar.bz2
cd PyQwt-Polar-5.2/
cd configure/
python configure.py -Q ../qwt-5.2
make
sudo make install

Then continue:
wget http://waix.dl.sourceforge.net/project/miramath/Release%20Tarballs/miramath0.020.tar.bz2
cd ~/miramath
sed -i 's/sympy.abs/sympy.Abs/g' symbolics_init.py
chmod +x main.py
./main.py


30 May 2013

435. Briefly: Frippery extensions for Gnome 3.8

I'm running Debian Wheezy on most of my machines, Debian Jessie on my laptop, and Arch on my home multimedia centre. Arch has Gnome 3.8 which up until recently didn't have the extensions that I had relied on to make Gnome 3.8 use-able.

Well, the Frippery extensions for Gnome 3.8 are out now: http://intgat.tigress.co.uk/rmy/extensions/index.html

Sure, you can install them the boring way by going to extensions.gnome.org, but it's more fun to do
cd ~
wget http://intgat.tigress.co.uk/rmy/extensions/gnome-shell-frippery-0.6.0.tgz
tar xvf gnome-shell-frippery-0.6.0.tgz

The files will be extracted into their proper locations.
At this point I (foolishly) did alt+F2, type r, and hit enter (to reload gnome shell) -- which promptly crashed GNOME 3. At this point I simply rebooted from a tty since restarting gdm didn't help. 

Use gnome-tweak-tool/Shell Extensions to enable and disable extensions.

Using the Frippery extensions (favourites, bottom panel etc.). Note that this desktop also uses conky to display the  sysinfo on the right hand of the desktop.