12 June 2013

450. Tor and Chrome on Debian

Note:
* For the Tor bundle see http://verahill.blogspot.com.au/2013/05/408-briefly-tor-on-debian-quick-option.html
* For securing your dropbox, see http://verahill.blogspot.com.au/2013/04/398-securing-your-dropbox.html
* For encrypting your filesystem with encfs, see http://verahill.blogspot.com.au/2013/05/408-briefly-tor-on-debian-quick-option.html
* For one-time passwords (OTPW), see http://verahill.blogspot.com.au/2013/04/385-otpw-connecting-from-insecure.html
* For encryption in general using PGP/GPG, OTR, SRTP for chat, email, voice and video, see http://verahill.blogspot.com.au/2013/04/381-encrypting-chat-voice-video.html
* For truecrypt with dropbox, see http://verahill.blogspot.com.au/2012/04/using-truecrypt-with-dropbox.html

Post begins:
I think it's fair to say that online privacy is in the spotlight again, temporarily,  in particular if you are not living in the US. After all, the rest of the world is offered no protection from US agencies.

There are two levels of snooping that (can) go on:
Case 1:  outright intercept of communications
In this case your emails are read, your browsing data is intercepted and your phone conversations tapped. This is the most intrusive form, and I think even in the US a warrant is required for the intercept of this type of data (whether that's too easy of difficult to get is another question entirely).

Case 2: mining of 'meta-data'
In this case data such as recipient/sender of emails, URLs that you've been visiting, and whom you have been calling/called by are collected. In addition, e.g. cell phone tower records can be collected to track your whereabouts 24/7.

While the contents of your conversations isn't known, your entire social and professional life can be charted.
As far as I understand this is what NSA has been engaging in. Likewise, knowing exactly where you are at any given point in time, a pretty detailed picture of your life can be painted.

Begin Rant
I don't have anything to hide, but I am not too keen on the government having better records of my life than I do myself. And I should be the one deciding what to share as long as the presumption of innocence holds.

Also, we're making the presumption that the government is benign, and as has been shown repeatedly, it isn't always. That goes for the US government, the UK government and just about any bloody imaginable government, and for a simple reason: the government is made up of people. In particular people who are keen on 'leading' i.e. controlling others. Even a benign despot is a despot.

There's no use being naive -- in either direction. There are legitimate reasons for clandestine organisations wanting to mine data, and there are legitimate reasons for why we should not give them a carte blanche.

Whether you use PGP/GPG or not won't affect the mining of meta-data. Nor will OTR, although it might in theory give you a somewhat better level of deniability (but not really).

Using PGP/GPG, OTR and encryption of data in general will only protect the content of your conversations, not the fact that they occurred. Not that it's easy getting people to start using encryption of their email, especially not since hotmail and gmail provided the final push into getting people to do all their email processing in the browser rather than using a more capable email client. Obviously Google would not be pleased if all communication was PGP encrypted, since this would create issues with targeted ads.

Finally, what really irks me is the fact that because John Doe won't use encryption -- or learn how to do it -- I also cannot use it. Instead we have to play according to the rules of the least technologically informed.
End Rant

Anyway. There are a few things you can do -- at least to make you feel better. Whether they have any real impact on your privacy depends on what other sources of information leakage there are in your life.

The simplest thing you can do is to do all your browsing anonymously, including setting up and checking your email. And the easiest way to do that is by using Tor.

It's easy enough to use the Tor Bundle, e.g. http://verahill.blogspot.com.au/2013/05/408-briefly-tor-on-debian-quick-option.html

However, I for some forsaken reason like using Chrome.

To set up Proxy SwitchySharp I'm following this post:
http://lifehacker.com/5614732/create-a-tor-button-in-chrome-for-on+demand-anonymous-browsing

NOTE: there are many layers to managing your privacy, and you're only as anonymous as your worst habits allow you to be. I'm a pessimist -- I think it is virtually impossible to protect yourself against a determined adversary. However, trying won't hurt.


Step 0. Block cookies by default and install an ad blocker

Cookies
Pretending to be anonymous won't help if you give the game away by exposing cookies that you acquired while surfing without Tor.

You'll be surprised how many websites require you to accept cookies -- however, it's up to you whether you want to put up with that. I only allow cookies with services that I've signed up to and that I trust. I refuse to allow in particular commercial sites to require cookies for me to simply visit.

In Chrome, go to Settings, Content Settings, and check:
* Block sites from setting any data
* Block third-party cookies and site data
* Clear cookies and other site and plug-in data when I close my browser

Disable:
* Allow local data to be set

You may want to restrict e.g. image loading, javascript, pop-ups, plugins etc. as well. It's down to you to weight inconvenience vs privacy.

Set Cookie and Site Data exceptions manually, and make sure to distinguish between Session Only and Allow.

Ads
Also, install e.g. simple adblock:
https://chrome.google.com/webstore/search/simple%20adblock



Step 1. Install the HTTPS everywhere extension
https://chrome.google.com/webstore/search/https%20everywhere?hl=en-GB



Step 2. Install Proxy SwitchySharp
https://chrome.google.com/webstore/search/proxy%20switchysharp?hl=en-GB

Set up a profile called Tor to use SOCK 5 with 127.0.0.1:9050
Go to the General Tab and enable Quick Switch.
Make sure to drag both Tor and Direct Connection into the Quick Switch field.



Step 3. Install Tor and Vidalia
Add the following to your /etc/apt/sources.list
deb http://deb.torproject.org/torproject.org wheezy main

Then do
sudo apt-get update
sudo apt-get install deb.torproject.org-keyring
sudo apt-get update
sudo apt-get install vidalia

Tor should run in the background whether you start Vidalia or not.

Step 4. Prevent DNS leaks:
[for fun, do
sudo apt-get install tcpdump
sudo tcpdump -pni eth0 'port domain'
before turning off prefetching. ]

To make sure that your DNS requests aren't being read (i.e. providing meta-data to your ISP), you will need to turn of DNS pre-fetching in Chrome.

Google is sneaky about it though -- to turn off prefetching go to Settings/Under the Bonnet and uncheck "Predict network actions to improve page load performance".

[If you set up tcpdump before you'll see how suddenly the IPs and URLs stop streaming by.]


Step 5. Start Tor/Vidalia
You don't seem to be able to launch Vidalia from the terminal, so launch Vidalia from within e.g. gnome.
In fact, you probably don't have launch vidalia as Tor should be run in the background.
Then open Chrome and navigate to e.g. whatsmyip.org or ipchicken.com:


You can turn on and off the proxy by clicking on the icon in the top right corner.

Step 6. Enable private browsing:
You don't want to risk one website being able to see what another website left behind. It shouldn't happen, but it has happened in the past.

Anyway, it's easy: open an Incognito window (ctrl + shift + N).


Done.
As far as I can tell this should give you some privacy. However, the question is how effective this is in the long run since it's difficult to maintain enough discipline to prevent any information leakage to occur.

449. Nwchem 6.3 -- updated sources. Compiling on Debian

The previous post uses the sources from the 17th of May.


What's new?
Maybe it's just me, but I can't find any obvious location where they list the differences between the 28th of June and the 17th of June releases. In fact, I wouldn't have known that there was a new version out unless someone had specifically pointed it out to me.

Anyway, luckily there's diff. There were more changes than the ones I'm showing below, but most of them were minor ones such as .stamp files and a re-done pair of QA test output files.
  1 Only in nwchem-6.3-src.2013-05-28/QA/tests: rodft-cam
  2 diff -r nwchem-6.3-src.2013-05-17/src/nwdft/scf_dft/dft_canorg.F nwchem-6.3-src.2013-05-28/src/nwdft/scf_dft/dft_canorg.F
  3 6c6
  4 < c     $Id: dft_canorg.F 23846 2013-03-19 04:08:25Z edo $
  5 ---
  6 > c     $Id: dft_canorg.F 24271 2013-05-24 06:48:33Z niri $
  7 279c279,280
  8 <          if(iter.ge.nfock/2)iswitc = iswitc+1
  9 ---
 10 >          if(iter.ge.nfock/2) iswitc = iswitc+1
 11 >          if(abs(delta).lt.1d-6) iswitc = iswitc+2
 12 diff -r nwchem-6.3-src.2013-05-17/src/nwdft/scf_dft/dft_scf.F nwchem-6.3-src.2013-05-28/src/nwdft/scf_dft/dft_scf.F
 13 9c9
 14 < c     $Id: dft_scf.F 23988 2013-04-08 23:06:52Z d3y133 $
 15 ---
 16 > c     $Id: dft_scf.F 24269 2013-05-24 00:55:11Z edo $
 17 819c819
 18 <          iswitc = 1
 19 ---
 20 >          iswitc = 2
 21 diff -r nwchem-6.3-src.2013-05-17/src/nwdft/scf_dft_cg/dft_roks_fock.F nwchem-6.3-src.2013-05-28/src/nwdft/scf_dft_cg/dft_roks_fock.F
 22 6c6
 23 < * $Id: dft_roks_fock.F 23999 2013-04-10 18:23:02Z d3y133 $
 24 ---
 25 > * $Id: dft_roks_fock.F 24274 2013-05-24 07:34:16Z niri $
 26 102c102
 27 <       double precision edisp    ! [input] dispersion corrrection
 28 ---
 29 >       double precision Edisp    ! [input] dispersion correction
 30 107c107
 31 <       double precision errmax, ebq
 32 ---
 33 >       double precision errmax, Ebq
 34 124c124
 35 <       integer g_tmp(2)
 36 ---
 37 >       integer g_tmp(nset)
 38 326a327,335
 39 >         call ga_zero(g_tmp(1))
 40 >         if (nopen.gt.0) then
 41 >           g_tmp(2) = ga_create_atom_blocked(geom, basis,
 42 >      $                                      'dft_roks_fock: tmp2')
 43 >           g_tmp(3) = ga_create_atom_blocked(geom, basis,
 44 >      $                                      'dft_roks_fock: tmp3')
 45 >           call ga_zero(g_tmp(2))
 46 >           call ga_zero(g_tmp(3))
 47 >         endif
 48 330d338
 49 <         call ga_zero(g_tmp(1))
 50 339c347
 51 <      $     tol2e, oskel, iv_dens, g_tmp(1), .false., .false.)
 52 ---
 53 >      $     tol2e, oskel, iv_dens, g_tmp, .false., .false.)
 54 340a349,352
 55 >         if (nopen.gt.0) then
 56 >           call ga_dadd(1d0,iv_fock(2),1d0,g_tmp(2),iv_fock(2))
 57 >           call ga_dadd(1d0,iv_fock(3),1d0,g_tmp(3),iv_fock(3))
 58 >         endif
 59 344a357,360
 60 >         if (nopen.gt.0) then
 61 >           call ga_zero(g_tmp(2))
 62 >           call ga_zero(g_tmp(3))
 63 >         endif
 64 353c369
 65 <      $     tol2e, oskel, iv_dens, g_tmp(1), .false., .true.)
 66 ---
 67 >      $     tol2e, oskel, iv_dens, g_tmp, .false., .true.)
 68 354a371,374
 69 >         if (nopen.gt.0) then
 70 >           call ga_dadd(1d0,iv_fock(2),1d0,g_tmp(2),iv_fock(2))
 71 >           call ga_dadd(1d0,iv_fock(3),1d0,g_tmp(3),iv_fock(3))
 72 >         endif
 73 358a379,385
 74 >         if (nopen.gt.0) then
 75 >           if (.not. ga_destroy(g_tmp(2))) call errquit
 76 >      $               ('xc_getv: ga corrupt?',0, GA_ERR)
 77 >           if (.not. ga_destroy(g_tmp(3))) call errquit
 78 >      $               ('xc_getv: ga corrupt?',0, GA_ERR)
 79 >         endif
 80 > c
 81 383c410
 82 <       etwo = etwo_closed + etwo_open + edisp
 83 ---
 84 >       etwo = etwo_closed + etwo_open + Edisp
 85 diff -r nwchem-6.3-src.2013-05-17/src/tools/GNUmakefile nwchem-6.3-src.2013-05-28/src/tools/GNUmakefile
 86 342a343,346
 87 > ifeq ($(ARMCI_NETWORK),SOCKETS)
 88 >         MAYBE_ARMCI = --with-sockets
 89 > endif # SOCKETS
 90 >
 91 Only in nwchem-6.3-src.2013-05-28/src/util: util_ga_version.F
 92 diff -r nwchem-6.3-src.2013-05-17/src/util/util_nwchem_version.F nwchem-6.3-src.2013-05-28/src/util/util_nwchem_version.F
 93 4c4
 94 <       nwrev="24252"
 95 ---
 96 >       nwrev="24277"

The section in red is the Iswtch.patch in http://verahill.blogspot.com.au/2013/05/424-nwchem-63-on-debian-wheezy.html.

I'm guessing that this patch is also included: http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id840/rodft_and_range_separated-functi....html
I've indicated it in blue.

Purple is probably to make sure that SOCKETS gets implemented correctly -- there were issues with that before.

I f I were to guess I'd say that if you compiled nwchem as shown here: http://verahill.blogspot.com.au/2013/05/424-nwchem-63-on-debian-wheezy.html
AND if you are not going to use CAM with open-shell molecules, then you are not in a hurry to recompile.

The former issue is manifested in slow run times, while the second issue is manifested in crashing calculations, so neither should be able to fly under the radar.

Anyway

The GabEdit Patch
The following patch allows you to use GabEdit as a GUI to nwchem, and allows you to compile nwchem with python support.
Copy the following and paste it into a file, e.g. 6.3.patch, and put it into /opt/nwchem/nwchem-6.3-src.2013-05-28:

diff -rupN src.original/config/makefile.h src/config/makefile.h
--- src.original/config/makefile.h      2013-04-15 12:41:45.016853322 +1000
+++ src/config/makefile.h       2013-04-15 12:38:44.933319544 +1000
@@ -2039,7 +2039,7 @@ endif
 
      ifeq ($(BUILDING_PYTHON),python)
 #   EXTRA_LIBS += -ltk -ltcl -L/usr/X11R6/lib -lX11 -ldl
-     EXTRA_LIBS +=    -lnwcutil  -lpthread -lutil -ldl
+     EXTRA_LIBS +=    -lnwcutil  -lpthread -lutil -ldl -lssl -lz
   LDOPTIONS = -Wl,--export-dynamic 
      endif
 ifeq ($(NWCHEM_TARGET),CATAMOUNT)
diff -rupN src.original/ddscf/movecs_pr_anal.F src/ddscf/movecs_pr_anal.F
--- src.original/ddscf/movecs_pr_anal.F 2013-04-15 12:41:45.036852381 +1000
+++ src/ddscf/movecs_pr_anal.F  2013-04-15 12:23:28.100409225 +1000
@@ -195,7 +195,7 @@ c
  22         format(1x,2('  Bfn.  Coefficient  Atom+Function  ',5x))
             write(LuOut,23)
  23         format(1x,2(' ----- ------------  ---------------',5x))
-            do klo = 0, min(n-1,9), 2
+            do klo = 0, min(n-1,199), 2
                khi = min(klo+1,n-1)
                write(LuOut,2) (
      $              int_mb(k_list+k)+1, 
diff -rupN src.original/ddscf/rohf.F src/ddscf/rohf.F
--- src.original/ddscf/rohf.F   2013-04-15 12:41:45.036852381 +1000
+++ src/ddscf/rohf.F    2013-04-15 12:23:28.100409225 +1000
@@ -153,7 +153,7 @@ c
             ilo = 1
             ihi = nmo
          endif
-         call movecs_print_anal(basis, ilo, ihi, 0.15d0, g_movecs, 
+         call movecs_print_anal(basis, ilo, ihi, 0.01d0, g_movecs, 
      $        'ROHF Final Molecular Orbital Analysis', 
      $        .true., dbl_mb(k_eval), oadapt, int_mb(k_irs),
      $        .true., dbl_mb(k_occ))
diff -rupN src.original/ddscf/scf_vec_guess.F src/ddscf/scf_vec_guess.F
--- src.original/ddscf/scf_vec_guess.F  2013-04-15 12:41:45.036852381 +1000
+++ src/ddscf/scf_vec_guess.F   2013-04-15 12:23:28.100409225 +1000
@@ -511,19 +511,19 @@ c
          nprint = min(nclosed+nopen+30,nmo)
          if (scftype.eq.'RHF' .or. scftype.eq.'ROHF') then
             call movecs_print_anal(basis, 1,
-     &           nprint, 0.15d0, g_movecs, 
+     &           nprint, 0.01d0, g_movecs, 
      &           'ROHF Initial Molecular Orbital Analysis', 
      &           .true., dbl_mb(k_eval), oadapt, int_mb(k_irs),
      &           .true., dbl_mb(k_occ))
          else
             nprint = min(nalpha+20,nmo)
             call movecs_print_anal(basis, max(1,nbeta-20),
-     &           nprint, 0.15d0, g_movecs, 
+     &           nprint, 0.01d0, g_movecs, 
      &           'UHF Initial Alpha Molecular Orbital Analysis', 
      &           .true., dbl_mb(k_eval), oadapt, int_mb(k_irs),
      &           .true., dbl_mb(k_occ))
             call movecs_print_anal(basis, max(1,nbeta-20),
-     &           nprint, 0.15d0, g_movecs(2), 
+     &           nprint, 0.01d0, g_movecs(2), 
      &           'UHF Initial Beta Molecular Orbital Analysis', 
      &           .true., dbl_mb(k_eval+nbf), oadapt, int_mb(k_irs+nmo),
      &           .true., dbl_mb(k_occ+nbf))
diff -rupN src.original/ddscf/uhf.F src/ddscf/uhf.F
--- src.original/ddscf/uhf.F    2013-04-15 12:41:45.036852381 +1000
+++ src/ddscf/uhf.F     2013-04-15 12:23:28.096409414 +1000
@@ -144,11 +144,11 @@ C
          enddo
          ihi = max(ihi-1,1)
  9611    continue
-         call movecs_print_anal(basis, ilo, ihi, 0.15d0, g_movecs, 
+         call movecs_print_anal(basis, ilo, ihi, 0.01d0, g_movecs, 
      $        'UHF Final Alpha Molecular Orbital Analysis', 
      $        .true., dbl_mb(k_eval), oadapt, int_mb(k_irs),
      $        .true., dbl_mb(k_occ))
-         call movecs_print_anal(basis, ilo, ihi, 0.15d0, g_movecs(2), 
+         call movecs_print_anal(basis, ilo, ihi, 0.01d0, g_movecs(2), 
      $        'UHF Final Beta Molecular Orbital Analysis', 
      $        .true., dbl_mb(k_eval+nbf), oadapt, int_mb(k_irs+nmo),
      $        .true., dbl_mb(k_occ+nbf))
diff -rupN src.original/mcscf/mcscf.F src/mcscf/mcscf.F
--- src.original/mcscf/mcscf.F  2013-04-15 12:41:45.000854073 +1000
+++ src/mcscf/mcscf.F   2013-04-15 12:23:23.748613695 +1000
@@ -719,7 +719,7 @@ c
       if (util_print('final vectors analysis', print_default))
      $     call movecs_print_anal(basis, 
      $     max(1,nclosed-10), min(nbf,nclosed+nact+10),
-     $     0.15d0, g_movecs, 'Analysis of MCSCF natural orbitals',
+     $     0.01d0, g_movecs, 'Analysis of MCSCF natural orbitals',
      $     .true., dbl_mb(k_evals), .true., int_mb(k_sym), 
      $     .true., dbl_mb(k_occ))
 c     
diff -rupN src.original/nwdft/scf_dft/dft_mxspin_ovlp.F src/nwdft/scf_dft/dft_mxspin_ovlp.F
--- src.original/nwdft/scf_dft/dft_mxspin_ovlp.F        2013-04-15 12:41:45.604825677 +1000
+++ src/nwdft/scf_dft/dft_mxspin_ovlp.F 2013-04-15 12:23:28.228403211 +1000
@@ -184,14 +184,14 @@ c
       call ga_sync()
 c
       call movecs_print_anal(basis,int_mb(k_non),int_mb(k_non)
-     & ,0.15d0,g_alpha,'Alpha Orbitals without Beta Partners',
+     & ,0.01d0,g_alpha,'Alpha Orbitals without Beta Partners',
      &   .false., 0.0 ,.false., 0 , .false., 0 )
 c
       if (nct.GE.2) then
       do i = 2,nct
       ind = int_mb(k_non+i-1)
       call movecs_print_anal(basis,ind,ind
-     & ,0.15d0,g_alpha,' ',
+     & ,0.01d0,g_alpha,' ',
      &   .false., 0.0 ,.false., 0 , .false., 0 )
       enddo
       endif
@@ -350,7 +350,7 @@ c      endif
 c      endif
 c 9990 format(/,18x,'THERE ARE',i3,1x,'UN-PARTNERED ALPHA ORBITALS')
 c
-       call movecs_print_anal(basis, 1, nalp, 0.15d0, g_ualpha,
+       call movecs_print_anal(basis, 1, nalp, 0.01d0, g_ualpha,
      & 'Alpha Orb. w/o Beta Partners (after maxim. alpha/beta overlap)',
      &   .false., 0.0 ,.false., 0 , .false., 0 )
 c
diff -rupN src.original/nwdft/scf_dft/dft_scf.F src/nwdft/scf_dft/dft_scf.F
--- src.original/nwdft/scf_dft/dft_scf.F        2013-04-15 12:41:45.608825490 +1000
+++ src/nwdft/scf_dft/dft_scf.F 2013-04-15 12:23:28.228403211 +1000
@@ -1774,7 +1774,7 @@ c
             else
                blob='DFT Final Beta Molecular Orbital Analysis' 
             endif
-            call movecs_print_anal(ao_bas_han, ilo, ihi, 0.15d0, 
+            call movecs_print_anal(ao_bas_han, ilo, ihi, 0.01d0, 
      &           g_movecs(ispin), 
      &           blob, 
      &           .true., dbl_mb(k_eval(ispin)), oadapt, 
diff -rupN src.original/nwdft/scf_dft_cg/dft_cg_solve.F src/nwdft/scf_dft_cg/dft_cg_solve.F
--- src.original/nwdft/scf_dft_cg/dft_cg_solve.F        2013-04-15 12:41:45.612825303 +1000
+++ src/nwdft/scf_dft_cg/dft_cg_solve.F 2013-04-15 12:23:28.220403588 +1000
@@ -183,7 +183,7 @@ c
             blob = 'DFT Final Beta Molecular Orbital Analysis'
           endif
           call movecs_fix_phase(g_movecs(ispin))
-          call movecs_print_anal(basis, ilo, ihi, 0.15d0,
+          call movecs_print_anal(basis, ilo, ihi, 0.01d0,
      &         g_movecs(ispin),blob,
      &         .true., dbl_mb(k_eval+(ispin-1)*nbf),
      &         oadapt, int_mb(k_irs+(ispin-1)*nbf),




Compiling nwchem
First install ACML or OpenBlas:
http://verahill.blogspot.com.au/2013/05/423-openblas-on-debian-wheezy.html
http://verahill.blogspot.com.au/2013/05/422-set-up-acml-on-linux.html


sudo apt-get install build-essential gfortran python2.7-dev libopenmpi-dev openmpi-bin
sudo mkdir /opt/nwchem -p
sudo chown $USER:$USER /opt/nwchem
cd /opt/nwchem
wget http://www.nwchem-sw.org/download.php?f=Nwchem-6.3.revision1-src.2013-05-28.tar.gz -O Nwchem-6.3.revision1-src.2013-05-28.tar.gz
tar xvf Nwchem-6.3.revision1-src.2013-05-28.tar.gz
cd nwchem-6.3-src.2013-05-28/

Apply the Gabedit patch:
patch -p0 < 6.3.patch 
patching file src/config/makefile.h patching file src/ddscf/movecs_pr_anal.F patching file src/ddscf/rohf.F patching file src/ddscf/scf_vec_guess.F patching file src/ddscf/uhf.F patching file src/mcscf/mcscf.F patching file src/nwdft/scf_dft/dft_mxspin_ovlp.F patching file src/nwdft/scf_dft/dft_scf.F patching file src/nwdft/scf_dft_cg/dft_cg_solve.F

export NWCHEM_TOP=`pwd`
export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export PYTHONVERSION=2.7
export PYTHONHOME=/usr
export BLASOPT="-L/opt/acml/acml5.3.1/gfortran64_int64/lib -lacml"
#export BLASOPT="-L/opt/openblas/lib -lopenblas"

export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export LIBRARY_PATH="$LIBRARY_PATH:/usr/lib/openmpi/lib:/opt/acml/acml5.3.1/gfortran64_int64/lib"
#export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openmpi/lib:/opt/openblas/lib

export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"
export ARMCI_NETWORK=SOCKETS

cd $NWCHEM_TOP/src

make clean
make nwchem_config
make FC=gfortran 1> make.log 2>make.err

cd $NWCHEM_TOP/contrib
export FC=gfortran
./getmem.nwchem

Comment out the bold parts and uncomment the commented parts to compile with openblas instead of ACML e.g. if you're compiling on an intel machine instead of AMD.

Setting up your ~/.bashrc
As usual, do
echo 'export NWCHEM_EXECUTABLE=/opt/nwchem/nwchem-6.3-src.2013-05-28/bin/LINUX64/nwchem' >> ~/.bashrc
echo 'export NWCHEM_BASIS_LIBRARY=/opt/nwchem/nwchem-6.3-src.2013-05-28/src/basis/libraries/' >> ~/.bashrc
echo 'export PATH=$PATH:/opt/nwchem/nwchem-6.3-src.2013-05-28/bin/LINUX64' >> ~/.bashrc

Also, put a .nwchemrc in your home folder:
nwchem_basis_library /opt/nwchem/nwchem-6.3-src.2013-05-28/src/basis/libraries/
ffield amber
amber_1 /opt/nwchem/nwchem-6.3-src.2013-05-28/src/data/amber_s/
amber_2 /opt/nwchem/nwchem-6.3-src.2013-05-28/src/data/amber_x/
amber_3 /opt/nwchem/nwchem-6.3-src.2013-05-28/src/data/amber_q/
amber_4 /opt/nwchem/nwchem-6.3-src.2013-05-28/src/data/amber_u/
amber_5 /opt/nwchem/nwchem-6.3-src.2013-05-28/src/data/custom/
spce /opt/nwchem/nwchem-6.3-src.2013-05-28/src/data/solvents/spce.rst
charmm_s /opt/nwchem/nwchem-6.3-src.2013-05-28/src/data/charmm_s/
charmm_x /opt/nwchem/nwchem-6.3-src.2013-05-28/src/data/charmm_x/

You're done.

448. Metal-pi bonds in ECCE

It should be easy, since it's one of the implemented bond types in ECCE together with single, double, triple and pi-arene bonds, but it doesn't function quite as advertised (supposed to ignore nubs).

I think it might be a bug, but given the very few resources currently given to ECCE, it's better to develop work-arounds than to demand quick bug fixes.

A reasonable criticism of this post would be that it really doesn't matter in the long run though, as the bonds that we make here are eye-candy only -- DFT doesn't 'know' what bonds are in the sense of the ball-and-stick model. On the other hand, ECCE will complain loudly if there's a discrepancy between oxidation states, multiplicity and number of vacant sites.

In the end, if you have a crystal structure to start from, use that. If not, then this is a fairly simple way of yielding a visually pleasing and reasonably intuitive model.

Anyway, let's use Noyori's catalyst as an example:


Start ECCE, set up a new job and click on Builder.

Click on Import from Structure Library, select Alicycles, and click on Cyclooctane, then click in the main window.


Delete one hydrogen on each of the sp2 carbons, and change the bond type to double bond:

 Change the sp2 carbons to TrigonalPlanar, and Clean (click the broom which does MM (UFF) geometry optimization).

Insert an octahedral iridium atom -- you'll need two 'nubs' for each pi bond, and two nubs for the two phosphine ligands.

 Change the sp2 carbons back to tetrahedral so you get 'nubs' to bond to (not supposed to be necessary according to the manual, but it is). Make Metal-pi bonds.
 Make sure all the bonds (check the first one in particular) are Metal-Pi bonds, and Clean the structure (i.e. MM/UFF optimization)


 Insert two tetrahedral phosphorous atoms, and make single bonds
Click on Add H if you want PH3 ligands, and clean the structure (i.e. optimize geo with MM/UFF):


At this point the structure isn't quite perfectly square planar like we would like it to be. The easiest way to sort that out is to set the torsional angles between the H-C(=)-Ir-P to 0 (uncheck rotate group around bond) for each P, then Clean. The result isn't perfect, but still not too shabby:

447. Multiuser ECCE

A recent comment lead me to make this post:
[..] Can you advise a correct way of setting up ECCE so that everyone can have an individual account and run on the same machine?

It's a valid question. I haven't had any need for it since I'm running my own cluster, which I currently don't share with anyone. I also manage a multi-user cluster overseas, but I am the only one using ECCE, and so I'm fine running the ECCE server here.

Anyway. I built ECCE 6.4 as shown e.g. here and here.

Below I install the ECCE server as the user verahill, and then install the client software as the users lindqvst and me. In the former case I configure the server using a copied config_from_server script, and in the latter case I use config_with_server. Both methods worked fine. All users are on the same machine, but by editing apps/siteconfig/DataServers you can quite easily configure the ECCE client to connect to a remote server.


Server:

hostname
helium
sudo adduser verahill sudo adduser lindqvst ./install_ecce.v6.4.csh
Main ECCE installation menu =========================== 1) Help on main menu options 2) Prerequisite software check 3) Full install 4) Full upgrade 5) Application software install 6) Application software upgrade 7) Server install 8) Server upgrade IMPORTANT: If you are uncertain about any aspect of installing or running ECCE at your site, please refer to the detailed ECCE Installation and Administration Guide at http://ecce.pnl.gov/docs/installation/2864B-Installation.pdf Hit at prompts to accept the default value in brackets. Selection: [1] 7 Host name: [helium] Server installation directory: [/home/verahill/ecce-v6.4/server] Enter the path to the ECCE application software directory that will use this server (even if this directory is not accessible from the current machine) or [return] if you have not installed application software yet or don't want to update it: ECCE v6.4 will be installed using the settings: Installation type: [server install] Host name: [helium] Server installation directory: [/home/verahill/ecce-v6.4/server] Are these choices correct (yes/no/quit)? [yes] ECCE installation succeeded. *************************************************************** !! You MUST perform the following steps in order to use ECCE !! -- Transfer the script: /home/verahill/ecce-v6.4/server/ecce-admin/config_from_server to the machine where the application software is installed. Run this script as the same user as the application software installation. (This step can be skipped if you have not installed the application software and the server name is specified during that install) -- Start the ECCE server as 'verahill' by running: /home/verahill/ecce-v6.4/server/ecce-admin/start_ecce_server ***************************************************************
cp /home/verahill/ecce-v6.4/server/ecce-admin/config_from_server ~ chmod ugo+r config_from_server


First client
su lindqvst
cd ~
cp /home/verahill/config_from_server .
cp /home/verahill/install_ecce.v6.4.csh .
./install_ecce.v6.4.csh
Main ECCE installation menu =========================== 1) Help on main menu options 2) Prerequisite software check 3) Full install 4) Full upgrade 5) Application software install 6) Application software upgrade 7) Server install 8) Server upgrade IMPORTANT: If you are uncertain about any aspect of installing or running ECCE at your site, please refer to the detailed ECCE Installation and Administration Guide at http://ecce.pnl.gov/docs/installation/2864B-Installation.pdf Hit at prompts to accept the default value in brackets. Selection: [1] 5 Host name: [helium] Server installation directory: [/home/lindqvst/ecce-v6.4/apps] ECCE installation succeeded. *************************************************************** !! You MUST perform the following steps in order to use ECCE !! -- Configure the application software to use the desired ECCE server by running the script: /home/lindqvst/ecce-v6.4/apps/scripts/config_with_server as the same user as the application software installation. (This step can be skipped if you prefer to copy over and run the config_from_server script created during the server installation in the ecce-admin directory) -- To register machines to run computational codes, please see the installation and compute resource registration manuals at http://ecce.pnl.gov/using/installguide.shtml -- Before running ECCE each user must source an environment setup script. For csh/tcsh users add this to ~/.cshrc: if ( -e /home/lindqvst/ecce-v6.4/apps/scripts/runtime_setup ) then source /home/lindqvst/ecce-v6.4/apps/scripts/runtime_setup endif For sh/bash users, add this to ~/.profile or ~/.bashrc: if [ -e /home/lindqvst/ecce-v6.4/apps/scripts/runtime_setup.sh ]; then . /home/lindqvst/ecce-v6.4/apps/scripts/runtime_setup.sh fi ***************************************************************
Next, configure:
./config_from_server
Enter the ECCE application software installation home directory: /home/lindqvst/ecce-v6.4/apps ECCE application software home directory is /home/lindqvst/ecce-v6.4/apps Is this correct? [yes] Adding data server URL for helium to siteconfig/DataServers Adding URL for online help to siteconfig/site_runtime Adding message server URL for helium to siteconfig/jndi.properti
And then do
echo 'export ECCE_HOME=/home/lindqvst/ecce-v6.4/apps' >> ~/.bashrc
echo 'export PATH=${ECCE_HOME}/scripts:${ECCE_HOME}/scripts/parsers:${PATH}' >> ~/.bashrc


Second client:
I set up a third user, installed ECCE as shown above for lindqvst, but instead of running config_from_server
I did:
/home/me/tmp/ecce-v6.4/ecce-v6.4/apps/scripts/config_with_server
Enter the full host name of the machine where the ECCE server is installed: helium ECCE server host name is helium Is this correct? [yes]
And then do
echo 'export ECCE_HOME=/home/me/ecce-v6.4/apps' >> ~/.bashrc
echo 'export PATH=${ECCE_HOME}/scripts:${ECCE_HOME}/scripts/parsers:${PATH}' >> ~/.bashrc
source ~/.bashrc


Tying it all together:

Start the server: As user verahill, start the server.
su verahill
/home/verahill/ecce-v6.4/server/ecce-admin/start_ecce_server
/home/verahill/ecce-v6.4/server/httpd/bin/apachectl start: httpd started [1] 13951 INFO BrokerService - ActiveMQ 5.1.0 JMS Message Broker (localhost) is starting INFO BrokerService - ActiveMQ JMS Message Broker (localhost, ID:helium-39358-1370944321599-0:0) started
ps a|grep 13951
13951 pts/3 S 0:00 grep -v -e ACTIVEMQ -e Loading -e AMQ -e Kaha -e help -e Transport
You should be able to autostart ecce using something along the lines of the following in your /etc/rc.local:
su verahill '/home/verahill/ecce-v6.4/server/ecce-admin/start_ecce_server' &
As a user, start ecce:
su lindqvst
/home/lindqvst/ecce-v6.4/apps/scripts/ecce
Create a password

Create a job
Then log out and log in as another user, e.g. me

su me
ecce
me is in reality called something else
Note that as user me you can't see any of the files in lindqvst's folder -- nor access them without inputting the correct (ECCE) username and password.

And again, note that there are alternative ways of setting this up -- you could have everyone log in as the same linux user, but still retain different ECCE identities. I think what I show here is more in line with how most people would want to use ECCE though.

Also, note that you can quite easily be a client on a different computer too -- the key lies in editing the apps/siteconfig/DataServers file.

11 June 2013

446. B3LYP and WAH -- the confusion

Quite a while back I was looking at the WAH (Wilson-Amos-Handy) functional, and while it turned out to be a bit more complicated than I had hoped, it led me to type up a brief discussion about b3lyp in different computational packages.

The issue is that there are several different definitions, and that even if a paper is kind enough to provide the Becke 1993 communication as a reference, this is rarely the actual form of b3lyp used. In fact, the LYP part would speak directly against it. As someone whom isn't well-versed in the computational and theoretical arts I do think it would be nice if we could get to the point where we can get useful information by doing point-and-click computations, but as exchange-correlation functionals essentially are fudge-factors, this probably won't happen for some time.  In other words, 6-31G/B3LYP may be a winning combination for the computation of electronic energies of a limited range of (mostly organic) small molecules in the gas phase, it doesn't always yield anything useful about real-world systems.

NOTE: I wrote the original text as I was trying to figure out what WAH was -- and I thought at that point that it was a simple form of B3PW91 with tweaked prefactors. It isn't -- it's requires changes in the way the GIAOs are computed (I think). So don't focus on the WAH discussion.

Anyway, here's the story as a bench chemist (i.e. not a computational or theoretical chemist) understands it, in the context of trying to understand what the WAH exchange-correlation functional looks like.

The WAH functional is a hybrid exchange correlation functional which was developed to provide accurate NMR shift calculations.

Note that the WAH functional is correctly implemented in PQS -- see the manual.

But the story is really about B3LYP...


1. Definition of the WAH exchange-correlation functional:
The definition[1] consists of
We found that by using hybrid Kohn-Sham orbitals and eigenvalues with an adjusted 'exact-exchange' coefficient Cx, the NMR shielding parameters gave an accuracy approaching the best coupled-cluster calculations for molecules containing first and second row atoms. For B3LYP[2] we find Cx=0.05[..] give(s) best values (note that the coefficient of Local Density Exchange is (1-Cx) in the amended B3LYP. We name the resulting NMR values B3LYPGGA0.05[..].
This is the paper which is cited by the PQS manual. The definition is brief, but not unreasonable.

2 Definition of B3LYP
[The first-person account of the background to B3LYP is found here: http://www.ccl.net/chemistry/resources/messages/2002/05/22.008-dir/]

In the paper which is cited as a source of B3LYP, Becke defined[2] a functional as
EXC=EXCLSDA+a0 (ExHF-ExLSDA )+axΔ ExBecke88+acΔ EcPW91   (eq 1)
where EXC denotes exchange-correlation functional, Ex denotes exchange functional, Ec denotes correlation functional, Δ denotes non-local contribution, LSDA is the local spin-density approximation, Becke88[3] is the gradient-corrected LDA and PW91 is the Perdew-Wang 1991 gradient correction.[4]. The B3 in B3LYP refers to the three parameters it involves: a0=0.2, ax=0.72 and ac=0.81. Note that a0 is the same as Cx above. We'll refer to equation 1 as B3PW91.

LSDA is poorly defined but is normally taken to be the SVWN of the form
EXCLSDA=EXLSDA+ECLSDA  (eq 2)
=ExSlater+EcVWN   (eq 3)
although there's a slew of Vosko-Wilk-Nusair (VWN) functionals -- most sources suggest that Becke referred to VWN5, while my reading of the literature is a bit different (Becke states he uses the electron-gas parametrization in [4]). Either way, equation 1 now becomes
EXC=a0 ExHF+(1-a0 )ExSlater+EcVWN +axΔ ExBecke88+acΔ EcPW91 (eq 4)
This is implemented as the hybrid exchange-correlation functional acm in NWChem (the adibatic connection method) using VWN5. A very brief summary of Becke '93 vs Gaussian '92 (don't ask me about the chronology) is also available by Stephens et al. in J. Phys. Chem. 1994, 98(45), p. 11624.

2.1 Gaussian '92
You may at this point be forgiven for asking yourself why it is called B3LYP and not B3PW91. In 1991 Gaussian hadn't yet implemented PW91 (fair enough) and substituted it with the Lee-Yang-Parr (LYP) correlation functional (ΔEcLYP). Since it's difficult to separate the local component (and we want the non-local component as indicated by Δ), they wrote
Δ EcLYP=EcLYP-EcVWN (eq 5)
which turns equation 4 into
EXC=a0 ExHF+(1-a0)ExSlater+axΔ ExBecke88+acEcLYP +(1-ac)EcVWN (eq 6)
In addition, in the original Gaussian implementation VWN_1_RPA was used, which sources tell me is 100\% wrong when taking Becke's intentions into account.
EXC=a0 ExHF+(1-a0)ExSlater +axΔ ExBecke88+acEcLYP +(1-ac)EcVWN_1_RPA (eq 7)
To make matters worse, today Gaussian uses VWN_3 and it seems they know how to get the nonlocal component of LYP directly (see below). Equation 7 is what you use if you use B3LYP in most software packages (though not all -- e.g. Gamess US uses VWN5 instead of VWN_1_RPA) So in G09 it's now
EXC=a0 ExHF+(1-a0)ExSlater+axΔ ExBecke88+EcVWN_3+acEcLYP (eq 8)

2.2 PQS
PQS uses the old gaussian version (eq. 7) as the b3lyp functional, but it doesn't explicitly state which form -- b3lyp or b3pw91 -- is used for the WAH functional.


3 So what definition did WAH use?
All we really care about is reproducing the original paper by Handy et al. -- not whether Becke would approve or not. But here's where the problem of citing papers you may not have read becomes an issue.

Wilson, Amos and Handy cite the 1993 paper by Becke which defines the canonical version of B3LYP (i.e. B3PW91), which should settle it in favour of WAH being defined as shown in equation 4.

However, they used CADPACK, which implements it as in equation 7. Reading the CADPACK manual I can see what Handy et al. probably did: the way you define custom parameters for hybrid functionals in CADPACK is by doing
hybrid a0 ax ac
so that they during the development of their functional most likely typed in
b3lyp 0.05 0.72 0.81
which meant they probably used
EXC=0.05 ExHF+0.95 ExSlater+ +0.72 Δ ExBecke88+0.81 EcLYP +0.19 EcVWN_1_RPA (eq 9)

4 Implementing it in your package of choice
[NOTE: this will NOT set up WAH correctly -- I'm leaving it as it shows how to set up custom XCs in nwchem, G09 and Dalton]

4.1 NWCHEM
The canonical version of Becke's functional, B3PW91, is implemented as acm in NWChem and which is manually entered as
xc HFexch 0.2 slater 0.8 becke88 nonlocal 0.72 vwn_5 1 Perdew91 0.81
while the Gaussian '92 form is manually entered as
xc HFexch 0.2 slater 0.8 becke88 nonlocal 0.72 vwn_1_rpa 0.19 lyp 0.81
This means that the two possible forms of WAH are:
xc HFexch 0.05 slater 0.95 becke88 nonlocal 0.72 vwn_5 1 Perdew91 0.81
and
 xc HFexch 0.05 slater 0.95 becke88 nonlocal 0.72 vwn_1_rpa 0.19 lyp 0.81
We'll refer to them as B3PW910.05 and B3LYP0.05, respectively.

4.2 Gaussian 09
Gaussian is a lot less elegant. The canonical version of Becke's functional, B3PW91, is implemented as acm in Gaussian as B3PW91, which is manually entered as
BPW91 IOp(3/76=1000002000) IOp(3/77=0720008000) IOp(3/78=0810010000)
while the old Gaussian form is manually entered as
BLYP IOp(3/76=1000002000) IOp(3/77=0720008000) IOp(3/78=0810001900
This means that the two forms of WAH are:
BPW91 IOp(3/76=1000000500) IOp(3/77=0720009500) IOp(3/78=0810010000)
and
BLYP IOp(3/76=1000000500) IOp(3/77=0720009500) IOp(3/78=0810001900)
We'll refer to them as B3PW910.05 and B3LYP0.05, respectively.

4.3 Dalton
The notations are (more or less)
Combine HF=0.20 Slater=0.80 Becke=0.72 PW91c=0.81 VWN5=1
Combine HF=0.20 Slater=0.80 Becke=0.72 LYP=0.81 VWN=0.19
Combine HF=0.05 Slater=0.95 Becke=0.72 PW91c=0.81 VWN5=1
Combine HF=0.05 Slater=0.95 Becke=0.72 LYP=0.81 VWN=0.19
As an aside, I don't think anyone at this point would be surprised to learn that B3PW91 in Dalton and Gaussian are two completely different exchange-correlation functionals...

Performance
NOTE: the results don't quite make any sense to me anymore -- using the same XCs and basis sets and structures one would expect to get the exact same results for all three packages. I don't know why that wasn't the case, assuming that I implemented the XCs correctly, and assuming that the basis sets really are the same (which often they actually aren't...). So keep that in mind.

Original:
I used the same equilibrium structures as Handy (i.e. those of Cybulski), and used the aug-cc-pVTZ basis set with a fine DFT grid (not the same as Handy -- but should be as good. Also, I've done this with def2-qzvp and 6-31+g* as well). All calculations are for the gas phase. Handy's values are for the Huzinaga IV basis set in their GIAO paper[5], but it doesn't really matter much which basis set is chosen. The results are tabulated in table 1.

Note that Handy also saw chemical shifts for the oxygen in carbon monoxide which were around -80 ppm. Basically, I can reproduce everything except for his B3LYP0.05gga. I did a few test-runs with Huz-IV in Dalton and it was just as bad as the other methods.

Table 1: Tabulated calculated gas phase NMR shifts using different combinations of exchange-correlation functionals and the aug-cc-pvtz basis set. Note that Handy's experimental data are not in agreement with my sources, which are 1.0, -42.3 and 344 ppm for CO, CO and H2O, respectively.
Package b3lypa gb3lypb acmc b3pw91b b3lyp0.05 b3pw910.05 Handy
Expt.
Carbon in CO (ppm)
NWChem -10.32 -10.32 -8.80 -8.80 -8.08 -6.55 5.6 2.8
G09 -7.71 -7.71 -9.95 -9.95 -5.66 -7.74
Dalton -10.21 -10.21 N/A -12.65 -7.97 -10.38
Oxygen in CO (ppm)
NWChem -71.51 -71.51 -72.34 -72.34 -70.06 -70.93 -40 -36.7
G09 -78.82 -78.82 -77.89 -77.89 -77.57 -76.86
Dalton -71.50 -71.50 N/A -72.03 -70.04 -70.89
Oxygen in water
NWChem 328.45 328.45 329.28 329.28 329.79 330.48 327 358
G09 N/A 328.76 328.39 328.39 328.95 328.31
Dalton 328.50 328.50 N/A 328.02 329.85 329.07
aNote that G09 uses eq. 8 by default (gives -11.70, -77.63, and 328.34 ppm), but I forced it to use eq. 7. b These were my manual implementations of 'b3lyp' and 'acm' to make sure I got things right. c This is acm in NWChem and B3PW91 in Gaussian 09.


References:
[1] P. J. Wilson, R. D. Amos, H. N. C., Chem. Phys. Letters 1999, 321, 475-484.
[2] A. D. Becke, J. Chem. Phys. 1993, 98, 5648-5652.
[3] A. D. Becke, Phys. Rev. A 1988, 38, 3098-3100.
[4] J. P. Perdew, Y. Wang, Phys. Rev. B 1991, 45, 13244-13249.
[5] T. Helgaker, P. J. Wilson, R. D. Amos, N. C. Handy, J. Chem. Phys. 2000, 113, 2983-2989.

445. GAMESS US 2013 R1 on Debian (Wheezy) -- w/o GPU

Update 27/6/2013:
Please note that Kirill Berezovsky has published a series of posts on GAMESS US, including how to compile it for both CPU and GPU use. See
http://biochemicalmatters.blogspot.com.au/2013/06/gamess-us-frequently-asked-questions_26.html
http://biochemicalmatters.blogspot.ru/2013/06/gamess-us-frequently-asked-questions_1687.html
http://biochemicalmatters.blogspot.ru/2013/06/gamess-us-frequently-asked-questions_1447.html
http://biochemicalmatters.blogspot.com.au/2013/06/gamess-us-frequently-asked-questions.html


Original post:
A new version of GAMESS is out now (2013 R1): https://groups.google.com/forum/?fromgroups#!topic/gamess-announce/8j1esKifzEo

GPU support will be a later post.

0. Install a math library
You can use e.g. acml or atlas. See http://verahill.blogspot.com.au/2013/05/422-set-up-acml-on-linux.html for acml (which I've only had luck with on AMD machines).

To get the debian ATLAS libs do
sudo apt-get install libatlas3-base libatlas-dev

If you want to compile your own ATLAS libs, see e.g. http://verahill.blogspot.com.au/2012/09/rocks-543-atlas-and-gromacs-on-xeon.html or http://verahill.blogspot.com.au/2012/09/compile-atlas-gromacs-nwchem-on-amd-fx.html

You can also link to openblas, as shown in this post: http://verahill.blogspot.com.au/2012/09/compiling-and-testing-gamess-us-on.html


1. Get GAMESS US
Go to http://www.msg.chem.iastate.edu/GAMESS/download/register/
Check the tick boxes next to the architectures you intend to use GAMESS on. Fill out your email address, and hit Submit. You'll receive an email with instructions almost immediately.

The email will contain a URL to an archive with the source code, and a password for downloading it.
I'll presume that you downloaded the file, gamess-current.tar.gz, to ~/Downloads.


2. Untar and prepare
I'm presuming that you don't already have any /opt/gamess or /opt/gamess/gamess directories. In my earlier write-ups (e.g. this) I put everything in /opt/gamess, which isn't a good long-term strategy since you often want to keep earlier versions of computational software alongside newer ones.

sudo apt-get install build-essential gfortran openmpi-bin libopenmpi-dev libboost-all-dev
sudo mkdir /opt/gamess -p
sudo chown $USER:$USER /opt/gamess
cd /opt/gamess
cp ~/Downloads/gamess-current.tar.gz gamess-2013r1.tar.gz
tar xvf gamess-2013r1.tar.gz
mv gamess gamess-2013r1
cd gamess-2013r1/


3. Configure
NOTE: Even if you may have gfortran 4.7 (e.g. Wheezy) you should give the version as 4.6 during configure (see below).


./config
please enter your target machine name: linux64 GAMESS directory? [/opt/gamess/gamess-2013r1] GAMESS build directory? [/opt/gamess/gamess-2013r1] Version? [00] 13 Please enter your choice of FORTRAN: gfortran Please enter only the first decimal place, such as 4.1 or 4.6: 4.6
ACML:
Enter your choice of 'mkl' or 'atlas' or 'acml' or 'none': acml enter this full pathname: /opt/acml/acml5.3.1 Math library 'acml' will be taken from /opt/acml/acml5.3.1/gfortran64_int64/lib
Atlas:
I've compiled ATLAS myself in the past, but the libs have not worked with all programs. Here we use the debian libs instead.
Enter your choice of 'mkl' or 'atlas' or 'acml' or 'none': atlas Please enter the Atlas subdirectory on your system: /usr/lib/atlas-base Math library 'atlas' will be taken from /usr/lib/atlas-base
mpi:
communication library ('sockets' or 'mpi')? mpi Enter MPI library (impi, mvapich2, mpt, sockets): openmpi Please enter your openmpi's location: /usr/lib/openmpi

4. Build
Edit comp and change
1663 # -fno-whole-file suppresses argument's data type checking 1664 set OPT='-O2' 1665 if (".$GMS_DEBUG_FLAGS" != .) set OPT="$GMS_DEBUG_FLAGS"
to
1663 # -fno-whole-file suppresses argument's data type checking 1664 set OPT='-O0' 1665 if (".$GMS_DEBUG_FLAGS" != .) set OPT="$GMS_DEBUG_FLAGS"
or exam44.inp will fail.

cd ddi/
./compddi
cd ../
./compall

If you are building with openmpi, edit lked and change
958 case openmpi: 959 set MPILIBS="-L$GMS_MPI_PATH/lib64" 960 set MPILIBS="$MPILIBS -lmpi" 961 breaksw
to
958 case openmpi: 959 set MPILIBS="-L$GMS_MPI_PATH/lib" 960 set MPILIBS="$MPILIBS -lmpi" 961 breaksw
Make the proper symlinks if you are using ATLAS:
sudo ln -s /usr/lib/atlas-base/libatlas.so.3 /usr/lib/atlas-base/libatlas.so
sudo ln -s /usr/lib/atlas-base/libf77blas.so.3 /usr/lib/atlas-base/libf77blas.so

./lked gamess 13r1 


I use my own script called gmrun:
#!/bin/csh
set TARGET=mpi
set SCR=$HOME/scratch
set USERSCR=/scratch
set GMSPATH=/opt/gamess/gamess-2013r1
set JOB=$1
set VERNO=$2
set NCPUS=$3

if ( $JOB:r.inp == $JOB ) set JOB=$JOB:r
echo "Copying input file $JOB.inp to your run's scratch directory..."
cp $JOB.inp $SCR/$JOB.F05

setenv TRAJECT $USERSCR/$JOB.trj
setenv RESTART $USERSCR/$JOB.rst
setenv INPUT $SCR/$JOB.F05
setenv PUNCH $USERSCR/$JOB.dat
if ( -e $TRAJECT ) rm $TRAJECT
if ( -e  $PUNCH ) rm $PUNCH
if ( -e  $RESTART ) rm $RESTART
source $GMSPATH/gms-files.csh

setenv LD_LIBRARY_PATH /usr/lib/openmpi/lib:$LD_LIBRARY_PATH
mpiexec -n $NCPUS $GMSPATH/gamess.$VERNO.x|tee $JOB.out
cp $PUNCH .

To run, do e.g.
gmrun exam01.inp 13r1 2

10 June 2013

444.Building Wine 1.6-rc1 on Debian Wheezy using a chroot

Here's a generic way of building wine which works for 1.6-rc1 (and 1.5.28 and everything in between except 1.5.30).

The build instructions are recycled from my 1.5-series instructions -- the 1.5 series is the development version and Wine 1.6 is the new stable release, and so Wine 1.5.31 and 1.6-rc1 should be pretty similar. As a release candidate (rc) 1.6-rc1 isn't meant for production work. For that, use Wine 1.4 which is the current stable series.


Here's a link to the release announcement: http://www.winehq.org/announce/1.6-rc1

See here for information about 3D acceleration using libGL/U with Wine: http://verahill.blogspot.com.au/2013/05/429-briefly-wine-libglliubglu-blender.html

Getting started:
If you set up a e.g. chroot to build 1.5.28 or  before or 1.5.31 (1.5.30 had a bug which needed patching), you don't need to set up a new chroot to build 1.6-rc1. In that case, skip the set-up step below and instead re-enter your existing chroot like this:
sudo mount -o bind /proc wine32/proc
sudo cp /etc/resolv.conf wine32/etc/resolv.conf
sudo chroot wine32
su sandbox
cd ~/tmp

And skip to 'Building wine'.

Otherwise do this:
Setting up the Chroot
sudo apt-get install debootstrap
mkdir $HOME/tmp/architectures/wine32 -p
cd $HOME/tmp/architectures
sudo debootstrap --arch i386 wheezy $HOME/tmp/architectures/wine32 http://ftp.au.debian.org/debian/
sudo mount -o bind /proc wine32/proc
sudo cp /etc/resolv.conf wine32/etc/resolv.conf
sudo chroot wine32

You're now in the chroot:
apt-get update
apt-get install locales sudo vim
echo 'export LC_ALL="C"'>>/etc/bash.bashrc
echo 'export LANG="C"'>>/etc/bash.bashrc
echo '127.0.0.1 localhost beryllium' >> /etc/hosts
source /etc/bash.bashrc
adduser sandbox
usermod -g sudo sandbox
echo 'Defaults !tty_tickets' >> /etc/sudoers
su sandbox
cd ~/

Replace 'beryllium' with the name your host system (it's just to suppress error messages)

Building Wine
While still in the chroot, continue (the i386 is ok; don't worry about it -- you don't actually need it):

sudo apt-get install libx11-dev:i386 libfreetype6-dev:i386 libxcursor-dev:i386 libxi-dev:i386 libxxf86vm-dev:i386 libxrandr-dev:i386 libxinerama-dev:i386 libxcomposite-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libdbus-1-dev:i386 libgnutls-dev:i386 libncurses-dev:i386 libsane-dev:i386 libv4l-dev:i386 libgphoto2-2-dev:i386 liblcms-dev:i386 libgstreamer-plugins-base0.10-dev:i386 libcapi20-dev:i386 libcups2-dev:i386 libfontconfig-dev:i386 libgsm1-dev:i386 libtiff-dev:i386 libpng-dev:i386 libjpeg-dev:i386 libmpg123-dev:i386 libopenal-dev:i386 libldap-dev:i386 libxrender-dev:i386 libxml2-dev:i386 libxslt-dev:i386 libhal-dev:i386 gettext:i386 prelink:i386 bzip2:i386 bison:i386 flex:i386 oss4-dev:i386 checkinstall:i386 ocl-icd-libopencl1:i386 opencl-headers:i386 libasound2-dev:i386 build-essential
mkdir ~/tmp
cd ~/tmp
wget http://prdownloads.sourceforge.net/wine/wine-1.6-rc1.tar.bz2

tar xvf wine-1.6-rc1.tar.bz2
cd wine-1.6-rc1/
./configure
time make -j3
sudo checkinstall --install=no
checkinstall 1.6.2, Copyright 2009 Felipe Eduardo Sanchez Diaz Duran This software is released under the GNU GPL. The package documentation directory ./doc-pak does not exist. Should I create a default set of package docs? [y]: Preparing package documentation...OK Please write a description for the package. End your description with an empty line or EOF. >> wine 1.6-rc1 >> ***************************************** **** Debian package creation selected *** ***************************************** This package will be built according to these values: 0 - Maintainer: [ root@beryllium ] 1 - Summary: [ wine 1.6-rc1] 2 - Name: [ wine-1.6 ] 3 - Version: [ 1.6-rc1] 4 - Release: [ 1 ] 5 - License: [ GPL ] 6 - Group: [ checkinstall ] 7 - Architecture: [ i386 ] 8 - Source location: [ wine-1.6-rc1 ] 9 - Alternate source location: [ ] 10 - Requires: [ ] 11 - Provides: [ wine-1.6 ] 12 - Conflicts: [ ] 13 - Replaces: [ ]
Checkinstall takes a little while (In particular this step: 'Copying files to the temporary directory...').

Installing Wine

Exit the chroot
sandbox@beryllium:~/tmp/wine-1.6-rc1$ exit
exit
root@beryllium:/# exit
exit
me@beryllium:~/tmp/architectures$ 

On your host system
 Enable multiarch* and install ia32-libs, since you've built a proper 32 bit binary:

sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install ia32-libs

*At some point I think ia32-libs may be replaced by proper multiarch packages, but maybe not. So we're kind of doing both here.

 Copy the .deb package and install it
sudo cp wine32/home/sandbox/tmp/wine-1.6-rc1/wine_1.6-rc1-1_i386.deb .
sudo chown $USER wine_1.6-rc1-1_i386.deb
sudo dpkg -i wine_1.6-rc1-1_i386.deb

443. Briefly: Running the QA tests in NWChem

To make sure that everything is working properly and that you get the expected results from your nwchem binaries, you should run the QA tests that come with nwchem.

Here's how to do it with the nwchem 6.3 QA tests.

I'm presuming that you built nwchem with mpi as shown e.g. here: http://verahill.blogspot.com.au/2013/05/424-nwchem-63-on-debian-wheezy.html

In this particular case I'm using nwchem linked with openblas on an AMD Phenom II 1055T with 8 Gb RAM since it was the only node that was free.

0. Go to the QA directories.
In my case everything is housed in /opt/nwchem/nwchem-6.3-src.patched and the QA tests are in /opt/nwchem/nwchem-6.3-src.patched/QA

1. Run the tests
First set the environmental variables, then start the tests. The 6 in './doqmtests.mpi 6' is the number of threads i.e. processors to use in parallel.

export NWCHEM_TOP=/opt/nwchem/nwchem-6.3-src.patched
export NWCHEM_TARGET=LINUX64
./doqmtests.mpi 6 |tee doqmtests.mpi.log
====================================================== QM: Running all tests (including some really big ones) ====================================================== Running tests/h2o_opt/h2o_opt cleaning scratch copying input and verified output files running nwchem (/opt/nwchem/nwchem-6.3-src.patched/bin/LINUX64/nwchem) 26.3u 8.8s 0:07.13 492.8% (0t+0ds+0avg+49046max)k 0i+6199464o 18pf 0swaps verifying output ... OK Running tests/c2h4/c2h4 cleaning scratch copying input and verified output files running nwchem (/opt/nwchem/nwchem-6.3-src.patched/bin/LINUX64/nwchem) 55.9u 2.1s 0:10.92 532.3% (0t+0ds+0avg+59834max)k 0i+808848o 19pf 0swaps verifying output ... OK [..]

2. Verify

Once the runs are done, go through the log to find out which, if any, failed. In my case, I had
Running tests/autosym/autosym 
 
     cleaning scratch
     copying input and verified output files
     running nwchem (/opt/nwchem/nwchem-6.3-src.patched/bin/LINUX64/nwchem)
 
15.3u 2.5s 0:04.20 426.4% (0t+0ds+0avg+48842max)k 0i+1325784o 17pf 0swaps
     verifying output ... failed

Running tests/dft_s12gh/dft_s12gh 
 
     cleaning scratch
     copying input and verified output files
     running nwchem (/opt/nwchem/nwchem-6.3-src.patched/bin/LINUX64/nwchem)
 
619.4u 4.3s 1:45.33 592.1% (0t+0ds+0avg+76724max)k 2472i+1938952o 26pf 0swaps
     verifying output ... failed
 
Failed
 
Running tests/cosmo_trichloroethene/cosmo_trichloroethene 
 
     cleaning scratch
     copying input and verified output files
     running nwchem (/opt/nwchem/nwchem-6.3-src.patched/bin/LINUX64/nwchem)
 
113.6u 2.3s 0:19.57 592.8% (0t+0ds+0avg+63700max)k 64i+1149456o 20pf 0swaps
     verifying output ... failed

 Running tests/bsse_dft_trimer/bsse_dft_trimer 
 
     cleaning scratch
     copying input and verified output files
     running nwchem (/opt/nwchem/nwchem-6.3-src.patched/bin/LINUX64/nwchem)
 
228.2u 4.8s 0:40.16 580.5% (0t+0ds+0avg+53806max)k 0i+2716224o 21pf 0swaps
     verifying output ... failed
 
Failed

and so on. Not a good start.

3. Troubleshoot the failed tests
To find out whether the failures are significant, we first need to understand how the script is doing the testing.

In runtests.mpi.unix
369 # Now verify the output 370 371 echo -n " verifying output ... " 372 373 perl $NWPARSE $STUB.out >& /dev/null 374 if ($status) then 375 echo nwparse.pl failed on test output $STUB.out 376 set overall_status = 1 377 continue 378 endif 379 perl $NWPARSE $STUB.ok.out >& /dev/null 380 if ($status) then 381 echo nwparse.pl failed on verified output $STUB.ok.out 382 set overall_status = 1 383 continue 384 endif 385 386 diff -w $STUB.ok.out.nwparse $STUB.out.nwparse >& /dev/null 387 @ diff1status = $status 388 # 389 endif 390 # 391 392 if ($diff1status) then 393 echo "failed" 394 set overall_status = 1 395 continue 396 else

In my case autosym failed:

cd testoutputs
diff autosym.ok.out.nwparse autosym.out.nwparse 
45c45 < Effective nuclear repulsion energy (a.u.) 4265.6221 --- > Effective nuclear repulsion energy (a.u.) 4265.6222
It seems to be a rounding error. As far as I know the precision at which the data is stored is significantly higher than at which it is reported, so this doesn't necessarily need to be a problem (it's still not a good thing though). Note that everything else, such as the thermochemical parameters, are identical.

Continuing:
diff dft_s12gh.ok.out.nwparse dft_s12gh.out.nwparse 
52c52 < The Zero-Point Energy (Kcal/mol) = 21.82496 --- > The Zero-Point Energy (Kcal/mol) = 21.82497 128c128 < H 0.0123 0.0030 0.0000 --- > H 0.0122 0.0030 0.0000
Same thing.

Here's a list over the tests that failed for me (-> indicates that the execution failed -- more details below; * indicates that it is expected to fail):

autosym
dft_s12gh
cosmo_trichloroethene
bsse_dft_trimer
cosmo_h3co
cosmo_h3co_gp
h2o_diag_to_cg_ub3lyp
* oh2
dft_cr2
dft_x
dft_ozone
hess_nh3_ub3lyp
pspw_SiC
paw
-> tddft_h2o_mxvc20
-> tddft_h2o_uhf_mxvc20
tce_cr_eom_t_ch_rohf
hi_zora_sf
o2_zora_so
lys_qmmm
ethane_qmmm
qmmm_opt0
prop_ch3f
ch3f-lc-wpbe
ch3f-lc-wpbeh
ch3radical_rot
ch3radical_unrot
cho_bp_props
-> prop_cg_nh3_b3lyp
acr-camb3lyp-cdfit
acr-camb3lyp-direct
acr_lcblyp
o2_bnl
fh_m06 ???
disp_dimer_ch4
disp_dimer_ch4_cgmin
mep-test
sif_sodft
h2o_raman_3
h2o_raman_4
tropt-ch3nh2
h3_dirdyvtst
h2o_hcons
etf_hcons
cho_bp_props
-> dntmc_h2o_nh3
5h2o_core
co_core
talc
neb-fch3cl
neb-isobutene
nwxc_pspw_1he
nwxc_pspw_1ne
nwxc_pspw_4n
nwxc_pspw_4p
nwxc_pspw_new_1he
nwxc_pspw_new_3he
nwxc_pspw_new_1ne
nwxc_pspw_new_4n
nwxc_pspw_new_1ar
nwxc_pspw_new_4p
nwxc_pspw_new_1kr
nwxc_pspw_new_4as
nwxc_pspw_new_1xe
nwxc_pspw_new_4sb
hess_nh3_dimer
pbo_nesc1e
h2o_selci
hess_biph
-> ch4_zts
-> ch4cl_zts

All the jobs without a '->' or '*' failed due to rounding errors. To quickly go through them I put the list of failed jobs in a file, and then did
cat fails |xargs -I {} diff testoutputs/{}.ok.out.nwparse testoutputs/{}.out.nwparse|less


The jobs that failed outright are listed below:

-> tddft_h2o_mxvc20
tddft_diagon: negative excitation energy 0 ------------------------------------------------------------------------ This type of error is most commonly associated with calculations not reaching convergence criteria
-> tddft_h2o_uhf_mxvc20
Last System Error Message from Task 5:: Numerical result out of range tddft_diagon: negative excitation energy 0
-> prop_cg_nh3_b3lyp
task hessian incompatible with cgmin 0 ------------------------------------------------------------------------ A feature requested has not yet been implemented
-> dntmc_h2o_nh3
********** Destroying SubGroups *********** ******************************************* deleting cloned rtdb deleting cloned rtdb Closing subgroup Closing subgroup 1:1:ga_pgroup_destroy_:Attempt to destroy process group with attached GAs:: 2 (rank:1 hostname:boron pid:1941):ARMCI DASSERT fail. ../../ga-5-2/armci/src/common/armci.c:ARMCI_Error():208 cond:0 2:2:ga_pgroup_destroy_:Attempt to destroy process group with attached GAs:: 2 (rank:2 hostname:boron pid:1942):ARMCI DASSERT fail. ../../ga-5-2/armci/src/common/armci.c:ARMCI_Error():208 cond:0 Last System Error Message from Task 1:: No such file or directory Last System Error Message from Task 2:: No such file or directory
-> ch4_zts
scf string failed 0 ------------------------------------------------------------------------ This type of error is most commonly associated with calculations not reaching convergence criteria
-> ch4cl_zts
scf string failed 0 ------------------------------------------------------------------------ This type of error is most commonly associated with calculations not reaching convergence criteria

It's time to go back and compare with 1. nwchem-6.3/acml and 2. nwchem-6.1.1/openblas and 3. a different processor architecture...

The question is how serious this is. In most cases I think the rounding errors are ok, but errors do accumulate, and especially when large and small numbers are multiplied they can become significant.

08 June 2013

442. Down? www.nwchem-sw.org

Epilogue: looks like it was announced (indirectly) here: http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id856/#post_3167

Update 10/6/2013: the site is back online.

Original post:
www.nwchem-sw.org is currently (Fri 7 pm 7 June 2013 PDT; Sat 12 pm 8 June AEST)  rejecting all connections with

"The specified URL cannot be found"

It's been doing it for a couple of hours now. It doesn't matter what your target URL is -- all connections are rejected.

07 June 2013

441. Links to two good articles on windows, piracy and linux.

I don't like the idea of simply posting links to other blogs, but in this case I'll make an exception due to the quality of the articles.

These two articles from 2008 discuss the lack of adoption of linux based on the notion that windows is de facto free (gratis) to most people:
http://autotelic.com/windows_is_free
http://autotelic.com/no_really_-_windows_is_free

I find them well-written and well-reasoned.

And they address the basis of one of the more disparaging (although to some extent true) remarks by people who don't see what the fuss over linux is all about: That we can't get people to use Linux even by giving it away for free.

Anyway.From a more personal POV:

* Having lived in China I definitely agree with the idea that piracy of windows is one biggest problems when it comes to the adoption of free software in the developing world. Not only windows of course -- I could even buy SPSS, Origin and Matlab in my local computer software street stall or my local DVD store (all pirated, of course).

* Having lived in the developing world I also agree that Windows is free in the sense that you can hardly buy a computer without getting a copy of a recent windows version included (whether you want it or not).

* Working at a university -- and having worked at five in total -- I also agree that it's easy enough to get free access to most pieces of proprietary software and since the distinction between Home and Work is a bit fluid in academia, for all intents and purposes I have free access to Office, Windows, Photoshop, SPSS etc.

* And finally, having bought my first computer in as a teenager in 1993 (a 1.8 MHz 386SX, 2 Mb RAM, 28 Mb HDD -- second hand) I also grew up swapping floppies with windows (3.11 FTW!), DOS (the box had 5.0, but got DOS 6 from a class mate), and various pieces of free/shareware that we ordered via mailorder...(or bbs -- but anything over 100 kb took forever). I don't think teens of today look at things much differently from how we did back then.


06 June 2013

440. Briefly: Upgrading Arch -- consolidation of /sbin, /bin and /usr/sbin

Anyone running Arch and attempting to upgrade recently will have run into this:

(83/83) checking for file conflicts  [##########] 100%
error: failed to commit transaction (conflicting files)
filesystem: /bin exists in filesystem
filesystem: /sbin exists in filesystem
filesystem: /usr/sbin exists in filesystem
Errors occurred, no packages were upgraded.


There's a solution:
http://news.softpedia.com/news/Read-This-Article-Before-Updating-Your-Arch-Linux-358104.shtml

What I'm doing in this post is simply testing that solution. The intellectual contribution from me is very small.

1. Non-official packages
pacman -Qqo /bin /sbin /usr/sbin | pacman -Qm -
bootchart2-git 20130314-1
Not too bad.
sudo pacman -R bootchart2-git

2. Unofficial repositories
cat /etc/pacman.conf | grep '^\['
[options] [core] [extra] [community] [multilib

OK. Let's see whether there are any packages around:

paclist options | awk ' { print $1 } ' | pacman -Ql - | grep ' /s\?bin/\| /usr/sbin/'
apache /usr/sbin/ apache /usr/sbin/ab apache /usr/sbin/apachectl apache /usr/sbin/apxs [..] wpa_supplicant /usr/sbin/wpa_supplicant zvbi /usr/sbin/ zvbi /usr/sbin/zvbid
paclist core | awk ' { print $1 } ' | pacman -Ql - | grep ' /s\?bin/\| /usr/sbin/' paclist extra | awk ' { print $1 } ' | pacman -Ql - | grep ' /s\?bin/\| /usr/sbin/' paclist community | awk ' { print $1 } ' | pacman -Ql - | grep ' /s\?bin/\| /usr/sbin/' paclist multilib | awk ' { print $1 } ' | pacman -Ql - | grep ' /s\?bin/\| /usr/sbin/'

I think 'options' would qualify as an Official repository, so we are fine.
3. Orphan packages
find /bin /sbin /usr/sbin -exec pacman -Qo -- {} + >/dev/null

comes up blank. Good.

4. Upgrade
sudo pacman -Syu --ignore filesystem,bash
[..] (27/81) upgrading libgdm [################] 100% (28/81) upgrading gdm [################] 100% warning: directory permissions differ on /var/log/gdm/ filesystem: 711 package: 1770 (29/81) upgrading libx11 [..] (67/81) upgrading pacman-mirrorlist [################] 100% warning: /etc/pacman.d/mirrorlist installed as /etc/pacman.d/mirrorlist.pacnew (68/81) upgrading pm-utils [..] (80/81) upgrading wicd-gtk [################] 100% (81/81) upgrading zvbi [################] 100%

So far, so good.

sudo pacman -S bash
resolving dependencies... looking for inter-conflicts... Packages (1): bash-4.2.045-4 Total Installed Size: 3.51 MiB Net Upgrade Size: -0.10 MiB :: Proceed with installation? [Y/n] :: Proceed with installation? [Y/n] (1/1) checking keys in keyring [###################] 100% (1/1) checking package integrity [###################] 100% (1/1) loading package files [###################] 100% (1/1) checking for file conflicts [###################] 100% (1/1) checking available disk space [###################] 100% (1/1) upgrading bash
sudo pacman -Su
:: Starting full system upgrade... resolving dependencies... looking for inter-conflicts... Packages (1): filesystem-2013.05-2 Total Installed Size: 0.01 MiB Net Upgrade Size: -0.30 MiB :: Proceed with installation? [Y/n] (1/1) checking keys in keyring [###########] 100% (1/1) checking package integrity [###########] 100% (1/1) loading package files [###########] 100% (1/1) checking for file conflicts [###########] 100% (1/1) checking available disk space [###########] 100% (1/1) upgrading filesystem [###########] 100%

So far so done!

05 June 2013

439. Calculate frequencies from a hessian file from NWChem: example in Octave (matlab)

I wanted to calculate normal modes (frequencies) for specific atoms in a calculation, and so I had to write my own code.

This Octave code calculates frequencies for the first N atoms, where N is given in the input.mass file.

Background
The format that NWChem uses for the Hessian is that of a flat, triangular matrix i.e. a triangular matrix such as
1  
2 3 
4 5 6
is represented as
1
2
3
4
5
6

The Hessian is symmetric around the diagonal, so the full Hessian matrix is
1 2 4
2 3 5
4 5 6

The Hessian is independent of the masses of the atom pairs, while the frequencies are heavily dependent on the masses (isotope effects are quite visible for light elements).

To get the mass-weighted matrix we divide by the square root of the product of the masses (H * /(sqrt(m1*m2))). Note that the matrix reported in the nwchem output ("MASS-WEIGHTED NUCLEAR HESSIAN (Hartree/Bohr/Bohr/Kamu)") is multiplied by 1,000.

Once you have the mass-weighted hessian you need to calculate the eigenvalues, sort them and convert them to cm-1 using a scaling factor.

That's it.

The code:
See below for example input.mass and input.hess

%% prepare
clear;
format long

%%Calculate conversion factor from H/B/B/amu to cm-1

%% csi=299792458; %speed of light, m/s 
%% t2au=2.418884326505E-17; % seconds per a.u.
%% Better to do it by hand to avoid rounding errors:
cau=(2.99792458 * 2.418884326505)*1E-9; %c in metres per t(a.u.)

%% 1 electron (au)=9.10938291E-31 kg
%% 1 amu = 1.66053892E-27 kg
%% Better to do by hand to avoid rounding errors:
amu2au=(1.66053892/9.10938291)*1E4;% 1 amu in a.u. (via kgs)
%% For clarity
cmtom=1/100; %m per cm
%% And finally we get our scaling factor:
scaling=cmtom*(1/(2*pi*cau*sqrt(amu2au))); %( m/cm * 1/((m/au) * au) = m/cm * 1/m = 1/cm)


%%read masses
% The mass file contains the masses of the atoms
% The first line is the number of atoms in the file
% The remaining lines are the atom masses in the same order
% as the atoms are given in the nwchem input
protomasses=fopen("input.mass");
natoms=str2num(fgetl(protomasses));
for i = 1:natoms
 mass(end+1)=str2num(fgetl(protomasses));
end
fclose(protomasses);

%% Read and construct hessian from flat hessian in .hess file
%% The .hess file provided by nwchem is flat (i.e. one
%% dimensional) and is the triangular form (i.e half) of 
%% the full hessian. We use fgetl/str2num so that we can deal 
%% with instances of scientific notation in the hessian file.
%% While we"re at it we construct the mass-weighted force matrix too.
protohessian=fopen("input.hess"); 
hessian=zeros(3*natoms);
massweighted=zeros(3*natoms);

for i = 1:3*natoms
 for j=1:i
  hessian(i,j)=str2num(fgetl(protohessian));
  massweighted(i,j)=hessian(i,j)/sqrt( mass(ceil(i/3))*mass(ceil(j/3)));
 end
end

for i=1:3*natoms
 for j=1:i
  hessian(j,i)=hessian(i,j);
  massweighted(j,i)=massweighted(i,j);
 end
end

%% Diagonalize and compute frequencies in cm^{-1}
eigen=sort(eig(massweighted));
freqs=sqrt(eigen).*scaling;

%% Make imaginary frequencies negative and store them 
%% in a new array
for n=1:size(freqs,1)
 if imag(freqs(n))==0
  frequencies(end+1,1)=real(freqs(n));
 else
  frequencies(end+1,1)=-imag(freqs(n));
 end
end

%% Echo frequencies to stdout
printf("%10.4f \n",frequencies)
%% Save frequencies as well to modes.outs
outfile=fopen("normal.out","w");
fprintf(outfile,"%i \n",natoms);
fprintf(outfile,"%10.10f \n",frequencies);
fclose(outfile);
%save 'modes.out' -ascii  frequencies

input.mass (for water):
3
1.5994910D+01
1.0078250D+00
1.0078250D+00

input.hess (this one has imaginary frequencies as well):
     6.6177469151D-01
    -5.8658669668D-12
    -1.0013075598D-05
     1.0754299967D-09
     4.5060920407D-10
     3.6644723357D-01
    -3.3088202114D-01
     2.1099357839D-10
     1.6617441386D-01
     3.6163164885D-01
     2.5270659061D-12
     4.0920019206D-06
     3.2209366184D-11
     1.6382988861D-11
     8.3427731090D-07
     2.3904755566D-01
    -2.2311539742D-10
    -1.8322029567D-01
    -2.0261099118D-01
     1.1292349908D-10
     1.7796238990D-01
    -3.3088202212D-01
    -2.4469194991D-10
    -1.6617441477D-01
    -3.0749615389D-02
     1.2368245322D-10
    -3.6436594678D-02
     3.6163164980D-01
     2.5272503844D-12
     4.0920029550D-06
     3.2022391582D-11
     1.6289326095D-11
    -4.9229359909D-06
    -1.5407535297D-11
    -1.8816632580D-11
     8.3427660670D-07
    -2.3904755666D-01
    -2.2750774181D-10
    -1.8322029575D-01
     3.6436523006D-02
     1.1512005611D-10
     5.2580053385D-03
     2.0261099171D-01
     1.1238793371D-10
     1.7796238961D-01

Output:
 
  -11.0036 
   -1.6327 
    3.1676 
    3.9298 
    7.5811 
   12.2862 
 1619.0207 
 3616.0904 
 3781.1341

c.f.
 ----------------------------------------------------------------------------
 Normal Eigenvalue ||                 Infra Red Intensities
  Mode   [cm**-1]  || [atomic units] [(debye/angs)**2] [(KM/mol)] [arbitrary]
 ------ ---------- || -------------- ----------------- ---------- -----------
    1      -11.004 ||    0.426523           9.840       415.796      59.477
    2       -1.633 ||    0.000029           0.001         0.028       0.004
    3        3.168 ||    0.000003           0.000         0.003       0.000
    4        3.930 ||    0.000700           0.016         0.682       0.098
    5        7.581 ||    0.134394           3.101       131.014      18.741
    6       12.286 ||    0.000000           0.000         0.000       0.000
    7     1619.021 ||    0.070174           1.619        68.409       9.786
    8     3616.091 ||    0.004517           0.104         4.404       0.630
    9     3781.135 ||    0.009065           0.209         8.837       1.264
 ----------------------------------------------------------------------------