456. Adding NWChem basis sets to ECCE. Part 2. A solution: nwchem2ecce.py

UPDATED!

I've moved the finished scripts to here:
https://sourceforge.net/projects/nwbas2ecce/

They work! I've also added a number of converted basis sets to the sourceforge repo under 'examples'. You'll also find example ecp and ECPOrbital files.

Phew...

Here's the README:

The programmes are not 'intelligent' -- they won't check that you are doing something reasonable. Bad input = bad output.

__Installation__:
Download eccepag and nwbas2ecce

They are both python (2.7) programmes, so you will need to install python to run them.

On linux, this is normally very easy. E.g. on debian, run
'sudo apt-get install python2.7'
and you are done.
If you want, you can put the files in /usr/local/bin and do
'sudo chmod +x /usr/local/bin/eccepage'
'sudo chmod +x /usr/local/bin/nwbas2ecce'
and you will be able to call the scripts from any directory.

__Usage__
nwbas2ecce can turn a full basis set, or a, ECP basis set, into an ECCE compatible set of basis set files.
Typically, an nwchem basis set consists of a single file, e.g. 3-21g. It can also be divided into several files, e.g. def2-svp and def-ecp, where the effective core potentials (ecps) are in def2-ecp. Other basis set files, like lanl2dz_ecp, contains both the orbital and the contraction parts.

Typically, a ECCE basis set suite consists of: 
basis.BAS
basis.BAS.meta
basis.POT (for ECP)
basis.POT.meta (for ECP)

Sometimes polarization and diffuse functions are separated from the main .BAS file. E.g. 3-21++G* consists of
3-21G.BAS
3-21GS.BAS
POPLDIFF.BAS

, in addition to the meta files. The meta files are just markup-language type files with e.g. references. Note that you don't HAVE to break up the basis set components like this.

Since the basis set data can be broken up into smaller files, the overall basis set is defined as an entry in a category file.
For example, 3-21G is defined in the category file 'pople', and points to 3-21G.BAS. 3-21G* is also defined in pople, but point to both 3-21G.BAS and 3-21GS.BAS.

ECP works in a similar way, by combining a .BAS and a .POT file. Note that the .POT files look different from the .BAS files.

nwbas2ecce generates .BAS and .POT files based on whether there are basis/end or ecp/end sections in the nwchem basis set file. If there are both, both POT and BAS files are generated.

All these files are contained in server/data/Ecce/system/GaussianBasisSetLibrary

Finally, you need to generate .pag and .dir files that go into the server/data/Ecce/system/GaussianBasisSetLibrary/.DAV directory. The .dir file is always empty, while the .pag file is unfortunately a binary file. eccepag can, however, generate it with the right input.

See e.g. http://verahill.blogspot.com.au/2013/06/455-adding-nwchem-basis-sets-to-ecce.html for more detailed information

__Example__
We'll use def2-svp as an example. The nwchem basis set file def2-svp contains the basis set, while def2-ecp contains the core potentials. Use def2-svp to generate DEF2_SVP.BAS, DEF2_SVP.BAS.meta. Use def2-ecp to generate DEF2_ECP.POT, DEF2_ECP.POT.meta. 

As part of the generation, .descriptor files are also generated. These contain information that should go into the category file(s).

Then generate the .pag files for both the POT and the BAS files, and touch the .dir files into existence.

Do like this:
nwbas2ecce -i def2-svp -o DEF2_SVP.BAS -n 'def2-svp'
nwbas2ecce -i def2-ecp -p DEF2_ECP.POT -n 'def2-ecp'
eccepag -n def2-svp -t ECPOrbital -c ORBITAL -y Segmented -s Y -o DEF2_SVP.BAS.pag
eccepag -n def2-ecp -t ecp -c AUXILIARY -o DEF2_ECP.POT.pag

NOTE: I don't actually know if def2-svp is segmented, and spherical. I don't think it matters for the .pag file generation. Also note that most inputs are case sensitive. Look at a similar .pag file for hints.

You now have the following files:
DEF2_ECP.POT
DEF2_ECP.POT.descriptor
DEF2_ECP.POT.meta
DEF2_ECP.POT.pag
DEF2_SVP.BAS
DEF2_SVP.BAS.descriptor
DEF2_SVP.BAS.meta
DEF2_SVP.BAS.pag

Copy the files. Note that you need to select the correct target directory, and that will vary with where you installed ECCE. I'll assume it's in /opt/ecce
cp DEF2* /opt/ecce/server/data/Ecce/system/GaussianBasisSetLibrary
cd /opt/ecce/server/data/Ecce/system/GaussianBasisSetLibrary
mv *.pag .DAV/
touch .DAV/DEF2_SVP.BAS.dir .DAV/DEF2_ECP.POT.dir
cat DEF2_SVP.BAS.descriptor >> ECPOrbital
cat DEF2_ECP.POT.descriptor >> ECPOrbital
cat DEF2_ECP.POT.descriptor >> ecp

Edit ECPOrbital so that it reads:
name= def2-svp
files= DEF2_SVP.BAS DEF2_ECP.POT
atoms= H He Li Be B C N O F Ne Na Mg Al Si P S Cl Ar K Ca Sc Ti V Cr Mn Fe Co Ni Cu Zn Ga Ge As Se Br Kr Rb Sr Y Zr Nb Mo Tc Ru Rh Pd Ag Cd In Sn Sb Te I Xe Cs Ba La Hf Ta W Re Os Ir Pt Au Hg Tl Pb Bi Po At Rn
atoms= Rb Sr Y Zr Nb Mo Tc Ru Rh Pd Ag Cd In Sn Sb Te I Xe Cs Ba La Hf Ta W Re Os Ir Pt Au   Hg Tl Pb Bi Po At Rn 

/pre>

195. Frequency calcs in NWChem

It's no secret that I'm a computational 'noob'. As such as I'm learning both by reading and by doing.

The doing part consists of checking 1) what the time penalty for different methods is and 2) what the accuracy/differences between different methods are.

Again, these are short calculations for simple molecules. Longer calculations with more exciting features (unpaired electrons, closely spaced MOs, highly negative charges etc.) may well behave completely different.

Today's focus is vibrational calcs.

Test Molecule: CHClF(OH) (chloro-fluoro-methanol)

  1 Title "Freq_test"
  2 
  3 Start  Freq_test
  4 
  5 echo
  6 
  7 charge 0
  8 
  9 geometry noautosym units angstrom
 10  C     0.0416942     -0.501783     0.399137
 11  H     0.0442651     -0.499048     1.48122
 12  O     1.21393     -1.00985     -0.0746688
 13  H     1.25125     -0.957351     -1.06923
 14  F     -1.08480     -1.08768     -0.134571
 15  Cl     -0.120345     1.41214     -0.0717951
 16 end
 17 
 18 ecce_print ecce.out
 19 
 20 basis "ao basis" cartesian print
 21   H library "3-21G"
 22   F library "3-21G"
 23   Cl library "3-21G"
 24   O library "3-21G"
 25   C library "3-21G"
 26 END
 27 
 28 dft
 29   mult 1
 30   odft
 31   mulliken
 32 end
 33 
 34 task dft energy
 35 task dft freq

All geometries were optimised in the gas phase using 3-21G.

0. Some useful statements:

hessian      print "hess_follow"
                 profile
end

1. Basis set (geometry optimised in 3-21g)
(time/enthalpy/entropy/scfe)
3-21G:              81s    24.984 kcal/mol    69.235 cal/mol-K   -671.17956992206 Hartree
6-31G:            105s    21.885 kcal/mol    68.793 cal/mol-K   -674.478768966106
6-31++G**:    399s   21.734 kcal/mol     68.818 cal/mol-K   -674.573524091623
cc-pVDZ:        325s    21.682 kcal/mol    68.819 cal/mol-K   -674.594059146606
aug-cc-pVDZ:  901s   21.605 kcal/mol    68.840 cal/mol-K   -674.623145113155

LANL2DZ(C)/6-+G* 262s  24.923 kcal/mol 68.981 cal/mol-K  -674.539040349134
UHF/aug-cc-pVDZ   373 s 26.196  kcal/mol 68.228 cal/mol-K -672.85402652170

Cation:
3-21G:               ---     21.164 kcal/mol     74.407 cal/mol-K    -670.763278724519 Hartree
6-31G:              142s   21.153 kcal/mol     74.645 cal/mol-K    -674.089132280731
6-31++G**:      637s   21.192 kcal/mol    73.768 cal/mol-K    -674.178146586266
cc-pVDZ:          399s   21.153 kcal/mol    73.736 cal/mol-K    -674.210312017948
aug-cc-pVDZ:   1776s 21.089 kcal/mol     73.774 cal/mol-K   -674.228204222891

LANL2DZ(C)/6-+G* 454s 24.795 kcal/mol  74.293 cal/mol-K -674.140922359750
UHF/aug-cc-pVDZ  741s 26.002 kcal/mol  72.462 cal/mol-K  -672.518095855130

2. Thermochemistry (ΔG of oxidation; gas phase)
3-21G:            -5.3620 kcal/mol +  261.22 kcal/mol =  6.814 V*
6-31G:            -2.4768 kcal/mol +  244.50 kcal/mol =  6.214 
6-31++G**:    -2.0178 kcal/mol+  248.10 kcal/mol =  6.390 
cc-pVDZ:        -1.9950 kcal/mol + 240.80 kcal/mol =  6.075 
aug-cc-pVDZ: -1.9871 kcal/mol + 247.83 kcal/mol =  6.380

LANL2DZ(C)/6-+G* -1.7118 kcal/mol + 249.82 kcal/mol = 6.478
UHF/aug-cc-pVDZ -1.4564 kcal/mol +210.80 kcal/mol = 4.797

* vs SHE=4.281 eV

3. Solvation (cosmo/water/scfe)
neutral
3-21g:                66s    22.097 kcal/mol    68.875 cal/mol-K   -671.1936338426 Hartree
6-31g:                82s    22.277 kcal/mol    68.609 cal/mol-K   -674.4934780299
6-31++g**:       277s   21.493 kcal/mol    69.353 cal/mol-K  -674.586704959695
cc-pVDZ:          266s   21.869 kcal/mol    68.808 cal/mol-K  -674.605608009070
aug-cc-pVDZ:    712s  22.116 kcal/mol    69.596 cal/mol-K   -674.635237990779

LANL2DZ(C)/6-31+G* 180s  25.022 kcal/mol   69.073 cal/mol-K -674.552417717602
UHF/aug-cc-pVDZ 412s  24.083 kcal/mol 70.519 cal/mol-K  -672.868085966222

cation (solvation energy)**

3-21G:               --- /26s        21.164 kcal/mol     74.407 cal/mol-K     -670.881469242560 Hartree
6-31G:              142s/51s      21.153 kcal/mol     74.645 cal/mol-K     -674.175491218588
6-31++G**:      637s/111s   21.192 kcal/mol    73.768 cal/mol-K      -674.267298880087
cc-pVDZ:          399s/129s   21.153 kcal/mol    73.736 cal/mol-K      -674.294609415029
aug-cc-pVDZ:   1776s/311s 21.089 kcal/mol     73.774 cal/mol-K     -674.316552324118

LANL2DZ(C)/6-31+G* 454s 24.795 kcal/mol  74.293 cal/mol-K -674.232656980139
UHF/aug-cc-pVDZ   741s 26.002 kcal/mol  72.462 cal/mol-K -672.451040948823
** UHF can't be used with COSMO according to nwchem. Instead we use the cation thermo calcs in the gas phase and use the scfe from a cosmo calc.

Thermochemistry*** (using gas phase freq for both cation and neutral species with scfe w/ cosmo given in parentheses):

3-21G:            -2.5824+195.88=  4.101 V (3.981 V)
6-31G:            -2.9236+199.54=  4.245 V (4.265 V)
6-31++G**:   -1.6173+200.43=  4.341 V (4.324 V)
cc-pVDZ:       -2.1853+195.15=  4.087 V (4.095 V)
aug-cc-pVDZ: -2.2727+199.98= 4.293 V (4.305 V)

LANL2DZ(C)/6-31+G*  -0.41322+200.65= 4.402
UHF/aug-cc-pVDZ 1.3397+261.7 (!)= 7.126
* vs SHE=4.281 eV

*** using freq calc of neutral species with cosmo, vs freq calc of cation in gas phase and energy w/ cosmo

4. Spectra
We'll use octave for this. First, using cat and gawk, I put the x/y coordinates in a file.

gauss= @(x,f,i,sigma)  i.*1./(sigma.*sqrt(2*pi)).*exp(-0.5.*((x-f)./sigma).**2)
subplot(3,2,1); axis([0 4000 0 2])
spc=load('321g.spc');sf=spc(:,1); si=spc(:,2);x=linspace(0,4000,800);spec=cumsum(gauss(x,sf,si,75)); 
title("321g"); plot(x,spec(18,:))
subplot(3,2,2)
spc=load('ccpvdz.spc');sf=spc(:,1); si=spc(:,2);x=linspace(0,4000,800);spec=cumsum(gauss(x,sf,si,75));
title("ccPVDZ");plot(x,spec(18,:))
subplot(3,2,3)
spc=load('631g.spc');sf=spc(:,1); si=spc(:,2);x=linspace(0,4000,800);spec=cumsum(gauss(x,sf,si,75));
title("631g"); plot(x,spec(18,:))
subplot(3,2,4)
spc=load('augccpvdz.spc');sf=spc(:,1); si=spc(:,2);x=linspace(0,4000,800);spec=cumsum(gauss(x,sf,si,75));
title("aug-ccPVDZ");plot(x,spec(18,:))
subplot(3,2,5)
spc=load('631gppdd.spc');sf=spc(:,1); si=spc(:,2);x=linspace(0,4000,800);spec=cumsum(gauss(x,sf,si,75));
title("631++g**"); plot(x,spec(18,:))

From top to bottom: Left: 3-21G, 6-31G, 6-31++G**. Right: cc-pVDZ, aug-cc-pVDZ

5. Conclusions
It may seem weird that as a test case I picked a species I don't have any reference potential for. However, the goal here was to understand how the basis set affects the results, without being distracted by such things as Real Life.

The observed spectra can be divided into two group: 3-21G/6-31G vs 6-31++G**/cc-pVDZ/aug-cc-pVDZ. Polarization (and diffuse functions) seem to play a large role.

In terms of thermochemistry, not surprisingly aug-cc-pVDZ and 6-31++G** give very similar results since they both implement pol/diff functions. The computational cost is, however, significantly higher for aug-cc-pVDZ than 6-31++G**, at least in nwchem.

There is also little difference between doing freq calculations in gas phase vs using cosmo when it comes to the calculated redox potential for the more extensive basis sets.

3-21G gives very varying results, with it giving the highest potential in the gas phase but the second lowest potential with cosmo. cc-pVDZ consistently gives the lowest potential.

UHF/ROHF/HF are fast, but wildly inaccurate. LANL2DZ/6-31+G* looks ok, results-wise, but the thermodynamic corrections are actually much smaller in conjunction with COSMO than the other methods, which is suspicious.

If given the time I may post a more detailed analysis of polarisation vs diffuse functions later.