657. More on charges in nwchem and gaussian

A now ten-year old paper introduced the concept of Pauling bond-strength conserving terminations (PBS ) in the use of molecular codes for calculations involving extended crystalline systems ('Quantum-Chemical Calculations of Carbon-Isotope Fractionation in CO2(g), Aqueous Carbonate Species, and Carbonate Minerals' by James R. Rustad, Sierra L. Nelmes, Virgil E. Jackson, and David A. Dixon --  see link). The authors used NWChem for the calculations, most likely due to the affiliation between the lead author and PNNL, where NWChem is developed, and where the researchers have been banned from using Gaussian.

I use Gaussian almost exclusively these days, mainly due to how fast it is.

Unfortunately, Gaussian and NWChem behave quite differently when it comes to introduction of specified nuclear charges, so I here compare the two codes in terms of how to set up PBS calculations.

NWChem (6.8):

scratch_dir /scratch
Title "charge"

Start  charge

echo

charge 0

geometry noautosym noautoz units angstrom
 Mg     0.00000     0.00000     0.00000
 O     0.00000     2.09000     0.00000
 O     1.47785     2.22045e-16     1.47785
 O     -1.47785     -1.11022e-16     1.47785
 O     0.00000     -2.09000     0.00000
 O     -1.47785     2.22045e-16     -1.47785
 O     1.47785     -1.11022e-16     -1.47785
 H1     -0.691981     2.65500     -0.691981 charge 0.5 
 H1     0.691981     2.65500     0.691981 charge 0.5 
 H1     1.87737     0.978609     1.87737 charge 0.5 
 H1     1.87737     -0.978609     1.87737 charge 0.5 
 H     -1.18539     7.33956e-09     2.56935
 H     -2.56935     -7.33957e-09     1.18539
 H     -0.691981     -2.65500     0.691981
 H     0.691981     -2.65500     -0.691981
 H     -1.87737     -0.978609     -1.87737
 H     -1.87737     0.978609     -1.87737
 H     1.18539     -2.20187e-08     -2.56935
 H     2.56935     2.20187e-08     -1.18539
end

basis "ao basis" spherical print
  H library "def2-svp"
  Mg library "def2-svp"
  O library "def2-svp"
END

dft
  mult 1
  direct
  XC pbe0
  grid xfine
  mulliken
end

task dft energy   

This gives an energy of -655.860806066326.

Removing the charges for H1 and setting the total charge to +2 gives an energy of -657.044328628867

Gaussian (16.A01):
WRONG:

%nprocshared=6
%Mem=800000000
%Chk=charge.chk
#P GFINPUT rPBE1PBE/def2svp 5D  NoSymm  Punch=(MO) Pop=(full) 

charge

0 1 ! charge and multiplicity
 Mg     0.00000     0.00000     0.00000
 O     0.00000     2.09000     0.00000
 O     1.47785     2.22045e-16     1.47785
 O     -1.47785     -1.11022e-16     1.47785
 O     0.00000     -2.09000     0.00000
 O     -1.47785     2.22045e-16     -1.47785
 O     1.47785     -1.11022e-16     -1.47785
 H(znuc=0.5)     -0.691981     2.65500     -0.691981
 H(znuc=0.5)     0.691981     2.65500     0.691981
 H(znuc=0.5)     1.87737     0.978609     1.87737
 H(znuc=0.5)     1.87737     -0.978609     1.87737
 H     -1.18539     7.33956e-09     2.56935
 H     -2.56935     -7.33957e-09     1.18539
 H     -0.691981     -2.65500     0.691981
 H     0.691981     -2.65500     -0.691981
 H     -1.87737     -0.978609     -1.87737
 H     -1.87737     0.978609     -1.87737
 H     1.18539     -2.20187e-08     -2.56935
 H     2.56935     2.20187e-08     -1.18539

gives an energy of -655.679686484!

However,

2 1  ! charge and multiplicity

gives an energy of -655.860712881, which is what we want.

Removing the znuc specifications and using

2 1  ! charge and multiplicity

gives an energy of -657.044229333

Keeping the znuc specifications and defining those protons as fragment 2, and the rest of the cluster as fragment 1

 2 1 -2 1 4 1! charge and multiplicity

gives an energy of -655.860712881


Conclusion: 
both NWChem and Gaussian can be made to use PBS, but while you use the intended cluster charge (0) in NWChem, you need to use the unmodified charge (+2) in gaussian.

609. NBO6 on a debian cluster (/w g09)

Curse blogspot and the lack of revision control and backups! I lost my post when it was almost finished.

So here's a briefer version. I have bought NBO6 and I want to integrate it with gaussian G09 rev. D (you can't use it directly with earlier binary versions)

My 'instructions' are basically copy/pasted from the NBO6 installation instructions -- this is a tl;dr version.

sudo cp nbo6.0-bin-linux-x86_64.tar.gz /opt/
cd /opt/
sudo tar xvf nbo6.0-bin-linux-x86_64.tar.gz 
sudo chown $USER:$USER nbo6 -R
vim nbo6/bin/gaunbo6

Edit:

3 set INT    = i8
  4 set BINDIR = /opt/nbo6/bin

In your queue file add

set path = ( /opt/nbo6/bin $path )

In my case, as I use ECCE I edited my apps/siteconfig/CONFIG.node files:

Gaussian-03Command{
set path = ( /opt/nbo6/bin $path )
setenv GAUSS_SCRDIR /home/me/scratch
setenv GAUSS_EXEDIR /opt/gaussian/g09d/g09/bsd:/opt/gaussian/g09d/g09/local:/opt/gaussian/g09d/g09/extras:/opt/gaussian/g09d/g09
/opt/gaussian/g09d/g09/g09< $infile > $outfile 
echo 0
}

I then tested it by running a basic gaussian calculation:

%Chk=H2O_631g.chk
#P rOPBE/6-31G 6D 10F SCRF=(PCM,Solvent=water)  Punch=(MO) pop=(nbo6)

H2O 6-31G

 0 1 ! charge and multiplicity
 O     0.00000     0.00000     0.118491
 H     0.00000     0.754898     -0.473964
 H     0.00000     -0.754898     -0.473964

and got

...
411          fchk file        "/home/me/scratch/Gau-24659.EFC"
412          mat. el file     "/home/me/scratch/Gau-24659.EUF"
413 
414  Writing Wrt12E file "/home/me/scratch/Gau-24659.EUF"
415  Gaussian matrix elements                                         Version   1 NLab= 7 Len12L=8 Len4L=8
416  Write GAUSSIAN SCALARS               from file   501 offset           0 to matrix element file.
..
429  Write ALPHA FOCK MATRIX              from file 10536 offset           0 to matrix element file.
430  No 2e integrals to process.
431  Perform NBO analysis...
432 
433  *********************************** NBO 6.0 ***********************************
434              N A T U R A L   A T O M I C   O R B I T A L   A N D
435           N A T U R A L   B O N D   O R B I T A L   A N A L Y S I S
436  ********************* Me *********************
..
447  Filename set to /home/me/scratch/Gau-24659
..
620           -------------------------------
621                  Total Lewis    9.99612  ( 99.9612%)
622            Valence non-Lewis    0.00029  (  0.0029%)
623            Rydberg non-Lewis    0.00358  (  0.0358%)
624           -------------------------------
625                Total unit  1   10.00000  (100.0000%)
626               Charge unit  1    0.00000
627 
628  $CHOOSE
629    LONE 1 2 END
630    BOND S 1 2 S 1 3 END
631  $END
632 
633  Maximum scratch memory used by NBO was 62605 words
634  Maximum scratch memory used by G09NBO was 9032 words
...

246. Cluster network performance testing (very basic) on Debian Testing using a gigabit switch

Playing with hpcc got me thinking about my network connection.

My cluster looks like this:
I've got four nodes which are connected via two networks, 192.168.2.0/24 and 192.168.1.0/24. The 192.168.1.0/24 network is connected using a gigabit switch. Be (see below) acts as the gateway. The 192.168.2.0/24 network is connected via a crappy old netgear 10/100 router (dhcp) and provides access to the outside world (hello mac spoofing :) ). Each box shares a folder via nfs using a unique name.
_Nodes_
Be: AMD II X3, 8 GB ram (192.168.1.1): Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8169 Gigabit Ethernet (rev 10)
Ta: Intel i5-2400, 8 GB ram (192.168.1.150):  Intel Corporation 82579LM Gigabit Network Connection (rev 04)
B: AMD Phenom II X6, 8 GB ram (192.168.1.101): Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 03)
Ne: AMD FX 8150 X8, 16 GB ram (192.168.1.120): Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8169 Gigabit Ethernet (rev 10)

So, time to test the network performance:
sudo apt-get install iperf

On all your boxes (e.g. using clusterssh) start the iperf daemon
iperf -s

Then on each of your nodes run:

iperf -c 192.168.1.1 && iperf -c 192.168.1.101 && iperf -c 192.168.1.150 && iperf -c 192.168.1.120

------------------------------------------------------------
Client connecting to 192.168.1.1, TCP port 5001
TCP window size: 45.7 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.101 port 37893 connected with 192.168.1.1 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   564 MBytes   473 Mbits/sec
------------------------------------------------------------
Client connecting to 192.168.1.101, TCP port 5001
TCP window size:  169 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.101 port 35926 connected with 192.168.1.101 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  15.5 GBytes  13.3 Gbits/sec
------------------------------------------------------------
Client connecting to 192.168.1.150, TCP port 5001
TCP window size: 22.9 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.101 port 48257 connected with 192.168.1.150 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   564 MBytes   473 Mbits/sec
------------------------------------------------------------
Client connecting to 192.168.1.120, TCP port 5001
TCP window size: 22.9 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.101 port 43236 connected with 192.168.1.120 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   617 MBytes   517 Mbits/sec

Overall, this is what I got
Client/Server (MBit/s)

     Be     B     Ta    Ne
Be   13.7G  310   308   316
B    564    15.5G 564   617
Ta   726    660   19.7G 936
Ne   882    484   917   19.4G

I'm not sure whether to expect a metric gigabit (1000 metric MBit) or a binary one (1024 binary MBit), but looking at our results our best is 936 Mbit/s and worst 308 Mbit/s. All of them should thus ideally reach at least 936 MBit/s. They all have gigabit network card.

And now, try to improve it:
I went through the whole shebang with
sudo ifconfig eth1 mtu 9000
sudo ifconfig eth1 mtu 8000
etc.
Anyway, I got the following MTUs that way:
Be  7100
B    7100
Ne  9000
Ta   9000

I then set the MTUs to 7100 on all the nodes and tried pinging from node to node, e.g.:
ping -s 7072 -M do 192.168.1.101

Well, that maxed out at 1472 i.e. about MTU 1500 which was the original value. So I'm a bit confused.

---

Settings:
Be:

eth1      Link encap:Ethernet  HWaddr 00:f0:4d:83:0a:48  
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::2f0:4dff:fe83:a48/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:24124966 errors:0 dropped:27064 overruns:0 frame:0
          TX packets:19569426 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:25859945667 (24.0 GiB)  TX bytes:14200267703 (13.2 GiB)

B:

eth1      Link encap:Ethernet  HWaddr 02:00:8c:50:2f:6b  
          inet addr:192.168.1.101  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::8cff:fe50:2f6b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:14540970 errors:0 dropped:36651 overruns:0 frame:0
          TX packets:16801915 errors:0 dropped:2 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:12347398135 (11.4 GiB)  TX bytes:18008416370 (16.7 GiB)

Ta:

eth1      Link encap:Ethernet  HWaddr 78:2b:cb:b3:a4:b7  
          inet addr:192.168.1.150  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::7a2b:cbff:feb3:a4b7/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:14717233 errors:0 dropped:68232 overruns:0 frame:0
          TX packets:17769966 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:13860096243 (12.9 GiB)  TX bytes:20207270880 (18.8 GiB)
          Interrupt:20 Memory:e1a00000-e1a20000 

Ne:

eth1      Link encap:Ethernet  HWaddr 90:2b:34:93:75:e6  
          inet addr:192.168.1.120  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::922b:34ff:fe93:75e6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:13567520 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10710054 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:13086635236 (12.1 GiB)  TX bytes:12381041605 (11.5 GiB)