506. Extracting optimized structures from a potential energy scan in nwchem

Another update:
It now dumps the energies in a file, energies.dat, as well.

Update:
some programmes, like ecce, are more picky about the xyz format than others (e.g. jmol, vmd). I've updated the code to output xyz files that ecce too can read.

Original post:
When you use scan_input() in nwchem to do a PES scan (see e.g. here: http://verahill.blogspot.com.au/2013/08/503-relaxed-pes-scanning-in-nwchem.html) you get the energies and the gradients for the optimized structures returned as the results. However, for a casual user the atomic actual coordinates is more informative.

Here's a very simple parser written in python (2.7) which extracts the optimized structures from the output file:

#!/usr/bin/python
import sys

def getrawdata(infile):
        f=open(infile,'r')
        opt=0
        geo=0
        energy=[]
        energies=[]
        struct=[]
        structure=[]
        for line in f:
                if "Total DFT" in line:
                        line=filter(None,line.rstrip('\n').split(' '))
                        energy=float(line[4])
                if 'Optimization converged' in line:
                        opt=1
                if opt==1 and 'Geometry' in line:
                        geo=1
                if      'Atomic Mass' in line and (opt==1 and geo==1):
                        opt=0
                        geo=0
                        struct+=[structure]
                        energies+=[energy]
                        structure=[]
                if opt==1 and geo==1:
                        structure+=[line.rstrip()]
        return struct,energies

def genxyzstring(coords,element):
        x_str='%10.5f'% coords[0]
        y_str='%10.5f'% coords[1]
        z_str='%10.5f'% coords[2]
 
        xyz_string=element+(3-len(element))*' '+10*' '+\
        (8-len(x_str))*' '+x_str+10*' '+(8-len(y_str))*' '+y_str+10*' '+(8-len(z_str))*' '+z_str+'\n'
 
        return xyz_string

def getstructures(rawdata):
        
        n=0
        for structure in rawdata:
                
                n=n+1
                num="%03d" % (n,)
                g=open('structure_'+num+'.xyz','w')
                itson=False
                cartesian=[]
                        
                for item in structure:
                        
                        if itson and not(item==""):
                                coords=filter(None,item.split(' '))
                                coordinates=[float(coords[3]),float(coords[4]),float(coords[5])]
                                element=coords[1]
                                cartesian+=[genxyzstring(coordinates,element)]
                                #cartesian+=[coords[1]+'\t'+coords[3]+'\t'+coords[4]+'\t'+coords[5]+'\n']
                
                        if "---" in item:
                                itson=True
                        if item=="" and itson==True:
                                itson=False
                                if not(len(cartesian)==0):
                                        g.write(str(len(cartesian))+'\n')
                                        g.write('Structure '+str(n)+'\n')
                                        for line in cartesian:
                                                g.write(line)
                                        g.close()
                                cartesian=[]
        return 0
        
if __name__ == "__main__":
        infile=sys.argv[1]
        rawdata,energies=getrawdata(infile)
        structures=getstructures(rawdata)

        g=open('energies.dat','w')
        for n in range(0,len(energies)):
                g.write(str(n)+'\t'+str(energies[n])+'\n')
        g.close()

Presuming that you've saved it as pes_parse.py you can then generate a series of xyz files with the structures, catenate them into a trajectory file, and open it in e.g. jmol. I'm using the output from example 1 in http://verahill.blogspot.com.au/2013/08/503-relaxed-pes-scanning-in-nwchem.html as the example:

chmod +x pes_parse.py
./pes_parse.py nwch.nwout
ls

nwch.nwout    structure_001.xyz  structure_003.xyz  structure_005.xyz  structure_007.xyz  structure_009.xyz  structure_011.xyz  structure_013.xyz  structure_015.xyz  structure_017.xyz  structure_019.xyz
pes_parse.py  structure_002.xyz  structure_004.xyz  structure_006.xyz  structure_008.xyz  structure_010.xyz  structure_012.xyz  structure_014.xyz  structure_016.xyz  structure_018.xyz

cat structure_*.xyz >> trajectory.xyz 
jmol trajectory.xyz

You can go through the structures by clicking on the arrows indicated by the white arrow:

Finally, using VMD it's easy to make videos -- note that they for some reason look awful here (seems like a lot of frames are removed, in particular from the beginning of the run):

And here's the SN2 reaction from post 503:

505. Testing KDE on Debian (if you're a gnome user) -- getting set up

Every now and again I get ticked off with something to do with planned 'features' in gnome 3. I'm still not happy about gnome-screenshot, but I patched and compiled my own version, so I'm doing alright. I don't really like the transition of menus from the, well, menu bar of applications to the top bar of the gnome desktop, but it's not a great issue.

Oh, and I don't really use vanilla gnome 3 anyway -- I've got a desktop set up with a top panel, and bottom panel, static desktops etc. The latest 'feature' which is making me want to move to a different desktop is the planned change in the use of the middle-button so that it no longer can be used to paste text.

(of course there's a bit of inconsistency here: KDE turns out to have a remapped middle-click, the screenshot application needs a few more clicks than the old gnome-screenshot etc.)

I've toyed with the idea of changing desktops in the past, and for a while was running nothing but KDE on my laptop (I'd rather switch on my 'casual' computer first, before making the transition on my work computer), but for some reason I switched back. I actually can't remember why, but something really annoyed me about KDE.

Before I forget: a good reason to try another desktop is simply to reassure yourself that you do have alternatives if your chose DE suddenly turns into something you can't work with.

Anyway, time to give it another go. But what desktop? I like the idea of going for KDE since it's 'the other desktop' -- it's been around longer than gnome, whereas lxde and xcfe are mere babies in comparison. Alternatively, one could always piece together a custom desktop using e.g. fluxbox. Or switch to something completely different, like xmonad. Or something a little bit different, like Enlightenment. I think I should probably be going for LXDE (I instinctively don't like XFCE), but let's go for KDE first.

Finally, the main danger is switching desktops and expecting the new one to act exactly like the old one (minus the bad parts, obviously). Which is pretty much how windows users (myself included once upon a time) behave when trying out linux for the first time.

Anyway, time for a week in KDE (4.10.5).

Here's my gnome desktop at the time of switching:

In addition, I'm using Guake, and my network manager is wicd.

So let's see what KDE will look like at the end -- whether I'll be able to use KDE as KDE, instead of as an ersatz gnome.

Step 1. Install KDE

sudo apt-get install kde-full wicd-kde kdesudo yakuake

I selected kdm.

Step 2. First boot
Reboot and make sure to click the little sun icon to select the desktop you want to use, in my case KDE/plasma.

2.1 Switching workspaces
I next set up proper workspace switching so that it behaves like in...gnome (falling into the old trap of making the new desktop just like the old one)...i.e. ctrl+alt right/left arrow.

Go to System Settings/WorksSpace behaviour, Virtual Desktops -- I incrased the number of desktops to 6 (1 row), then clicked on the Switching tab -- I set shortcuts for Switch One Desktop to the Left and Switch One Desktop to the Right to ctrl+alt+left and ctrl+at+right.

2.2 Disable nepomuk
At this point I noticed that I was using 100% of the CPU. virtuoso-t was chewing up a lot, as was nepomukservices. I had a look at this post: https://forum.kde.org/viewtopic.php?f=66&t=98892

I first went to Search and Indexing (in the menu), unchecked Monitor file and directory changes, Enable when running on battery, Only when computer is not being used.  You can also go to System Settings/Desktop Search and uncheck Enable Nepomuk Semantic Desktop.

2.3 konqueror crashing
I also had repeated issues with konqueror crashing when clicking on links, but it seems to have resolved itself after disabling nepomuk.

2.4 guake/yakuake
Guake worked from the beginning -- I didn't have to set it up at all. However, I still ended up changing to yakuake since I figure it's a bit more KDE like.To get yakuake to remember changed settings (e.g. height, width) you'll need to close it manually. Restarting it will use the new settings. Not sure how I got yakuake to autostart -- I think it just happened.

2.5 The bottom panel
I had a difficult time figuring out how to add shortcuts to the panel, and once I had accidentally figured it out, I promptly screwed up the entire bottom panel, then managed to fix it again.

2.6 Keyboard shortcuts
I then had to figure out how to bind shortcuts to opening applications -- right-click on the menu ('kicker'), click Edit Applications, select your application, advanced, Advanced, Current Shortcut Key.

A more general approach to creating shortcuts was shown to me by an anonymous commentator (see below this post) -- I've posted the screenshot provided to me below:

shortcuts, gestures and whatnots

2.7 Update notification
I promptly removed the Update Notification widget -- it was driving me crazy and reminds me too much of windows. I first tried simply disabling the update notifier:

I then did apt-get autoremove update-notifier-kde. The problem -- even after a reboot I still get notifications about updates! I then did System Settings/Software Management, and click on the tiny, tiny button on the far right (level with the search field), and then click on settings and set Check for updates to never.  This finally seems to have done it.

2.8 Theme
I also changed from the default 'air' theme to 'oxygen'.

2.9 Gnome/GTK programs in KDE
To make gnome (well, gtk) programs look acceptable under KDE, install gtk2-engines-oxygen and gtk3-engines-oxygen and reboot.

2.10 Tap-to-click with synaptics
I can't get synaptics tap to click to work. Based on the odd post I tried gpointing-device-settings, but while there's a checkbox to disable tap to click, it won't enable it. I then installed kde-config-touchpad and was happy again. Funnily enough this made me discover that middle-click is mapped in KDE -- the one reason I wanted to explore something other than gnome...oh well.

2.11 Conky
To get conky to work properly I only had to make some minimal changes. /etc/conky/conky.conf:

background no
own_window yes
own_window_type normal
own_window_argb_visual true
own_window_transparent yes
own_window_hints undecorated,below,sticky,skip_taskbar,skip_pager
double_buffer yes


alignment top_right
background yes
border_width 1
cpu_avg_samples 2
default_color white
default_outline_color white
default_shade_color white
default_graph_size 20 200
draw_borders no
draw_graph_borders yes
draw_outline no
draw_shades no
use_xft yes
xftfont DejaVu Sans Mono:size=12
gap_x 20
gap_y 60
minimum_size 5 5
net_avg_samples 2
no_buffers yes
out_to_console no
out_to_stderr no                                                       
extra_newline no                                                       
own_window_class Conky                                                 
own_window_transparent yes                                             
stippled_borders 0                                                     
update_interval 1.0                                                    
uppercase no                                                           
use_spacer none                                                        
show_graph_scale no                                                    
show_graph_range no                                                    

TEXT
${exec lsb_release -ds}
${execi 5 acpi|gawk '{print $3,$4}'}
${addr eth0}/${addr wlan0}
$hr
${color grey}Uptime:$color $uptime
CPU:$alignc $cpu%
$alignc$color ${cpugraph 10,200 ffff00 ff0000 -t}  
RAM:$alignc $mem/$memmax 
$alignc$color ${memgraph 10,200 ffff00 ff0000 -t}
I/O:$alignc $diskio_read/$diskio_write
$alignc$color ${diskiograph 10,200 ffff00 ff0000 -t}
${color grey}Frequency (in GHz):$color
        ${freq_g 1}, ${freq_g freq_g2}
${color grey}Temperature:
$color CPU: ${acpitemp}°C
$hr
${color grey}File systems:$alignr $color/ ${fs_used /}/${fs_size /}
$alignr /home $color${fs_used /home}/${fs_size /home}
${color grey}Networking:
${color grey}eth0 ${color red} ${upspeed eth0}/${color green} ${downspeed eth0} ${color grey}
${color grey} ${upspeedgraph eth0 10,100 ffff00 ff0000 -t} ${color grey} ${downspeedgraph eth0 10,100 0000ff 00ff00 -t} ${color grey}
${color grey}wlan0 ${color red} ${upspeed wlan0}/${color green} ${downspeed wlan0} ${color grey}
${color grey} ${upspeedgraph wlan0 10,100 ffff00 ff0000 } ${color grey} ${downspeedgraph wlan0 10,100 0000ff 00ff00 } ${color grey}
$hr
${color grey}Name                       CPU%   MEM%
${color} ${top name 1} ${top cpu 1} ${top mem 1}
${color} ${top name 2} ${top cpu 2} ${top mem 2}
${color} ${top name 3} ${top cpu 3} ${top mem 3}
$hr
${color}City            Ping $alignr Time  
$font${color}San Francisco ${execi 60 ping -c 1 sfgate.com -n|grep icmp_seq|sed 's/=/\t/g'|gawk '{print $10,$11}'}       $alignr${tztime America/Los_Angeles %a %H:%M}
$font${color}Copenhagen          $alignr${tztime Europe/Copenhagen %a %H:%M}
$font${color}Shanghai    $alignr${tztime Asia/Shanghai %a %H:%M}
$font${color}Melbourne   $alignr${tztime Australia/Melbourne %a %H:%M

And I've discovered that I really like the hot corner (top left) in gnome. There is a hot corner in kde too, but it does slightly different things.

KDE is pretty but it still gives me the occasional feeling that I'm using windows due to the bottom panel, the menu on the left and the task tray on the right, but these are things I'll hopefully grow out of. But yes, KDE is pretty -- very pretty. And it's pretty fast too.

My old gnome desktop (same as at the beginning of this post)

and my new KDE desktop

And one just to show how versatile KDE is -- you can even make it look (almost) like gnome 3

So...now it just remains to see how I feel after a week or so. The baby duck syndrome is pretty powerful...and I'm already well on my way to turning KDE into GNOME. But then, isn't the versatility of KDE the real selling point?

Postscript:
I ended up re-enabling nepomuk and spent some time struggling with akonadi to get kmail working -- the key to getting akonadi to work was to install the same server as the backend. In my case I did:

sudo apt-get install akonadi-backend-mysql mysql-server

504. Swap file and hibernation on debian

I've got an SSD and 4 gb ram on my laptop, and have no desire to use to actual swap during normal use. However, I'd like to be able to let my laptop go into hibernation -- even suspend seems to be draining my battery pretty fast (at maybe a quarter of the rate of keeping the laptop on).

Does it make sense hibernating a laptop with SSD i.e. one which boots in ten seconds flat? Probably not. But we humans are greedy by nature.

So I need to set up a swap file, disable swapping, and see if I can use it for hibernation.

As usual, the best source of information is the archlinux wiki.
https://wiki.archlinux.org/index.php/Swap#Swap_file
https://wiki.archlinux.org/index.php/Pm-utils#Using_Swap_file_instead_of_regular_swap_partition

Anyway, having made it work and tried it out I would say this is really not worth the hassle IF you have an SSD. 

---

1. The swap file
This is my file system layout:

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        39G   20G   18G  53% /
udev             10M     0   10M   0% /dev
tmpfs           380M  816K  380M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           760M  1.1M  759M   1% /run/shm
/dev/sda2       109G   71G   33G  69% /home

In my case I'd say that / has enough space to handle a swap file. My RAM is '4 gb' (really 3.71 Gb) -- according to this and this my swap should equal my RAM, which makes intuitive sense.

Since 4,000,000,000 byte is 3.76 gb, and 4000*1024*1024 (i.e. 4000M) is 3.91 Gb, I think 4000M should be ok:

su -
fallocate -l 4000M /swapfile
chmod 600 /swapfile
mkswap /swapfile

Setting up swapspace version 1, size = 4095996 KiB
no label, UUID=2a8de3d1-14f6-473f-b40f-31618fd81169

echo 'vm.swappiness=1' >> /etc/sysctl.d/50-local.conf
echo '/swapfile none swap defaults 0' >> /etc/fstab

---

2. "PM: Swap header not found"

To try it out without rebooting:

sudo sysctl -w vm.swappiness=1
sudo swapon /swapfile
pm-is-supported --hibernate 
echo $?

0

If you got 0, then you're good to go.
sudo pm-hibernate

Trying it out the first time I got "PM: swap header not found" and some weird behaviour. This has been mentioned e.g. here. A step-by-step guide is here: https://ubuntuforums.org/showthread.php?t=1042946

Get the UUID of the partition on which the swapfile is located:

mount | grep " / "

/dev/disk/by-uuid/8adf424c-c375-4035-8d5d-181489b4461b on / type ext4 (rw,noatime,nodiratime,discard,errors=remount-ro,data=ordered)

sudo filefrag -v /swapfile | grep "First block:"

The latter command gave nothing, so I then did:

sudo filefrag -v /swapfile|less

Filesystem type is: ef53
File size of /swapfile is 4194304000 (1024000 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..       0:    7182336..   7182336:      1:            
   1:        1..    6143:    7182337..   7188479:   6143:             unwritten
   2:     6144..    8191:    7190528..   7192575:   2048:    7188480: unwritten

So now we have the UUID (8adf424c-c375-4035-8d5d-181489b4461b) and the offset (7182336).

su -
echo "resume=UUID=8adf424c-c375-4035-8d5d-181489b4461b resume_offset=7182336" | sudo tee /etc/initramfs-tools/conf.d/resume
exit

Edit /etc/default/grub and add the same line to GRUB_CMDLINE_LINUX_DEFAULT:

GRUB_CMDLINE_LINUX_DEFAULT="quiet resume=UUID=8adf424c-c375-4035-8d5d-181489b4461b resume_offset=7182336"

Run

sudo update-grub
sudo update-initramfs -u

Reboot.

---

3. Hibernating.

sudo pm-hibernate

On my lenovo sl410 what I see is the screen go blank save for a blinking "-", and the little crescent light at the front of the laptop starting to blink (next to the battery light).

After 5-10 seconds the laptop turns off.

Hitting the power button starts up the laptop -- you get the bios screen, then the grub menu, and at this point you're thinking that the whole exercise has failed -- but it ends up booting into the same situation as you had when you did pm-hibernate i.e. it worked. I mean, this is how hibernation works -- but I had honestly never used anything other than suspend before, so...well..there's a first time for everything.

Overall, on a laptop with an SSD, resuming from hibernation is about as slow as a cold start -- with entering hibernation taking longer than a shutdown. On a laptop with a spinning disk this could presumably be worth it.