428. system-config-kickstart on debian

Don't know much about kickstart, but you can compile the redhat tool using the Canonical-patched sources. This is in response to this post: http://forums.debian.net/viewtopic.php?f=17&t=104286

Note that:
1. The Debian way to create pre-configured installations is using Preseed
2. Debian has python-pykickstart for those who want kickstart.
3. I haven't tested system-config-kickstart beyond making sure that it runs

Dependencies and preparation:

sudo apt-get install build-essential gfortran checkinstall python-all-dev cdbs debhelper quilt intltool python-central rarian-compat pkg-config gnome-doc-utils samba python-libuser libuser1 python-glade2 console-setup hwdata python-apt
sudo apt-get install isoquery
mkdir ~/tmp
cd ~/tmp

---

localechooser

cd ~/tmp
wget http://archive.ubuntu.com/ubuntu/pool/main/l/localechooser/localechooser_2.49ubuntu4.tar.gz
tar xvf localechooser_2.49ubuntu4.tar.gz
cd localechooser/
dpkg-buildpackage -uc -us
sudo dpkg -i ../localechooser-data_2.49ubuntu4_all.deb 

---

system-config-kickstart

cd ~/tmp
wget http://archive.ubuntu.com/ubuntu/pool/main/s/system-config-kickstart/system-config-kickstart_2.5.20.orig.tar.gz
tar xvf system-config-kickstart_2.5.20.orig.tar.gz
wget http://archive.ubuntu.com/ubuntu/pool/main/s/system-config-kickstart/system-config-kickstart_2.5.20-0ubuntu22.diff.gz
gunzip system-config-kickstart_2.5.20-0ubuntu22.diff.gz
patch -p0 < system-config-kickstart_2.5.20-0ubuntu22.diff
cd system-config-kickstart-2.5.20/
dpkg-buildpackage -uc -us
sudo apt-get install ../system-config-kickstart_2.5.20-0ubuntu22_all.deb

---

Edit line 46 in /usr/share/system-config-kickstart/packageGroupList.py

availparse = apt_pkg.TagFile(availfile)

ParseTagFile is deprecated in Debian, so you'll need to do a bit of impromptu patching:

sudo sed -i 's/ParseTagFile/TagFile/g' /usr/share/system-config-kickstart/*.py
sudo sed -i 's/availparse.Step/availparse.step/g' /usr/share/system-config-kickstart/*.py
sudo sed -i 's/availparse.Section/availparse.section/g' /usr/share/system-config-kickstart/*.py

Then start with

gksu system-config-kickstart

427. Very Briefly: encfs on windows (XP, Vista)

I'm using encfs to encrypt files in dropbox -- and it turns out that I'm not the only one. In fact, I'm a late-comer to this apparently.

I've just installed Windows on physical hardware for the first time in a couple of years -- I figured I should learn more about this experimental, unstable OS that a lot of people have been talking about. I hear that it's not even based on linux, and that their leader isn't Torvalds (it's some guy called Gates, apparently). It looks pretty slick actually, but their software management model leaves a lot to desired (you have to use your browser to download software and there's no way of keeping track of security updates for anything other than the core OS). I also haven't figure out how to switch to any of the other virtual desktops/workspaces.

Whatever. The FOSS connection lies in the use of Dokan and encfs4win.

Anyway, to get encfs up and running on Windows, go to http://dokan-dev.net/wp-content/uploads/DokanInstall_0.6.0.exe and install

Then go to http://members.ferrara.linux.it/freddy77/encfs.html and download encfs.zip

I dragged the entire encfs4win folder to Program Files. Start encfsw. A notification icon (shaped like a key) should show up in the task tray. Left-click on it, select Preferences and set to Launch at Startup.

To mount an existing encrypted directory, or to create a new one, left-click, select Open/Create and follow the instructions. Simple.

426 Multibooting Windows XP, Vista and Debian

This post will most likely not be particularly interesting to anyone. It's basically just a collection of notes of me putting XP, Vista and Debian on the same box. Turns out that it's actually pretty straightforward -- the lack of drama in this post is what makes it a bit dull. Maybe the chief value of this post is to reassure anyone wanting to do the same of its feasibility.

Anyway.

I've decided to revive an old Dell C521 from 2007 (I accidentally blew the PSU when moving to Oz from the US, I've upgraded the graphics card, and put a 1 Tb hdd in it) and since I don't have any windows machines and it isn't useful for anything high-powered (dual core Athlon), I figure I might as well use it for a bit of experimentation.

This isn't going to be a detailed step-by-step how-to guide -- it's more of an overview of how to set up triplebooting with vista in case I need to help someone at a later point (parents/parents-in-law, I'm looking at you).

I have no real desire to use windows, but I could see the usefulness of having a windows box around. Partly because I'd like to do my bit to help Windows users move towards using FOSS instead of the usual commercial fare (familiarity with the software ecosystem on linux will presumably help adoption). Partly because I haven't played Halo 2 for years...

I'm bracing myself for experiencing the pain of vista again...XP I can just about tolerate -- it's a decade old, so I can accept that it has some limitations.

Anyway, triple-booting Vista, XP and Debian seems ambitious enough for a blog post, given what a pain Vista (and 7) are in terms of playing nicely with other OS:s.

Looking at this post:

When dual-booting, you always install the old OS first, then the new.

Sounds about right...Debian last then ;)

Note that the hard disk was unpartitioned at the beginning. The screen was connected via a KVM switch to the NS210 graphics card (via the VGA port). The on-board ethernet port was connected to a router set up with dhcp.

The remaining original hardware is as follows:
AMD Athlon 64 X2 3800+ 2.0 GHz
2048 Mb DDR2 RAM
Broadcom 440x 10/100

The added bits are:
Realtek RTL8169/8110 Family PCI Gigabit ethernet
MSI NF210 (Geforce 210)
Western Digital Green 1 Tb 3.5"

---

Installing XP (32 bit):
First I had to burn a CD from an iso from my university. I don't have any cd writer on my desktop, but happen to have one on one of my nodes (an old work computer). So I copied the iso via nfs, and then burned it with

sudo burn -I -n WinXP\ Pro\ US\ with\ SP1\ \&\ MS03-39.iso

There isn't much to say about installing XP, other than me making a partition at the beginning of the disk with 50,000 Mb space. I made the paritition using the windows installation program (as part of the install), chose quick format (ntfs).

The reboot step during XP installs gets me every time and I always hit a key to boot from CD automatically. Don't do that.

I only installed XP. I didn't bother chasing down drivers etc. Those who complain about hardware support in linux don't realise what it's like setting up windows on a computer using a vanilla installation disk....

XP didn't have working internet (didn't recognize either network card) and the video resolution was 640x480. Once the network had been sorted out (R149798; downloaded in Vista and moved to the XP partition)I downloaded SP3, after which I could install the nvidia drivers. Luckily, most of the remaining drivers could be downloaded from Dell (nVidia_SMBus_A02_R132919, R132395, R133065)).

Setting up multiple ethernet cards was...fun. The challenge was the routing, which required a registry change and reboot. And I never knew XP had 'shutdown -r'. It's like a whole new OS to me now.

---

Installing Vista (32 bit):
Once XP was installed I popped in the Dell DVD (yes, once upon a time you actually got the installation CDs with your computer...) and rebooted.

Select install, and choose custom. Click on the unallocated space, select Drive Options (advanced), then click on New. I set the size to 100,000 Mb (visa is a space hog, but I don't really plan on actually using it so...). Click Apply. A new partition (Disk 0 Partition 2; Primary) should appear. Note that because windows set everything to primary and because GPT only can handle four primary partitions, you become somewhat limited in the number of OSs you can install (there are reasonably simple ways around it though). Click Next and let Vista have a go at your HDD.

The vista installation wasn't too bad, and there was decent hardware support on boot. Note that this was installed using the Vista DVD that came with the Dell I just installed it on i.e. the drivers were presumably included on the DVD.

Both network cards were detected (ipconfig) and I had a working internet connection (ping google.com). The default resolution was 800x600 pixels (display settings), but it was easily changed to 1024x768.

The device manager had an exclamation point next to Standard VGA Graphics Adapter under Display Adapters. The 'Windows Experience Index' was 1.0  due to poor Graphics and Gaming Graphics.

My post install steps consisted of installing Google Chrome, then allowing windows to install updates (451 Mb). Among those updates was GF210 support. Simply downloading 451 Mb took an hour (!) even though I'm on a university connection (i.e. fast -- typically +3M/s). Installing the updates took another hour after that. And that wasn't the end of it.

The nvidia was recognised after the reboot, and I now had a 4.1 "experience rating", and I could set the resolution to 1280x960.

I had another revelation (I've been gone from Windows for a while): I plugged in an Airlink101 USB wlan dongle (rtl8187b), and downloaded the driver from CNET. The installer tried to install two pieces of software without clearly advertising that it was doing so (top arcade something, and 7 wonders something else), then wanted to throw in zonealarm and change my home page. I seem to remember cnet being one of the reasonably trustworthy sites? I don't remember it bundling junk/spy-ware. Huh. Anyway, turns out the drivers got installed via windows update anyway.

Anyway, sorting out the updates was a PITA since a number of them kept failing. Download was slow (microsoft's servers pretty awful -- maybe they should switch to linux or bsd?) and installation takes forever. In the end I had to download SP1 and SP2 manually from the 'service pack center' and install them.

And LINUX is the one they call difficult? Good riddance.

---

Installing Debian:
While jigdo has worked well for me in the past, I was lazy and simply downloaded an iso. Because of university restrictions I could not use bit torrent.

wget http://cdimage.debian.org/debian-cd/7.0.0/amd64/iso-cd/debian-7.0.0-amd64-CD-1.iso
sudo burn -I -n debian-7.0.0-amd64-CD-1.iso

I popped the CD in the drive, and booted. Manual partition: 100 Gb for / (primary), 200 Gb for /home (logical) and 2 Gb for swap (logical). I used a local network mirror to install.

I installed the GRUB boot-loader to the MBR

Note that only Vista shows up in grub -- if you select Vista in GRUB, you get the Vista boot manager ('loader'), and can select 'Earlier versions of Windows' i.e. XP.

On booting into Debian the full GNOME 3 experience was available i.e. the nouveau driver for GeForce 210 is apparently good enough that we don't need to install the nvidia drivers. glxgears looks really pretty too, with over 700 FPS. Sweet!

Anyway, that's it.

I expected it to be a bit trickier, but even Vista behaved itself and didn't throw a fit on debian being installed.

---

425. Briefly: Megapov

I want to use glow in a pov, and because of the number of objects it might be easier to use megapov than to add extra objects for each glowing one.

sudo mkdir /opt/megapov
sudo chown $USER:$USER /opt/megapov
wget http://megapov.inetart.net/packages/unix/megapov-1.2.1.tgz
tar xvf megapov-1.2.1.tgz
cd megapov-1.2.1/
./configure --prefix=/opt/megapov COMPILED_BY="me@here"
make
make install
cp /usr/share/povray/include/* /opt/megapov/share/megapov-1.2.1/include/

I chose make install instead of checkinstall since it was easier -- otherwise the ~/.megapov/1.2.1/povray.ini and povray.conf files weren't properly created (it wanted to put them in root).

424. NWChem 6.3 on Debian Wheezy

Update 23 May 2013: The execution times are pretty much the same as for 6.1.1 with a new patch. I've updated the instructions below to incorporate this new patch (http://www.nwchem-sw.org/images/Iswtch.patch.gz)

Update 21 May 2013: The execution times can be improved considerably by setting

ARMCI_NETWORK=SOCKETS

They are still ca 30% longer than 6.1.1 though due to slower SCF convergence. See http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id834/Nwchem_6.3_running_2-5_times_slo....html

Update 20 May 2013: I did a bit of basic benchmarking. NWChem 6.3 is incredibly slow (ca 190s vs 40s for the 8 core, 3.6 GHz benchmark in http://verahill.blogspot.com.au/2013/05/414-frequency-vs-cores-crude.html). It's parallellising properly from what I can see (i.e. it is not running 8 serial jobs). I've repeated the calc with an unpatched version of nwchem 6.3, and it is just as slow.
 I'll post updates here if I figure this one out.

Original post:
NWChem 6.3 is just out. Here's how to build it for CPU computations.

To build on CentOS 5.6, see http://verahill.blogspot.com.au/2013/05/421-nwchem-63-on-rocks-543centos-56.html

---

Math library:
Use either openblas (for intel or AMD) or ACML (for AMD).

• To set up Openblas, see here:
 http://verahill.blogspot.com.au/2013/05/423-openblas-on-debian-wheezy.html

• To set up ACML, see here:
 http://verahill.blogspot.com.au/2013/05/422-set-up-acml-on-linux.html

---

My GabEdit/Python NWChem patch
This is NOT the patch alluded to in the 23 May update and is optional. It enables python support, and makes the output more verbose so that gabedit can be used as an alternative to ECCE. Hence, it is required if, but only if, you want to enable python and to be able to use GabEdit to open output files.

First create a patch file, e.g. diff.patch.

diff -rupN src.original/config/makefile.h src/config/makefile.h
--- src.original/config/makefile.h 2013-04-15 12:41:45.016853322 +1000
+++ src/config/makefile.h 2013-04-15 12:38:44.933319544 +1000
@@ -2039,7 +2039,7 @@ endif
 
      ifeq ($(BUILDING_PYTHON),python)
 #   EXTRA_LIBS += -ltk -ltcl -L/usr/X11R6/lib -lX11 -ldl
-     EXTRA_LIBS +=    -lnwcutil  -lpthread -lutil -ldl
+     EXTRA_LIBS +=    -lnwcutil  -lpthread -lutil -ldl -lssl -lz
   LDOPTIONS = -Wl,--export-dynamic 
      endif
 ifeq ($(NWCHEM_TARGET),CATAMOUNT)
diff -rupN src.original/ddscf/movecs_pr_anal.F src/ddscf/movecs_pr_anal.F
--- src.original/ddscf/movecs_pr_anal.F 2013-04-15 12:41:45.036852381 +1000
+++ src/ddscf/movecs_pr_anal.F 2013-04-15 12:23:28.100409225 +1000
@@ -195,7 +195,7 @@ c
  22         format(1x,2('  Bfn.  Coefficient  Atom+Function  ',5x))
             write(LuOut,23)
  23         format(1x,2(' ----- ------------  ---------------',5x))
-            do klo = 0, min(n-1,9), 2
+            do klo = 0, min(n-1,199), 2
                khi = min(klo+1,n-1)
                write(LuOut,2) (
      $              int_mb(k_list+k)+1, 
diff -rupN src.original/ddscf/rohf.F src/ddscf/rohf.F
--- src.original/ddscf/rohf.F 2013-04-15 12:41:45.036852381 +1000
+++ src/ddscf/rohf.F 2013-04-15 12:23:28.100409225 +1000
@@ -153,7 +153,7 @@ c
             ilo = 1
             ihi = nmo
          endif
-         call movecs_print_anal(basis, ilo, ihi, 0.15d0, g_movecs, 
+         call movecs_print_anal(basis, ilo, ihi, 0.01d0, g_movecs, 
      $        'ROHF Final Molecular Orbital Analysis', 
      $        .true., dbl_mb(k_eval), oadapt, int_mb(k_irs),
      $        .true., dbl_mb(k_occ))
diff -rupN src.original/ddscf/scf_vec_guess.F src/ddscf/scf_vec_guess.F
--- src.original/ddscf/scf_vec_guess.F 2013-04-15 12:41:45.036852381 +1000
+++ src/ddscf/scf_vec_guess.F 2013-04-15 12:23:28.100409225 +1000
@@ -511,19 +511,19 @@ c
          nprint = min(nclosed+nopen+30,nmo)
          if (scftype.eq.'RHF' .or. scftype.eq.'ROHF') then
             call movecs_print_anal(basis, 1,
-     &           nprint, 0.15d0, g_movecs, 
+     &           nprint, 0.01d0, g_movecs, 
      &           'ROHF Initial Molecular Orbital Analysis', 
      &           .true., dbl_mb(k_eval), oadapt, int_mb(k_irs),
      &           .true., dbl_mb(k_occ))
          else
             nprint = min(nalpha+20,nmo)
             call movecs_print_anal(basis, max(1,nbeta-20),
-     &           nprint, 0.15d0, g_movecs, 
+     &           nprint, 0.01d0, g_movecs, 
      &           'UHF Initial Alpha Molecular Orbital Analysis', 
      &           .true., dbl_mb(k_eval), oadapt, int_mb(k_irs),
      &           .true., dbl_mb(k_occ))
             call movecs_print_anal(basis, max(1,nbeta-20),
-     &           nprint, 0.15d0, g_movecs(2), 
+     &           nprint, 0.01d0, g_movecs(2), 
      &           'UHF Initial Beta Molecular Orbital Analysis', 
      &           .true., dbl_mb(k_eval+nbf), oadapt, int_mb(k_irs+nmo),
      &           .true., dbl_mb(k_occ+nbf))
diff -rupN src.original/ddscf/uhf.F src/ddscf/uhf.F
--- src.original/ddscf/uhf.F 2013-04-15 12:41:45.036852381 +1000
+++ src/ddscf/uhf.F 2013-04-15 12:23:28.096409414 +1000
@@ -144,11 +144,11 @@ C
          enddo
          ihi = max(ihi-1,1)
  9611    continue
-         call movecs_print_anal(basis, ilo, ihi, 0.15d0, g_movecs, 
+         call movecs_print_anal(basis, ilo, ihi, 0.01d0, g_movecs, 
      $        'UHF Final Alpha Molecular Orbital Analysis', 
      $        .true., dbl_mb(k_eval), oadapt, int_mb(k_irs),
      $        .true., dbl_mb(k_occ))
-         call movecs_print_anal(basis, ilo, ihi, 0.15d0, g_movecs(2), 
+         call movecs_print_anal(basis, ilo, ihi, 0.01d0, g_movecs(2), 
      $        'UHF Final Beta Molecular Orbital Analysis', 
      $        .true., dbl_mb(k_eval+nbf), oadapt, int_mb(k_irs+nmo),
      $        .true., dbl_mb(k_occ+nbf))
diff -rupN src.original/mcscf/mcscf.F src/mcscf/mcscf.F
--- src.original/mcscf/mcscf.F 2013-04-15 12:41:45.000854073 +1000
+++ src/mcscf/mcscf.F 2013-04-15 12:23:23.748613695 +1000
@@ -719,7 +719,7 @@ c
       if (util_print('final vectors analysis', print_default))
      $     call movecs_print_anal(basis, 
      $     max(1,nclosed-10), min(nbf,nclosed+nact+10),
-     $     0.15d0, g_movecs, 'Analysis of MCSCF natural orbitals',
+     $     0.01d0, g_movecs, 'Analysis of MCSCF natural orbitals',
      $     .true., dbl_mb(k_evals), .true., int_mb(k_sym), 
      $     .true., dbl_mb(k_occ))
 c     
diff -rupN src.original/nwdft/scf_dft/dft_mxspin_ovlp.F src/nwdft/scf_dft/dft_mxspin_ovlp.F
--- src.original/nwdft/scf_dft/dft_mxspin_ovlp.F 2013-04-15 12:41:45.604825677 +1000
+++ src/nwdft/scf_dft/dft_mxspin_ovlp.F 2013-04-15 12:23:28.228403211 +1000
@@ -184,14 +184,14 @@ c
       call ga_sync()
 c
       call movecs_print_anal(basis,int_mb(k_non),int_mb(k_non)
-     & ,0.15d0,g_alpha,'Alpha Orbitals without Beta Partners',
+     & ,0.01d0,g_alpha,'Alpha Orbitals without Beta Partners',
      &   .false., 0.0 ,.false., 0 , .false., 0 )
 c
       if (nct.GE.2) then
       do i = 2,nct
       ind = int_mb(k_non+i-1)
       call movecs_print_anal(basis,ind,ind
-     & ,0.15d0,g_alpha,' ',
+     & ,0.01d0,g_alpha,' ',
      &   .false., 0.0 ,.false., 0 , .false., 0 )
       enddo
       endif
@@ -350,7 +350,7 @@ c      endif
 c      endif
 c 9990 format(/,18x,'THERE ARE',i3,1x,'UN-PARTNERED ALPHA ORBITALS')
 c
-       call movecs_print_anal(basis, 1, nalp, 0.15d0, g_ualpha,
+       call movecs_print_anal(basis, 1, nalp, 0.01d0, g_ualpha,
      & 'Alpha Orb. w/o Beta Partners (after maxim. alpha/beta overlap)',
      &   .false., 0.0 ,.false., 0 , .false., 0 )
 c
diff -rupN src.original/nwdft/scf_dft/dft_scf.F src/nwdft/scf_dft/dft_scf.F
--- src.original/nwdft/scf_dft/dft_scf.F 2013-04-15 12:41:45.608825490 +1000
+++ src/nwdft/scf_dft/dft_scf.F 2013-04-15 12:23:28.228403211 +1000
@@ -1774,7 +1774,7 @@ c
             else
                blob='DFT Final Beta Molecular Orbital Analysis' 
             endif
-            call movecs_print_anal(ao_bas_han, ilo, ihi, 0.15d0, 
+            call movecs_print_anal(ao_bas_han, ilo, ihi, 0.01d0, 
      &           g_movecs(ispin), 
      &           blob, 
      &           .true., dbl_mb(k_eval(ispin)), oadapt, 
diff -rupN src.original/nwdft/scf_dft_cg/dft_cg_solve.F src/nwdft/scf_dft_cg/dft_cg_solve.F
--- src.original/nwdft/scf_dft_cg/dft_cg_solve.F 2013-04-15 12:41:45.612825303 +1000
+++ src/nwdft/scf_dft_cg/dft_cg_solve.F 2013-04-15 12:23:28.220403588 +1000
@@ -183,7 +183,7 @@ c
             blob = 'DFT Final Beta Molecular Orbital Analysis'
           endif
           call movecs_fix_phase(g_movecs(ispin))
-          call movecs_print_anal(basis, ilo, ihi, 0.15d0,
+          call movecs_print_anal(basis, ilo, ihi, 0.01d0,
      &         g_movecs(ispin),blob,
      &         .true., dbl_mb(k_eval+(ispin-1)*nbf),
      &         oadapt, int_mb(k_irs+(ispin-1)*nbf),

---

Compile NWChem
This examples uses the ACML libs. See e.g. this post for openblas settings.

sudo apt-get install build-essential gfortran python2.7-dev libopenmpi-dev openmpi-bin
sudo mkdir /opt/nwchem
sudo chown $USER:$USER /opt/nwchem
cd /opt/nwchem/
wget http://www.nwchem-sw.org/download.php?f=Nwchem-6.3-src.2013-05-17.tar.gz
mv download.php\?f\=Nwchem-6.3-src.2013-05-17.tar.gz Nwchem-6.3-src.2013-05-17.tar.gz
tar xvf Nwchem-6.3-src.2013-05-17.tar.gz
cd nwchem-6.3-src.2013-05-17/
patch -p0 < diff.patch

patching file src/config/makefile.h
patching file src/ddscf/movecs_pr_anal.F
patching file src/ddscf/rohf.F
patching file src/ddscf/scf_vec_guess.F
patching file src/ddscf/uhf.F
patching file src/mcscf/mcscf.F
patching file src/nwdft/scf_dft/dft_mxspin_ovlp.F
patching file src/nwdft/scf_dft/dft_scf.F
patching file src/nwdft/scf_dft_cg/dft_cg_solve.F

cd src/
wget http://www.nwchem-sw.org/images/Iswtch.patch.gz
gzip -d Iswtch.patch
patch -p0 < Iswtch.patch
cd ../
export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export PYTHONVERSION=2.7
export PYTHONHOME=/usr
export BLASOPT="-L/opt/acml/acml5.3.1/gfortran64_int64/lib -lacml"

export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export LIBRARY_PATH="$LIBRARY_PATH:/usr/lib/openmpi/lib:/opt/acml/acml5.3.1/gfortran64_int64/lib"
export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"

export ARMCI_NETWORK=SOCKETS

cd $NWCHEM_TOP/src
make clean
make nwchem_config
make FC=gfortran 1> make.log 2>make.err
cd $NWCHEM_TOP/contrib
export FC=gfortran
./getmem.nwchem

---

Settings
Create /opt/nwchem/default.nwchemrc

nwchem_basis_library /opt/nwchem/nwchem-6.3-src.2013-05-17/src/basis/libraries/
ffield amber
amber_1 /opt/nwchem/nwchem-6.3-src.2013-05-17/src/data/amber_s/
amber_2 /opt/nwchem/nwchem-6.3-src.2013-05-17/src/data/amber_x/
amber_3 /opt/nwchem/nwchem-6.3-src.2013-05-17/src/data/amber_q/
amber_4 /opt/nwchem/nwchem-6.3-src.2013-05-17/src/data/amber_u/
amber_5 /opt/nwchem/nwchem-6.3-src.2013-05-17/src/data/custom/
spce /opt/nwchem/nwchem-6.3-src.2013-05-17/src/data/solvents/spce.rst
charmm_s /opt/nwchem/nwchem-6.3-src.2013-05-17/src/data/charmm_s/
charmm_x /opt/nwchem/nwchem-6.3-src.2013-05-17/src/data/charmm_x/

Symmlink to this file in each user's home:

ln -s /opt/nwchem/default.nwchemrc ~/.nwchemrc

423. Openblas on Debian Wheezy

Update 23 May 2013: the url for the openblas tarball has been updated and a tab has been removed from the lapack url.

Note that the links to the openblas file tends to die after a while, so you might have to download it manually.

Original post:
I need more modular posts. I've covered how to build openblas several times before, but I need a simple post I can use as a reference. So here it is:

openblas

sudo mkdir /opt/openblas
sudo chown ${USER} /opt/openblas
cd ~/tmp
wget http://github.com/xianyi/OpenBLAS/tarball/v0.2.6

tar xvf v0.2.6
cd xianyi-OpenBLAS-87b4d0c

wget http://www.netlib.org/lapack/lapack-3.4.1.tgz
make all BINARY=64 CC=/usr/bin/gcc FC=/usr/bin/gfortran USE_THREAD=0 INTERFACE64=1 1> make.log 2>make.err
make PREFIX=/opt/openblas install

nm: '/lib/libpthread.so*': No such file
make -j 2 -f Makefile.install install
make[1]: Entering directory `/home/me/tmp/xianyi-OpenBLAS-87b4d0c'
Generating openblas_config.h in /opt/openblas/include
Generating f77blas.h in /opt/openblas/include
Generating cblas.h in /opt/openblas/include
Copying LAPACKE header files to /opt/openblas/lib
Copy the static library to /opt/openblas/lib
cp: cannot stat `libopenblas_opteronp-r0.2.6.a': No such file or directory
make[1]: *** [install] Error 1
make[1]: Leaving directory `/home/me/tmp/xianyi-OpenBLAS-87b4d0c'
make: *** [install] Error 2

Don't worry about the errors -- as long as you have libopenblas_PROCESSOR_rX.X.X.so, you're fine.

cp lib*.* /opt/openblas/lib

add

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/openblas/lib

to your ~/.bashrc, and/or add

/opt/openblas/lib

to your /etc/ld.so.conf

422. Set up ACML on linux

These are the same instructions as in post 409B. However, I've decided it's better to do the posts the unix/linux way -- have the do one thing, and do that thing well. It makes life easier for me if I can simply refer back to more modular posts.

Anyway, here's how to set up the ACML libs on debian.

ACML
Download both the 'regular' and the int64 gfortran packages from AMD:
http://developer.amd.com/tools-and-sdks/cpu-development/amd-core-math-library-acml/acml-downloads-resources/#download

tar xvf acml-5-3-1-gfortran-64bit-int64.tgz
tar xvf acml-5-3-1-gfortran-64bit.tgz
sh install-acml-5-3-1-gfortran-64bit-int64.sh

Where do you want to install ACML?  Press return to use
the default location (/opt/acml5.3.1), or enter an alternative path.
The directory will be created if it does not already exist.
> /opt/acml/acml5.3.1

sh install-acml-5-3-1-gfortran-64bit.sh

Where do you want to install ACML?  Press return to use
the default location (/opt/acml5.3.1), or enter an alternative path.
The directory will be created if it does not already exist.
> /opt/acml/acml5.3.1

You'll get something like this:

/opt/acml/acml5.3.1
|-- Doc
|-- gfortran64
|-- gfortran64_fma4
|-- gfortran64_fma4_int64
|-- gfortran64_fma4_mp
|-- gfortran64_fma4_mp_int64
|-- gfortran64_int64
|-- gfortran64_mp
|-- gfortran64_mp_int64
`-- util

where
*  fma4 is for cpus with FMA4 support (use util/cpuid to check)
*  int64 is for double-precision float (integer*8) I think
*  mp is for openmp. For MPI do not use the _mp_ libraries!

Pick your library/ies and add them to the LD_LIBRARY_PATH, e.g.:

echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/acml/acml5.3.1/gfortran64_int64/lib' >> ~/.bashrc
source ~/.bashrc

421. NWChem 6.3 on ROCKS 5.4.3/CentOS 5.6

Update 23 May 2013: The execution times are pretty much the same as for 6.1.1 with a new patch. I've updated the instructions below to incorporate this new patch (http://www.nwchem-sw.org/images/Iswtch.patch.gz)

Update 21 May 2013:
The execution times can be improved considerably by setting

ARMCI_NETWORK=SOCKETS

They are still ca 30% longer than 6.1.1 though due to slower SCF convergence.
See http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id834/Nwchem_6.3_running_2-5_times_slo....html

UPDATE 20 May 2013:
Nwchem 6.3 is very slow compared to 6.1.1. A six-core run (out of eight cores available) was 121 s using 6.1.1 but 254 seconds on 6.3!

I observed this on debian as well: 6.3 on debian is five times slower (190s vs 40 s for example at 8 cores in http://verahill.blogspot.com.au/2013/05/414-frequency-vs-cores-crude.html) than 6.1.1. Not sure why that is.

Original:
NWChem 6.3 is out now. Here's how to build it on ROCKS 5.4.3 (based on Centos 5.6) for CPU-based calculations (currently only CCSD(T) can take advantage of GPU/CUDA anyway).

To build on debian, see http://verahill.blogspot.com.au/2013/05/424-nwchem-63-on-debian-wheezy.html

This assumes that you've got a proper build environment (gcc, fortran, openmpi) installed.

---

Openblas:
I've added all users who do computations to the group compchem.

sudo mkdir /share/apps/openblas
sudo chown $USER:compchem /share/apps/openblas
cd ~/tmp
wget http://nodeload.github.com/xianyi/OpenBLAS/tarball/v0.1.1
tar xvf v0.1.1
cd xianyi-OpenBLAS-e6e87a2/
wget http://www.netlib.org/lapack/lapack-3.4.1.tgz
make all BINARY=64 CC=/usr/bin/gcc FC=/usr/bin/gfortran USE_THREAD=0 INTERFACE64=1 1> make.log 2>make.err

make PREFIX=/share/apps/openblas install
cp lib*.*  /share/apps/openblas/lib
sudo chmod 755 /share/apps/openblas -R

For later use with nwchem and ecce, add /share/apps/openblas/lib to /etc/ld.so.conf and do

sudo ldconfig

Put

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/share/apps/openblas/lib 

in ~/.bashrc and/or queue files.

---

NWChem
I've added all users who do computations to the group compchem.

sudo mkdir /share/apps/nwchem/
sudo chown $USER:compchem /share/apps/nwchem/

cd /share/apps/nwchem
wget http://www.nwchem-sw.org/download.php?f=Nwchem-6.3-src.2013-05-17.tar.gz
tar xvf Nwchem-6.3-src.2013-05-17.tar.gz 
cd nwchem-6.3-src.2013-05-17/
cd src/
wget http://www.nwchem-sw.org/images/Iswtch.patch.gz
gzip -d Iswtch.patch
patch -p0 < Iswtch.patch
cd ../
export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export PYTHONHOME=/opt/rocks
export PYTHONVERSION=2.4
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/opt/openmpi
export MPI_INCLUDE=/opt/openmpi/include
export LIBRARY_PATH=$LIBRARY_PATH:/opt/openmpi/lib:/share/apps/openblas
export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"
export BLASOPT="-L/share/apps/openblas/lib -lopenblas -lopenblas_nehalem-r0.1.1 -lopenblas_nehalemp-r0.1.1"

export ARMCI_NETWORK=SOCKETS

cd $NWCHEM_TOP/src
export FC=gfortran
make clean
make  nwchem_config
make  FC=gfortran
cd ../contrib
./getmem.nwchem
 sudo chmod 755 /share/apps/nwchem/nwchem-6.3-src.2013-05-17 -R

Create a default.nwchemrc in /share/apps/nwchem

nwchem_basis_library /share/apps/nwchem/nwchem-6.3-src.2013-05-17/src/basis/libraries/
ffield amber
amber_1 /share/apps/nwchem/nwchem-6.3-src.2013-05-17/src/data/amber_s/
amber_2 /share/apps/nwchem/nwchem-6.3-src.2013-05-17/src/data/amber_x/
amber_3 /share/apps/nwchem/nwchem-6.3-src.2013-05-17/src/data/amber_q/
amber_4 /share/apps/nwchem/nwchem-6.3-src.2013-05-17/src/data/amber_u/
amber_5 /share/apps/nwchem/nwchem-6.3-src.2013-05-17/src/data/custom/
spce /share/apps/nwchem/nwchem-6.3-src.2013-05-17/src/data/solvents/spce.rst
charmm_s /share/apps/nwchem/nwchem-6.3-src.2013-05-17/src/data/charmm_s/
charmm_x /share/apps/nwchem/nwchem-6.3-src.2013-05-17/src/data/charmm_x/

and put symmlinks to it in the users' home directories, e.g.

cd ~
ln -s /share/apps/nwchem/default.nwchemrc .nwchemrc

---

420. Setting up a FreeBSD virtual machine

I don't have any free hardware to make a proper install on at the moment, but have a few things I'd like to try out on Free BSD (basically -- how similar/different is the build process from e.g. debian) so I've decided to set up a virtual machine to play around with.

Not difficult, but someone else might benefit from the screenshots as well.

---

1. Get the installation medium:
Go to ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/amd64/ISO-IMAGES/9.1/ and download what you need. Interestingly there's an image meant for USB sticks as well.

I went for the CD iso which is around 686 Mb:

cd ~/Downloads
wget ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/amd64/ISO-IMAGES/9.1/FreeBSD-9.1-RELEASE-amd64-disc1.iso

---

2. Set up the virtual machine

 You need to change the CPU type from the default (http://forums.freebsd.org/showthread.php?t=25032):

If you don't change the CPU type you'll encounter the following on boot:

---

3. Start the installation

---

4. Final touches
I don't want to have to launch and work in VM via the virtualbox GUI. Instead, we want to launch the VM from the command line, and log in via ssh. I've covered this in part before in post 94. We set up port forwarding in the Virtualbox GUI:

Start the VM
VBoxHeadless -s FreeBSD

Oracle VM VirtualBox Headless Interface 4.2.12
(C) 2008-2013 Oracle Corporation
All rights reserved.

VRDE server is listening on port 3389.

Wait ca 30-60 seconds for the VM to boot (you won't get any feedback), then

ssh verahill@localhost -p 6534

Password:
Last login: Fri May 17 13:50:03 2013
FreeBSD 9.1-RELEASE (GENERIC) #0 r243825: Tue Dec  4 09:23:10 UTC 2012

Welcome to FreeBSD!

Before seeking technical support, please use the following resources:

o  Security advisories and updated errata information for all releases are
   at http://www.FreeBSD.org/releases/ - always consult the ERRATA section
   for your release first as it's updated frequently.

o  The Handbook and FAQ documents are at http://www.FreeBSD.org/ and,
   along with the mailing lists, can be searched by going to
   http://www.FreeBSD.org/search/.  If the doc package has been installed
   (or fetched via pkg_add -r lang-freebsd-doc, where lang is the
   2-letter language code, e.g. en), they are also available formatted
   in /usr/local/share/doc/freebsd.

If you still have a question or problem, please take the output of
`uname -a', along with any relevant error messages, and email it
as a question to the questions@FreeBSD.org mailing list.  If you are
unfamiliar with FreeBSD's directory layout, please refer to the hier(7)
manual page.  If you are not familiar with manual pages, type `man man'.

Edit /etc/motd to change this login announcement.

$ 

---

5. Basic set-up of FreeBSD
I'm a sudo-addict, and I can't live without vim. Log in as ROOT:

pkg

The package management tools is not yet installed on your system.
Do you want to fetch and install it now ? [y/N]: Y
Bootstrapping pkg, please wait.

pkg update

Updating repository catalogue

pkg_add -r sudo vim
echo 'verahill ALL=(ALL) ALL' >> /usr/local/etc/sudoers

If you want a graphical environment, you can follow this page: http://www.freebsd.org/doc/en/books/handbook/x11-wm.html
I installed xfce:

pkg_add -r xfce4
pkg_add -r xorg
echo "/usr/local/bin/startxfce4" > /home/verahill/.xinitrc
chown verahill:verahill /home/verahill/.xinitrc
echo 'hald_enable="YES"' >> /etc/rc.conf
echo 'dbus_enable="YES"' >> /etc/rc.conf

Restart and log in. You can launch XFCE with startx, which opens XFCE at tty9

Note that the terminal emulator didn't work properly out of the box.

sudo pkg_add -r virtualbox-ose-additions
sudo pw groupmod wheel -m verahill
sudo su
#echo 'vboxguest_enable="YES"' >> /etc/rc.conf
#echo 'vboxservice_enable="YES"' >> /etc/rc.conf
sudo shutdown -r now

---

Links to this post:
http://forum.opensource-srbija.org/topic/1870-freebsd-u-virtual-box-u/

419. Talking to Myself in Public: DKMS, your own kernel and Virtualbox

Brief version:

check your /lib/modules/KERNEL/build symmlinks if you compile your own kernel.

Original post:

A while back I installed the Oracle-packaged version Virtualbox 4.2 on my system since I had issues with the debian version when going above kernel 3.6 (http://verahill.blogspot.com.au/search?q=kernel+virtualbox)

It's been a bit annoying since I've had to run

sudo service vboxdrv setup

manually every time I upgrade the kernel, but it's not that much hassle. However, I've been seeing error messages such as

Error! Could not locate dkms.conf file.
File:  does not exist.

In addition, I've just upgraded the kernel from 3.8.8-ck1 to 3.9.2-ck1, but not rebooted, so there's no vboxdrv module for the running kernel at the moment. Both kernels were compiled on the machine they were running on.
Anyway, typically I should be able to solve that by doing

sudo dkms autoinstall -k 3.8.8-ck1

but again I get the above error about no dksm.conf file.

---

Troubleshooting:

The debian version
virtualbox-dkms belongs to the debian version and isn't needed for the oracle version -- installing it removes the Oracle version. Anyway, to remind myself of why I wasn't using the debian version (4.1.18-dfsg-2+deb7u1)) in Wheezy I did

sudo apt-get install virtualbox-dkms

which remove the oracle version and installed the debian one.

Building initial module for 3.9.2-ck1
Error! Bad return status for module build on kernel: 3.9.2-ck1 (x86_64)
Consult /var/lib/dkms/virtualbox/4.1.18/build/make.log for more information.
dpkg: error processing virtualbox-dkms (--configure):
 subprocess installed post-installation script returned error exit status 10

dmesg shows

[704511.419271] vboxdrv: disagrees about version of symbol module_layout

OK. False starts.

I then purged virtualbox and reinstalled the debian version:

sudo apt-get purge virtualbox
sudo apt-get install virtualbox-dkms

Same problem.
Looking at https://bbs.archlinux.org/viewtopic.php?id=151965

for i in /var/lib/dkms/*/[^k]*/source; do [ -e "$i" ] || echo "$i";done

/var/lib/dkms/vboxhost/4.1.10/source
/var/lib/dkms/vboxhost/4.2.8/source

sudo rm /var/lib/dkms/vboxhost/4.1.10/source
sudo rm /var/lib/dkms/vboxhost/4.2.8/source

sudo apt-get install virtualbox-dkms

Error! Bad return status for module build on kernel: 3.9.2-ck1 (x86_64)
Consult /var/lib/dkms/virtualbox/4.1.18/build/make.log for more information.
dpkg: error processing virtualbox-dkms (--configure):

sudo less /var/lib/dkms/virtualbox/4.1.18/build/make.log

CC [M]  /var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/r0drv/linux/memobj-r0drv-linux.o
/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/r0drv/linux/memobj-r0drv-linux.c: In function ‘rtR0MemObjNativeMapUser’:
/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/r0drv/linux/memobj-r0drv-linux.c:1451:38: error: ‘VM_RESERVED’ undeclared (first use in this function)
/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/r0drv/linux/memobj-r0drv-linux.c:1451:38: note: each undeclared identifier is reported only once for each function it appears in
make[2]: *** [/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv/r0drv/linux/memobj-r0drv-linux.o] Error 1
make[1]: *** [/var/lib/dkms/virtualbox/4.1.18/build/vboxdrv] Error 2
make: *** [_module_/var/lib/dkms/virtualbox/4.1.18/build] Error 2
make: Leaving directory `/home/me/tmp/linux-3.9.2'

The issues is known
http://siduction.org/index.php?name=PNphpBB2&file=viewtopic&t=3097
http://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg1086030.html
http://forums.linuxmint.com/viewtopic.php?f=47&t=52496

The oracle version
I gave up and downloaded the newest oracle version again:

wget http://download.virtualbox.org/virtualbox/4.2.12/virtualbox-4.2_4.2.12-84980~Debian~wheezy_amd64.deb
sudo dpkg -i virtualbox-4.2_4.2.12-84980~Debian~wheezy_amd64.deb

Trying to register the VirtualBox kernel modules using DKMS:Error! Your kernel headers for kernel 3.8.8-ck1 cannot be found.
Please install the linux-headers-3.8.8-ck1 package,
or use the --kernelsourcedir option to tell DKMS where it's located

Failed, trying without DKMS ... failed!
Recompiling VirtualBox kernel modules:.
Starting VirtualBox kernel modules:
modprobe vboxdrv failed. Please use 'dmesg' to find out why ... failed!

That last 'failed' is actually ok -- I think it built, but only for 3.9.2-ck1 while the running kernel is 3.8.8-ck1.

So, good enough to use, but not good enough for a blog post, since we're back to square one.
Hmm...

ls /lib/modules/3.8.8-ck1/build -lah
/lib/modules/3.8.8-ck1/build -> /home/me/tmp/ck-kernel/linux-3.8.8

Sure, that where I built it, but it should be pointing at /usr/src/linux-headers-3.8.8-ck1

sudo rm /lib/modules/3.8.8-ck1/build
sudo ln -s /usr/src/linux-headers-3.8.8-ck1/ /lib/modules/3.8.8-ck1/build

I then did the whole purge/install dance again:

sudo apt-get purge virtualbox-4.2
sudo dpkg -i virtualbox-4.2_4.2.12-84980~Debian~wheezy_amd64.deb

Trying to register the VirtualBox kernel modules using DKMS:.
Starting VirtualBox kernel modules:.

dkms status

nvidia, 304.88, 3.8.0, x86_64: installed
nvidia, 304.88, 3.8.8-ck1, x86_64: installed
nvidia, 304.88, 3.8.8, x86_64: installed
nvidia, 304.88, 3.9.2-ck1, x86_64: installed
vboxhost, 4.2.12, 3.8.8-ck1, x86_64: installed

sudo rm /lib/modules/3.9.2-ck1/build
sudo ln -s /usr/src/linux-headers-3.9.2-ck1/ /lib/modules/3.9.2-ck1/build
sudo dkms autoinstall -k 3.9.2-ck1
dkms status

nvidia, 304.88, 3.8.0, x86_64: installed
nvidia, 304.88, 3.8.8-ck1, x86_64: installed
nvidia, 304.88, 3.8.8, x86_64: installed
nvidia, 304.88, 3.9.2-ck1, x86_64: installed
vboxhost, 4.2.12, 3.8.8-ck1, x86_64: installed
vboxhost, 4.2.12, 3.9.2-ck1, x86_64: installed

Happy again. And no dkms.conf errors anymore!