75. [solved] Problems with homebuilt nwchem 6.1 on Debian Testing

EDIT 18 May 2012: 
It's now been solved
Compiling nwchem 6.1 with internal libs on debian:
 http://verahill.blogspot.com.au/2012/05/compiling-nwchem-61-with-internal-libs.html
Compiling nwchem 6.1 with openblas on debian:
 http://verahill.blogspot.com.au/2012/05/building-nwchem-61-on-debian.html


UPDATE April 2012: Someone else is having the same problem: http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id435/ . Binaries built on ROCKS 5.4.3 work, but binaries built on debian testing don't: the gfortran version is GNU Fortran (GCC) 4.1.2 20080704 (Red Hat 4.1.2-50). On debian, which yields a segfaulting binary, the version is GNU Fortran (Debian 4.6.3-1) 4.6.3


Nwchem 6.1 was released in February this year. The build instructions are ALMOST the same as for Nwchem 6.0 -- the difference is the use of export USE_MPIF4=y. Well, that and me not having much success in actually USING nwchem as opposed to building it.

There is now an nwchem version with mpi support in the debian unstable repos. I have not used or tested it.

I can build the 32 bit version of nwchem 6.1 just fine.Building the 64 bit version works absolutely fine too. However, once you attempt to run, it crashes. Ergo, this is NOT A SOLUTION. It's a bunch of error messages so that more seasoned and skilled operators than I may offer a solution. If you have an option, build and use version 6.0 instead.

Update:
I built a version with openmpi support as well, which also segfaults:
Here are the build instructions:

sudo apt-get install openmpi-bin openmpi-dev
export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=/home/me/tmp/nwchem-6.1
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES=all
export USE_MPI=y
export USE_MPIF=y
export MPI_LOC=/usr/lib/openmpi
export MPI_INCLUDE=/usr/lib/openmpi/include
export USE_MPIF4=y
export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openmpi/lib
export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77"
cd $NWCHEM_TOP/src
make clean
make  nwchem_config
make  FC=gfortran


and here's what happens on execution

[beryllium:24650] *** Process received signal ***
[beryllium:24650] Signal: Segmentation fault (11)
[beryllium:24650] Signal code: Address not mapped (1)
[beryllium:24650] Failing at address: 0x44000098
[beryllium:24650] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x324f0) [0x7f08deeb84f0]
[beryllium:24650] [ 1] /usr/lib/libmpi.so.0(PMPI_Comm_set_errhandler+0x60) [0x7f08e0526c30]
[beryllium:24650] [ 2] ./nwchem() [0x292d504]
[beryllium:24650] [ 3] ./nwchem() [0x292d596]
[beryllium:24650] [ 4] ./nwchem() [0x40657a]
[beryllium:24650] [ 5] ./nwchem() [0x406f7d]
[beryllium:24650] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f08deea4ead]
[beryllium:24650] [ 7] ./nwchem() [0x405189]
[beryllium:24650] *** End of error message ***

This only happens on 64 bit debian -- 32 bit deb and 64 bit centos are both fine

OLD POST:

--start here --

Here's what I've done so far

Put a hold on your mpich2 and mpich2-dev packages (see e.g. here for more details)
1. edit your /etc/apt/sources.list to allow packages from stable e.g.

deb ftp://ftp.au.debian.org/debian/ testing main contrib non-fre
deb ftp://ftp.au.debian.org/debian/ stable main contrib non-free

2. create an /etc/apt/preferences file e.g.

Package: *
Pin: release a=testing
Pin-Priority: 990
Package: *
Pin: release a=stable
Pin-Priority: -10

2. install v 1.2 explicitly

sudo apt-get update && sudo apt-get install mpich2=1.2.1.1-5 libmpich2-dev=1.2.1.1-5

3. put a hold on the packages

sudo su
echo "mpich2 hold"|dpkg --set-selections
echo "libmpich2-dev hold"|dpkg --set-selections

exit

Download the nwchem source
cd ~
wget http://www.nwchem-sw.org/images/Nwchem-6.1-2012-Feb-10.tar.gz
tar -xvf Nwchem-6.1-2012-Feb-10.tar.gz
cd nwchem-6.1

create buildconf.sh in ~/nwchem-6.1
Put the following in it (for 64 bit system):

export LARGE_FILES=TRUE
export TCGRSH=/usr/local/bin/ssh
export NWCHEM_TOP=/home/me/nwchem-6.1
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES=all
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/usr
export MPI_LIB=$MPI_LOC/lib
export MPI_INCLUDE=$MPI_LOC/include/mpich2
export LIBMPI="-lmpich -lfmpich"
export NWCHEM_MODULES="all"
cd $NWCHEM_TOP/src
make clean
make nwchem_config
make FC=gfortran

Build
Start the build
sh buildconf.sh

Building takes about half an hour. Everything builds fine. However, running -- with or without mpdrun -- causes the error below.

It doesn't matter how much memory I allocated. The error seems to have something to do with "Invalid write of size 8" which I understand to mean that pointers are 8 bytes long but don't have 8 bytes allocated to them. But then I'm not an expert.

Would it have something to do with
USE_MPIF4=y?

Without USE_MPIF4 I end up with the stupid_* error messages (stupid_sum, stupid_task etc.)

---

Error:
running e.g.  mpdrun -n 2 nwchem nwchem.nw gives:

      Screening Tolerance Information
      -------------------------------
          Density screening/tol_rho: 1.00D-10
          AO Gaussian exp screening on grid/accAOfunc:  14
          CD Gaussian exp screening on grid/accCDfunc:  20
          XC Gaussian exp screening on grid/accXCfunc:  20
          Schwarz screening/accCoul: 1.00D-08

0:Segmentation Violation error, status=: 11
(rank:0 hostname:tantalum pid:19944):ARMCI DASSERT fail. ../../ga-5-1/armci/src/common/signaltrap.c:SigSegvHandler():310 cond:0

More detail:
Running just nwchem nwchem.nw gives a bit more detail:

      Screening Tolerance Information
      -------------------------------
          Density screening/tol_rho: 1.00D-10
          AO Gaussian exp screening on grid/accAOfunc:  14
          CD Gaussian exp screening on grid/accCDfunc:  20
          XC Gaussian exp screening on grid/accXCfunc:  20
          Schwarz screening/accCoul: 1.00D-08

0:Segmentation Violation error, status=: 11
(rank:0 hostname:tantalum pid:19676):ARMCI DASSERT fail. ../../ga-5-1/armci/src/common/signaltrap.c:SigSegvHandler():310 cond:0
application called MPI_Abort(comm=0x84000001, 11) - process 0
*** glibc detected *** nwchem: corrupted double-linked list: 0x000000010ac34880 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x75ab6)[0x7f597b129ab6]
/lib/x86_64-linux-gnu/libc.so.6(+0x7754c)[0x7f597b12b54c]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x6c)[0x7f597b12e7ec]
/usr/lib/x86_64-linux-gnu/libgfortran.so.3(+0xcc811)[0x7f597bbd5811]
/usr/lib/x86_64-linux-gnu/libgfortran.so.3(+0xdba7f)[0x7f597bbe4a7f]
/usr/lib/x86_64-linux-gnu/libgfortran.so.3(+0xdbbaa)[0x7f597bbe4baa]
/usr/lib/x86_64-linux-gnu/libgfortran.so.3(+0x1ab09)[0x7f597bb23b09]
/lib64/ld-linux-x86-64.so.2(+0xe21c)[0x7f597c42421c]
/lib/x86_64-linux-gnu/libc.so.6(+0x36df2)[0x7f597b0eadf2]
/lib/x86_64-linux-gnu/libc.so.6(+0x36e45)[0x7f597b0eae45]
/usr/lib/libmpich.so.1.2(+0xbedc9)[0x7f597c101dc9]
/usr/lib/libmpich.so.1.2(MPID_Abort+0x6d)[0x7f597c122d0d]
/usr/lib/libmpich.so.1.2(PMPI_Abort+0x2f5)[0x7f597c090805]
nwchem[0x2896591]
nwchem[0x2883883]
/lib/x86_64-linux-gnu/libc.so.6(+0x324f0)[0x7f597b0e64f0]
nwchem[0x29b6043]
nwchem[0x27a04a0]
nwchem[0x27a3955]
nwchem[0x271492b]
nwchem[0x5cf410]
nwchem[0x5b3d18]
nwchem[0x5a9735]
nwchem[0x5a99b6]
nwchem[0x418ee8]
nwchemAborted

And more detail:

valgrind nwchem nwchem.nw



==19910==    by 0x5CF40F: diis_bld12_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)                                          
==19910==    by 0x5B3D17: dft_main0d_ (dft_main0d.F:549)                                                                      
==19910==    by 0x5A9734: nwdft_ (nwdft.F:274)                                                                                
==19910==    by 0x5A99B5: dft_energy_ (nwdft.F:18)                                                                            
==19910==    by 0x418EE7: task_energy_doit_ (task_energy.F:251)                                                              
==19910==    by 0x41A57B: task_energy_ (task_energy.F:95)                                                                    
==19910==    by 0x40DAD2: task_ (task.F:337)                                                                                  
==19910==    by 0x4068F5: MAIN__ (nwchem.F:251)                                                                              
==19910==  Address 0x199750a0 is not stack'd, malloc'd or (recently) free'd                                                  
==19910==                                                                                                                    
==19910== Invalid write of size 8                                                                                            
==19910==    at 0x29B6043: dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)                                              
==19910==    by 0x27A049F: GAI_DGEMM (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)                                            
==19910==    by 0x27A3954: pnga_matmul (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)                                          
==19910==    by 0x271492A: ga_dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)                                            
==19910==    by 0x5CF40F: diis_bld12_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)                                          
==19910==    by 0x5B3D17: dft_main0d_ (dft_main0d.F:549)                                                                      
==19910==    by 0x5A9734: nwdft_ (nwdft.F:274)                                                                                
==19910==    by 0x5A99B5: dft_energy_ (nwdft.F:18)                                                                            
==19910==    by 0x418EE7: task_energy_doit_ (task_energy.F:251)                                                              
==19910==    by 0x41A57B: task_energy_ (task_energy.F:95)                                                                    
==19910==    by 0x40DAD2: task_ (task.F:337)                                                                                  
==19910==    by 0x4068F5: MAIN__ (nwchem.F:251)                                                                              
==19910==  Address 0x199750b0 is not stack'd, malloc'd or (recently) free'd                                                  
==19910==                                                                                                                    
0:Segmentation Violation error, status=: 11                                                                                  
(rank:0 hostname:tantalum pid:19910):ARMCI DASSERT fail. ../../ga-5-1/armci/src/common/signaltrap.c:SigSegvHandler():310 cond:0
application called MPI_Abort(comm=0x84000001, 11) - process 0                                                                
==19910==                                                                                                                    
==19910== HEAP SUMMARY:                                                                                                      
==19910==     in use at exit: 4,303,284,335 bytes in 695 blocks                                                              
==19910==   total heap usage: 2,132 allocs, 1,437 frees, 4,305,897,103 bytes allocated                                        
==19910==                                                                                                                    
==19910== LEAK SUMMARY:                                                                                          
==19910==    definitely lost: 24 bytes in 1 blocks
==19910==    indirectly lost: 512 bytes in 1 blocks
==19910==      possibly lost: 0 bytes in 0 blocks
==19910==    still reachable: 4,303,283,799 bytes in 693 blocks
==19910==         suppressed: 0 bytes in 0 blocks
==19910== Rerun with --leak-check=full to see details of leaked memory
==19910==
==19910== For counts of detected and suppressed errors, rerun with: -v
==19910== Use --track-origins=yes to see where uninitialised values come from
==19910== ERROR SUMMARY: 662 errors from 9 contexts (suppressed: 4 from 4)

And way too much detail:
valgrind --leak-check=full --track-origins=yes --log-file=valgrind.log nwchem nwchem.nw
==20005== Memcheck, a memory error detector

==20005== Memcheck, a memory error detector
==20005== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
==20005== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info
==20005== Command: nwchem nwchem.nw
==20005== Parent PID: 19563
==20005==
==20005== Warning: set address range perms: large range [0x3952b040, 0x13352b110) (undefined)
==20005== Syscall param write(buf) points to uninitialised byte(s)
==20005==    at 0x12803980: __write_nocancel (syscall-template.S:82)
==20005==    by 0x127A8B92: _IO_file_write@@GLIBC_2.2.5 (fileops.c:1276)
==20005==    by 0x127A8809: new_do_write (fileops.c:530)
==20005==    by 0x127A8B34: _IO_do_write@@GLIBC_2.2.5 (fileops.c:503)
==20005==    by 0x127A9347: _IO_file_sync@@GLIBC_2.2.5 (fileops.c:905)
==20005==    by 0x1279DE19: fflush (iofflush.c:43)
==20005==    by 0xA3CF1F: hdbm_file_flush (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x8C84AA: rtdb_seq_put (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x8C55BD: rtdb_put (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x8C48B2: rtdb_put_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x9EAFAC: util_set_rtdb_state_ (util_rtdb_state.F:40)
==20005==    by 0x4067FD: MAIN__ (nwchem.F:222)
==20005==  Address 0x10950022 is not stack'd, malloc'd or (recently) free'd
==20005==  Uninitialised value was created by a stack allocation
==20005==    at 0x8C6130: rtdb_seq_put_info (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==
==20005== Invalid write of size 8
==20005==    at 0x29B6048: dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A049F: GAI_DGEMM (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A3954: pnga_matmul (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x271492A: ga_dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5CF40F: diis_bld12_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5B3D17: dft_main0d_ (dft_main0d.F:549)
==20005==    by 0x5A9734: nwdft_ (nwdft.F:274)
==20005==    by 0x5A99B5: dft_energy_ (nwdft.F:18)
==20005==    by 0x418EE7: task_energy_doit_ (task_energy.F:251)
==20005==    by 0x41A57B: task_energy_ (task_energy.F:95)
==20005==    by 0x40DAD2: task_ (task.F:337)
==20005==    by 0x4068F5: MAIN__ (nwchem.F:251)
==20005==  Address 0x19975040 is 0 bytes after a block of size 42,008,576 alloc'd
==20005==    at 0x1155679D: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==20005==    by 0x291F69B: morecore (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x291F793: kr_malloc (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A492B: pnga_matmul (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x271492A: ga_dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5CF40F: diis_bld12_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5B3D17: dft_main0d_ (dft_main0d.F:549)
==20005==    by 0x5A9734: nwdft_ (nwdft.F:274)
==20005==    by 0x5A99B5: dft_energy_ (nwdft.F:18)
==20005==    by 0x418EE7: task_energy_doit_ (task_energy.F:251)
==20005==    by 0x41A57B: task_energy_ (task_energy.F:95)
==20005==    by 0x40DAD2: task_ (task.F:337)
==20005==
==20005== Invalid write of size 8
==20005==    at 0x29B604D: dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A049F: GAI_DGEMM (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A3954: pnga_matmul (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x271492A: ga_dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5CF40F: diis_bld12_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5B3D17: dft_main0d_ (dft_main0d.F:549)
==20005==    by 0x5A9734: nwdft_ (nwdft.F:274)
==20005==    by 0x5A99B5: dft_energy_ (nwdft.F:18)
==20005==    by 0x418EE7: task_energy_doit_ (task_energy.F:251)
==20005==    by 0x41A57B: task_energy_ (task_energy.F:95)
==20005==    by 0x40DAD2: task_ (task.F:337)
==20005==    by 0x4068F5: MAIN__ (nwchem.F:251)
==20005==  Address 0x19975050 is not stack'd, malloc'd or (recently) free'd
==20005==
==20005== Invalid write of size 8
==20005==    at 0x29B6052: dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A049F: GAI_DGEMM (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A3954: pnga_matmul (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x271492A: ga_dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5CF40F: diis_bld12_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5B3D17: dft_main0d_ (dft_main0d.F:549)
==20005==    by 0x5A9734: nwdft_ (nwdft.F:274)
==20005==    by 0x5A99B5: dft_energy_ (nwdft.F:18)
==20005==    by 0x418EE7: task_energy_doit_ (task_energy.F:251)
==20005==    by 0x41A57B: task_energy_ (task_energy.F:95)
==20005==    by 0x40DAD2: task_ (task.F:337)
==20005==    by 0x4068F5: MAIN__ (nwchem.F:251)
==20005==  Address 0x19975060 is not stack'd, malloc'd or (recently) free'd
==20005==
==20005== Invalid write of size 8
==20005==    at 0x29B6057: dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A049F: GAI_DGEMM (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A3954: pnga_matmul (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x271492A: ga_dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5CF40F: diis_bld12_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5B3D17: dft_main0d_ (dft_main0d.F:549)
==20005==    by 0x5A9734: nwdft_ (nwdft.F:274)
==20005==    by 0x5A99B5: dft_energy_ (nwdft.F:18)
==20005==    by 0x418EE7: task_energy_doit_ (task_energy.F:251)
==20005==    by 0x41A57B: task_energy_ (task_energy.F:95)
==20005==    by 0x40DAD2: task_ (task.F:337)
==20005==    by 0x4068F5: MAIN__ (nwchem.F:251)
==20005==  Address 0x19975070 is not stack'd, malloc'd or (recently) free'd
==20005==
==20005== Invalid write of size 8
==20005==    at 0x29B6035: dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A049F: GAI_DGEMM (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A3954: pnga_matmul (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x271492A: ga_dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5CF40F: diis_bld12_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5B3D17: dft_main0d_ (dft_main0d.F:549)
==20005==    by 0x5A9734: nwdft_ (nwdft.F:274)
==20005==    by 0x5A99B5: dft_energy_ (nwdft.F:18)
==20005==    by 0x418EE7: task_energy_doit_ (task_energy.F:251)
==20005==    by 0x41A57B: task_energy_ (task_energy.F:95)
==20005==    by 0x40DAD2: task_ (task.F:337)
==20005==    by 0x4068F5: MAIN__ (nwchem.F:251)
==20005==  Address 0x19975080 is not stack'd, malloc'd or (recently) free'd
==20005==
==20005== Invalid write of size 8
==20005==    at 0x29B6039: dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A049F: GAI_DGEMM (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A3954: pnga_matmul (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x271492A: ga_dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5CF40F: diis_bld12_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5B3D17: dft_main0d_ (dft_main0d.F:549)
==20005==    by 0x5A9734: nwdft_ (nwdft.F:274)
==20005==    by 0x5A99B5: dft_energy_ (nwdft.F:18)
==20005==    by 0x418EE7: task_energy_doit_ (task_energy.F:251)
==20005==    by 0x41A57B: task_energy_ (task_energy.F:95)
==20005==    by 0x40DAD2: task_ (task.F:337)
==20005==    by 0x4068F5: MAIN__ (nwchem.F:251)
==20005==  Address 0x19975090 is not stack'd, malloc'd or (recently) free'd
==20005==
==20005== Invalid write of size 8
==20005==    at 0x29B603E: dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A049F: GAI_DGEMM (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A3954: pnga_matmul (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x271492A: ga_dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5CF40F: diis_bld12_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5B3D17: dft_main0d_ (dft_main0d.F:549)
==20005==    by 0x5A9734: nwdft_ (nwdft.F:274)
==20005==    by 0x5A99B5: dft_energy_ (nwdft.F:18)
==20005==    by 0x418EE7: task_energy_doit_ (task_energy.F:251)
==20005==    by 0x41A57B: task_energy_ (task_energy.F:95)
==20005==    by 0x40DAD2: task_ (task.F:337)
==20005==    by 0x4068F5: MAIN__ (nwchem.F:251)
==20005==  Address 0x199750a0 is not stack'd, malloc'd or (recently) free'd
==20005==
==20005== Invalid write of size 8
==20005==    at 0x29B6043: dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A049F: GAI_DGEMM (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x27A3954: pnga_matmul (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x271492A: ga_dgemm_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5CF40F: diis_bld12_ (in /home/me/nwchem-6.1/bin/LINUX64/nwchem)
==20005==    by 0x5B3D17: dft_main0d_ (dft_main0d.F:549)
==20005==    by 0x5A9734: nwdft_ (nwdft.F:274)
==20005==    by 0x5A99B5: dft_energy_ (nwdft.F:18)
==20005==    by 0x418EE7: task_energy_doit_ (task_energy.F:251)
==20005==    by 0x41A57B: task_energy_ (task_energy.F:95)
==20005==    by 0x40DAD2: task_ (task.F:337)
==20005==    by 0x4068F5: MAIN__ (nwchem.F:251)
==20005==  Address 0x199750b0 is not stack'd, malloc'd or (recently) free'd
==20005==
==20005==
==20005== HEAP SUMMARY:
==20005==     in use at exit: 4,303,284,355 bytes in 697 blocks
==20005==   total heap usage: 2,135 allocs, 1,438 frees, 4,305,900,787 bytes allocated
==20005==
==20005== 536 (24 direct, 512 indirect) bytes in 1 blocks are definitely lost in loss record 662 of 679
==20005==    at 0x1155679D: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==20005==    by 0x11D6E128: ??? (in /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==20005==    by 0x11E332F8: ??? (in /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==20005==    by 0x11E2C573: ??? (in /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==20005==    by 0x11D6BB47: ??? (in /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==20005==    by 0x1093CCCF: call_init (dl-init.c:85)
==20005==    by 0x1093CDC6: _dl_init (dl-init.c:134)
==20005==    by 0x1092FB29: ??? (in /lib/x86_64-linux-gnu/ld-2.13.so)
==20005==    by 0x1: ???
==20005==    by 0x7FF00033A: ???
==20005==    by 0x7FF000341: ???
==20005==
==20005== LEAK SUMMARY:
==20005==    definitely lost: 24 bytes in 1 blocks
==20005==    indirectly lost: 512 bytes in 1 blocks
==20005==      possibly lost: 0 bytes in 0 blocks
==20005==    still reachable: 4,303,283,819 bytes in 695 blocks
==20005==         suppressed: 0 bytes in 0 blocks
==20005== Reachable blocks (those to which a pointer was found) are not shown.
==20005== To see them, rerun with: --leak-check=full --show-reachable=yes
==20005==
==20005== For counts of detected and suppressed errors, rerun with: -v
==20005== ERROR SUMMARY: 663 errors from 10 contexts (suppressed: 4 from 4)



For comparison, here's using nwchem 6.0:
NOTE that this version works just fine and runs to completion without error messages normally.
valgrind --leak-check=full --track-origins=yes --log-file=valgrind.log nwchem nwchem.nw



==21014== Memcheck, a memory error detector
==21014== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
==21014== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info
==21014== Command: nwchem nwchem.nw
==21014== Parent PID: 20854
==21014==
==21014== Warning: set address range perms: large range [0x3952b040, 0x13352b110) (undefined)
==21014== Syscall param write(buf) points to uninitialised byte(s)
==21014==    at 0x11673980: __write_nocancel (syscall-template.S:82)
==21014==    by 0x11618B92: _IO_file_write@@GLIBC_2.2.5 (fileops.c:1276)
==21014==    by 0x11618809: new_do_write (fileops.c:530)
==21014==    by 0x11618B34: _IO_do_write@@GLIBC_2.2.5 (fileops.c:503)
==21014==    by 0x11619347: _IO_file_sync@@GLIBC_2.2.5 (fileops.c:905)
==21014==    by 0x1160DE19: fflush (iofflush.c:43)
==21014==    by 0x8A16D7: hdbm_file_flush (in /home/me/nwchem-6.0/bin/LINUX64/nwchem)
==21014==    by 0x83C973: rtdb_seq_put (in /home/me/nwchem-6.0/bin/LINUX64/nwchem)
==21014==    by 0x83ADA6: rtdb_put (in /home/me/nwchem-6.0/bin/LINUX64/nwchem)
==21014==    by 0x83A13F: rtdb_put_ (in /home/me/nwchem-6.0/bin/LINUX64/nwchem)
==21014==    by 0x85C72C: util_set_rtdb_state_ (util_rtdb_state.F:40)
==21014==    by 0x40636B: MAIN__ (nwchem.F:223)
==21014==  Address 0xf7c0022 is not stack'd, malloc'd or (recently) free'd
==21014==  Uninitialised value was created by a stack allocation
==21014==    at 0x83B7D0: rtdb_seq_put_info (in /home/me/nwchem-6.0/bin/LINUX64/nwchem)
==21014==
==21014== Conditional jump or move depends on uninitialised value(s)
==21014==    at 0x8A5E1F: sym_op_class_name_ (sym_op_cname.F:26)
==21014==    by 0x83FB41: sym_op_classify_ (sym_op_clsfy.F:49)
==21014==    by 0x845AE9: sym_movecs_adapt_ (sym_mo_adapt.F:77)
==21014==    by 0x719A4A: scf_movecs_sym_adapt_ (scf_sym_adap.F:70)
==21014==    by 0x731028: scf_vectors_guess_ (scf_vec_guess.F:403)
==21014==    by 0x58D4E6: dft_scf_ (dft_scf.F:526)
==21014==    by 0x58B67B: dft_main0d_ (dft_main0d.F:537)
==21014==    by 0x5818B3: nwdft_ (nwdft.F:309)
==21014==    by 0x581B24: dft_energy_ (nwdft.F:18)
==21014==    by 0x4174D7: task_energy_doit_ (task_energy.F:229)
==21014==    by 0x418AEB: task_energy_ (task_energy.F:74)
==21014==    by 0x40C646: task_ (task.F:301)
==21014==  Uninitialised value was created by a stack allocation
==21014==    at 0x83F9BD: sym_op_classify_ (sym_op_clsfy.F:32)
==21014==
==21014==
==21014== HEAP SUMMARY:
==21014==     in use at exit: 4,254,665,672 bytes in 20 blocks
==21014==   total heap usage: 10,491 allocs, 10,471 frees, 4,275,242,731 bytes allocated
==21014==
==21014== 17 bytes in 2 blocks are definitely lost in loss record 9 of 19
==21014==    at 0x103C679D: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21014==    by 0x11626881: strdup (strdup.c:43)
==21014==    by 0x24D33F7: pbeginf_ (in /home/me/nwchem-6.0/bin/LINUX64/nwchem)
==21014==    by 0x405F53: MAIN__ (nwchem.F:66)
==21014==    by 0x406964: main (nwchem.F:336)
==21014==
==21014== 536 (24 direct, 512 indirect) bytes in 1 blocks are definitely lost in loss record 13 of 19
==21014==    at 0x103C679D: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21014==    by 0x10BDE128: ??? (in /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==21014==    by 0x10CA32F8: ??? (in /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==21014==    by 0x10C9C573: ??? (in /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==21014==    by 0x10BDBB47: ??? (in /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0)
==21014==    by 0xF7ACCCF: call_init (dl-init.c:85)
==21014==    by 0xF7ACDC6: _dl_init (dl-init.c:134)
==21014==    by 0xF79FB29: ??? (in /lib/x86_64-linux-gnu/ld-2.13.so)
==21014==    by 0x1: ???
==21014==    by 0x7FF00032A: ???
==21014==    by 0x7FF000331: ???
==21014==
==21014== LEAK SUMMARY:
==21014==    definitely lost: 41 bytes in 3 blocks
==21014==    indirectly lost: 512 bytes in 1 blocks
==21014==      possibly lost: 0 bytes in 0 blocks
==21014==    still reachable: 4,254,665,119 bytes in 16 blocks
==21014==         suppressed: 0 bytes in 0 blocks
==21014== Reachable blocks (those to which a pointer was found) are not shown.
==21014== To see them, rerun with: --leak-check=full --show-reachable=yes
==21014==
==21014== For counts of detected and suppressed errors, rerun with: -v
==21014== ERROR SUMMARY: 178 errors from 4 contexts (suppressed: 4 from 4)

74. Building nwchem 6.1 on debian testing 32 bit only

EDIT 18 May 2012: 
It's now been solved on 64 bit as well
Compiling nwchem 6.1 with internal libs on debian: http://verahill.blogspot.com.au/2012/05/compiling-nwchem-61-with-internal-libs.html
Compiling nwchem 6.1 with openblas on debian: http://verahill.blogspot.com.au/2012/05/building-nwchem-61-on-debian.html


This doesn't work with the 64 bit version of nwchem 6.1. There's a separate post on that. Nwchem 6.1 64 bit will build just fine, but will crash when run. Again, see the other post.

Building on 32 bit debian testing:



Put a hold on your mpich2 and mpich2-dev packages (see e.g. here for more details)
1. edit your /etc/apt/sources.list to allow packages from stable e.g.

deb ftp://ftp.au.debian.org/debian/ testing main contrib non-fre
deb ftp://ftp.au.debian.org/debian/ stable main contrib non-free

2. create an /etc/apt/preferences file e.g.

Package: *
Pin: release a=testing
Pin-Priority: 990
Package: *
Pin: release a=stable
Pin-Priority: -10

2. install v 1.2 explicitly

sudo apt-get update && sudo apt-get install mpich2=1.2.1.1-5 libmpich2-dev=1.2.1.1-5

3. put a hold on the packages

sudo su
echo "mpich2 hold"|dpkg --set-selections
echo "libmpich2-dev hold"|dpkg --set-selections

exit

Make sure you have the necessary packages:
sudo apt-get install build-essential gfortran fort77

I had some error messages before installing fort77. Not sure they are related.

Download the nwchem source
cd ~
wget http://www.nwchem-sw.org/images/Nwchem-6.1-2012-Feb-10.tar.gz
tar -xvf Nwchem-6.1-2012-Feb.tar.gz
cd nwchem-6.1

create buildconf.sh in ~/nwchem-6.1


export LARGE_FILES=TRUE
export TCGRSH=/usr/local/bin/ssh
export NWCHEM_TOP=/home/me/nwchem-6.1
export NWCHEM_TARGET=LINUX
export NWCHEM_MODULES=all
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/usr
export MPI_LIB=$MPI_LOC/lib
export MPI_INCLUDE=$MPI_LOC/include/mpich2
export LIBMPI="-lmpich -lfmpich -lpthread"
export NWCHEM_MODULES="all"
cd $NWCHEM_TOP/src
make clean
make nwchem_config
make FC=gfortran

run
sh buildconf.sh


Building takes ages. But it works. Why it works for 32 bit and not 64 bit has me a bit confused, but it's probably a good hint to the solution.

73. Gnu screen- attach to a running screen session

Follows this with minor amendments.

The issue:
You start e.g. a compile or a number-crunching job on your work computer, and go home while leaving it running. When you get home you log on to your work computer via ssh, and want to connect to the terminal in which the job is running i.e. connect to the tty.

Another use for screen is if you are working over an unstable connection -- if you get booted off the screen session continues to run, so whatever was running in it is still sound. A normal pts will die and crash.

A solution using screen:
Be aware that this can cause a potential security risk.

1. install gnu screen
sudo apt-get install screen

2. enable multi-user mode
sudo chmod +s /usr/bin/screen
sudo chmod 755 /var/run/screen

Edit /etc/screenrc and add the following lines anywhere

multiuser on
acladd me

If you want primitive scrolling (shift + page up/down)

termcapinfo xterm|xterms|xs|rxvt ti@:te@

Replace me with the local username.

3. On your work computer,
screen -S testing

4. Start whatever job you were thinking of and leave it running

5. From home ssh into your work computer then
screen -x me/testing

Another neat thing is that both terminals look the same no matter whether you are at your work computer or at your home computer. Ergo, it may be a useful way of helping people remotely with their computers too.

6. To disconnect gracefully
Hold CTRL+a+d

Do not type exit or anything like that, as it will close the screen instance completely.

For a gnu screen cheat sheet, look here:
http://www.totiso.com/2009/05/03/gnu-screen-cheat-sheet

72. Building maxima from source on Debian Testing

The version of maxima in the debian repos is the same as in the sourceforge archives. There's only one good reason to compile your own version from what I can see -- to get around the dependency of maxima on gnuplot.

Let me explain: the gnuplot version in the debian repos has problems handling small numbers (<10**(-9)). Uninstalling gnuplot and replacing it with your a version you've compiled yourself removes maxima.

Luckily it's easy to build:

wget http://sourceforge.net/projects/maxima/files/latest/download?source=files

mv download\?source\=files maxima-5.26.0.tar.gz

./configure

make -j5

where 5 is the number of cores +1 (I have four cores). Change as needed.

sudo checkinstall

Checkinstall may or may not exit successfully if you have compiled your own version of gnuplot -- see below. Chances are that it built the package correctly.

or 

sudo make install

Error:

If you get

dpkg: error processing maxima_5.26.0-1_amd64.deb (--install):
trying to overwrite '/usr/local/share/info/dir', which is also in package gnuplot 4.4.4-1
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)

Solution: 

sudo dpkg --force-overwrite -i maxima_5.26.0-1_amd64.deb

You can see that the dir file is ok (GNUPlot is still referred to):

This is the file .../info/dir, which contains the
topmost node of the Info hierarchy, called (dir)Top.
The first time you invoke Info you start off looking at this node.

File: dir, Node: Top This is the top of the INFO tree
  This (the Directory node) gives a menu of major topics.
  Typing "q" exits, "?" lists all Info commands, "d" returns here,
  "h" gives a primer for first-timers,
  "mEmacs<Return>" visits the Emacs manual, etc.
  In Emacs, you can click mouse button 2 on a menu item or cross reference
  to select it.
* Menu:
Emacs
* Imaxima: (imaxima).           Image support for the computer algebra system
                                  Maxima. Interactive math minor mode.
Math
* GNUPLOT: (gnuplot).           An Interactive Plotting Program
* Maxima: (maxima).             A computer algebra system.
* Xmaxima: (xmaxima).           Graphical interface for Maxima.

71. Building Thunderbird 10.0.2 on debian testing

I use evolution for email, contacts and calendar because it integrates well with gnome and because it looks a whole lot prettier than the version of Thunderbird (i.e. Icedove) in the debian repos (3.1.16-1).

Well, sometimes you've got to check out the alternatives. Here's how to build thunderbird 10.0.2 from source.

-- START HERE --

sudo apt-get install libdbus-glib-1-dev gir1.2-notify-0.7 libnotify-dev  yasm checkinstall libzip-dev zip

cd ~/tmp
wget ftp://ftp.mozilla.org/pub/mozilla.org/thunderbird/releases/10.0.2/source/thunderbird-10.0.2.source.tar.bz2

tar -xvf thunderbird-10.0.2.source.tar.bz2 

cd comm-release/

./configure --disable-necko-wifi

..
updating cache ../../.././config.cache
creating ./config.status
creating Makefile
creating config/Makefile
creating config/autoconf.mk
creating ldap/Makefile
creating ldap/clients/tools/Makefile
creating ldap/include/Makefile
creating ldap/libraries/Makefile
creating ldap/libraries/libldap/Makefile
creating ldap/libraries/libprldap/Makefile
creating ldap/libraries/libldif/Makefile
creating ldap/libraries/liblber/Makefile
creating ldap/libraries/libiutil/Makefile
creating ldap/libraries/libssldap/Makefile
creating ldap/libraries/libutil/Makefile

make -jN

...
make[4]: Leaving directory `/home/me/tmp/comm-release/mail/test/mozmill'
make[3]: Leaving directory `/home/me/tmp/comm-release/mail'
make[2]: Leaving directory `/home/me/tmp/comm-release'
make[1]: Leaving directory `/home/me/tmp/comm-release'
if test -d ./mozilla/dist/bin ; then touch ./mozilla/dist/bin/.purgecaches ; fi

where N is the number of cores +1 -- in my case it's 7 since I have a six-core CPU. Be aware that building does take a while.

sudo make install

(sudo checkinstall ended with segfault for some reason)

You are now done.

Make sure that /usr/local/bin is in your PATH

me@beryllium:~/tmp/comm-release$ which thunderbird
/usr/local/bin/thunderbird

Interesting observation:
while thunderbird starts thunderbird the home-built version seems to be referred to as earlybird:

What's ugly or not is subjective, but you may want to use this add-on:
https://addons.mozilla.org/en-US/thunderbird/addon/gnome-linux-integration/?src=search

---

Troubleshooting:
Error:
checking MOZ_PANGO_CFLAGS... -pthread -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/freetype2
checking MOZ_PANGO_LIBS... -pthread -lpangoft2-1.0 -lpango-1.0 -lfreetype -lfontconfig -lgobject-2.0 -lgmodule-2.0 -lgthread-2.0 -lrt -lglib-2.0
checking for gnome-vfs-2.0 >= 2.0 gnome-vfs-module-2.0 >= 2.0... checking for gconf-2.0 >= 1.2.1 gobject-2.0 ... checking for dbus-glib-1 >= 0.60... Package dbus-glib-1 was not found in the pkg-config search path. Perhaps you should add the directory containing `dbus-glib-1.pc' to the PKG_CONFIG_PATH environment variable No package 'dbus-glib-1' found
configure: error: Library requirements (dbus-glib-1 >= 0.60) not met; consider adjusting the PKG_CONFIG_PATH environment variable if your libraries are in a nonstandard prefix so pkg-config can find them.

Solution:
sudo apt-get install libdbus-glib-1-dev

Error:

checking MOZ_PANGO_LIBS... -pthread -lpangoft2-1.0 -lfreetype -lfontconfig -lpangocairo-1.0 -lpango-1.0 -lcairo -lgobject-2.0 -lgmodule-2.0 -lgthread-2.0 -lrt -lglib-2.0
checking for gnome-vfs-2.0 >= 2.0 gnome-vfs-module-2.0 >= 2.0... checking for gconf-2.0 >= 1.2.1 gobject-2.0 ... checking for libnotify >= 0.4... Package libnotify was not found in the pkg-config search path. Perhaps you should add the directory containing `libnotify.pc' to the PKG_CONFIG_PATH environment variable No package 'libnotify' found
configure: error: Library requirements (libnotify >= 0.4) not met; consider adjusting the PKG_CONFIG_PATH environment variable if your libraries are in a nonstandard prefix so pkg-config can find them

Solution:
sudo apt-get install gir1.2-notify-0.7 libnotify-dev

Error:

checking MOZ_DBUS_GLIB_LIBS... -pthread -ldbus-glib-1 -ldbus-1 -lpthread -lgobject-2.0 -lgthread-2.0 -lrt -lglib-2.0
checking __attribute__ ((aligned ())) support... trying 64
64
configure: error: yasm is a required build tool for this architecture when webm is enabled. You may either install yasm or --disable-webm (which disables the WebM video format). See https://developer.mozilla.org/en/YASM for more details.
configure: error: ./configure failed for mozilla

Solution:
sudo apt-get install yasm

70. Installing Debian on a USB stick -- live usb vs a true and full installation

Update 17 March 2013: See this post for a faster, better way of creating a full install if you're already running Debian.

Original post:
Every now and again I get posts like this one, or this one, via google news. While those posts --describing the use of unetbootin to create a 'live' USB analogue to a live CD -- may be technically correct, there's just so much more you can do.

In 1  we do the old boring bog-standard 'write live cd to a usb' thingy that you see pasted all over the web, but using cat instead of unetbootin.
In 2  we make a real, bootable installation on a usb drive.

---

1.  Creating a USB version of a live CD -- the boring option
With the current 'hybrid' Debian iso's it couldn't be easier.
THIS WILL WIPE YOUR USB STICK

 --- START HERE ---

a. Download the iso
Using jigdo is a good option. Downloading a business-card/netinstall iso is another, if you'll have a working internet connection available.

b. Plug in and mount your USB device and  find out the device name of your USB drive.
If it's mounted you can use
df -h

rootfs                                                   93G   36G   54G  40% /
udev                                                    3.9G     0  3.9G   0% /dev
tmpfs                                                   801M  980K  800M   1% /run
/dev/disk/by-uuid/..-10a350f85687   93G   36G   54G  40% /
tmpfs                                                   5.0M     0  5.0M   0% /run/lock
tmpfs                                                   1.6G   52K  1.6G   1% /tmp
tmpfs                                                   1.6G  816K  1.6G   1% /run/shm
/dev/sda6                                               745G  183G  525G  26% /home
/dev/sdc1                                               2.0G  434M  1.5G  23% /media/XP-KOMKU

or

mount

../dev/sdc1 on /media/XP-KOMKU type vfat (rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0077,codepage=cp437,iocharset=utf8,shortname=mixed,showexec,utf8,flush,errors=remount-ro,uhelper=udisks)

to list mounted devices. In my case it's an old 2 G usb stick I used to create a Windows XP installation USB device on.

If you prefer a gui tool, start palimpsest (called Disk Utility in gnome)
It will be something akin to sdb1 or sdc1 etc. That means the device name is /dev/sdb or /dev/sdc, respectively. In our case, it's sdc.

c. Unmount but don't detach the device
You don't want anything else writing to it.

me@beryllium:~/Documents/screengrabs$ umount /dev/sdc1
me@beryllium:~/Documents/screengrabs$ df -h
Filesystem                                              Size  Used Avail Use% Mounted on
rootfs                                                   93G   36G   54G  40% /
udev                                                    3.9G     0  3.9G   0% /dev
tmpfs                                                   801M  976K  800M   1% /run
/dev/disk/by-uuid/..-10a350f85687   93G   36G   54G  40% /
tmpfs                                                   5.0M     0  5.0M   0% /run/lock
tmpfs                                                   1.6G   60K  1.6G   1% /tmp
tmpfs                                                   1.6G  820K  1.6G   1% /run/shm
/dev/sda6                                               745G  183G  525G  26% /home

d. Become root and use cat to write the iso to the USB device

sudo su

And write (assuming that the device name is sdc and you are using the businesscard iso)

cat debian-6.0.3-amd64-businesscard.iso > /dev/sdc

Done.

e. boot from your USB drive and go through the same steps as for a CD.  
Plug it in, then start your computer. Hit F12 (or F10 or del or F2) during the bios start-up to select boot medium. Choose to boot from usb. Older BIOSes can't boot from USB.

---

2. Really INSTALLING Debian on a USB
You can use either virtualbox or quemu for this. Here I show how to use virtualbox.

[NOTE: there's a leaner, faster, better way described here: http://verahill.blogspot.com/2013/03/361-installing-debian-on-usb-stick-from.html ]

•  Note that a tiny 2 Gb USB stick isn't suitable for a full Debian + Gnome installation. It may, however, work well with e.g. openbox and xmonad. Or just work in the terminal -- use mutt for email, mcabber for chat, elinks for http/www etc.
• Here I show how to install using the businesscard iso, but I've done this with the regular Debian DVD version in the past.
• Also, few USB devices operate at 480 Mbps -- so be prepared that installation via virtualbox onto a USB device will take considerably longer than a regular installation.
• Solid state devices degrade with I/O, so use your USB stick for back-up use during e.g. travel etc. Don't trust the integrity of it. 

There are a few good reasons for installing a full linux version on a thumb drive or a regular but external harddrive:
*  a thumb drive will allow you too carry an independent OS with you, allowing you to e.g. temporarily borrow a computer from a friend but running your own system. An installation, as opposed to a 'live cd', allows you to customise the setup and install programs. USB 2 is quite slow, and many thumb drives are slower than that, so you're probably better off running a light system than a full on gnome-shell desktop. That's not so say that you can't -- I have.
* installing onto a usb-connected harddrive will allow you to setup and prepare headless boxes or systems without a direct internet connection.

First you need to install virtualbox, which is available in the debian repos (sudo apt-get install virtualbox virtualbox-guest-additions), and -- in order to enable USB 2 support --  the "VirtualBox 4.1.8 Oracle VM VirtualBox Extension Pack" from https://www.virtualbox.org/wiki/Downloads
Start virtualbox and go to  File/Preferences/Extensions to install the extension pack you downloaded.

 --- START HERE ---

Installing debian (or any os) on an external USB stick/drive using virtualbox
a. Start virtualbox and click on New.

b. Click through the Name, Operating System and Version questions, Select a memory size which is reasonable for the intended use and target machine. Give it at least 256 Mb. Don't fret about this step -- on each boot your system will autodetect the amount of available RAM. The next question is the key to the whole undertaking: DO NOT SET A START-UP DISK (or any virtual harddisk)

In the next dialogue box you'll be asked if you want to continue or go back - yes, you want to continue.

c. Your new machine is now ready to be set up. Select it and click on settings; Got to storage, IDE controller and choose a virtual CD/DVD disk file. Select the debian installation iso.

d. Next click on USB, enable USB 2 controller, and add a filter for your device by e.g. hitting alt+insert and selecting it from the list.

e. You're ready to install! Start the virtual machine, and the installation should start without you having to select boot device. Most of the setting are the same as in any normal install. Pay attention when you come to the Partition Disk dialogue though. Make sure that you're not doing something silly and accidentally deleting something you shouldn't be a-deleting, even though the risks of that are very slim.

Delete the existing partitions and create a new one. I added a tiny swap space too, hoping that this will make the difference between a freeze and a slow crawl on underpowered systems.

f. Continue with the installation as normal.

g. Postscript:
* /dev/sda1 vs UUID. Once you're done with the installation  and have shut down your virtualbox, mount the USB drive as a normal thumb drive and have a look at fstab in particular -- if you're using relative paths (e.g. sda) instead of absolute paths using UUID, you may run into problems at some point.

In the screengrab you see that

# / was on /dev/sda1 during installation

UUID=8c0c9c98-e73f-4f5a-b7d7-f91f95c9d8ca /               ext4    errors=remount-ro 0       1

Which means we're good.

You may want to comment out

/dev/sda1       /media/usb0     auto    rw,user,noauto  0       0
/dev/sda2       /media/usb1     auto    rw,user,noauto  0       0
/dev/sda5       /media/usb2     auto    rw,user,noauto  0       0

But I haven't in the past and all has been good.

* network interfaces -- you may find that no network interface is configured for you and that no interface shows up when you use ifconfig. Use ip addr to get a list over interfaces. If the interface is e.g. eth2, you may  want to edit /etc/network/interfaces and add

auto eth2
iface eth2 inet dhcp

or (the ip's are just examples)

auto eth2
iface eth2 inet static
address 192.168.1.100
gateway 192.168.1.1
netmask 255.255.255.0

And then run sudo service networking restart

That's it. You now have a full installation on either a 'removable' drive or on an HDD which you can install in a desktop/laptop.

---

Links to this page:
http://vk.com/topic-50603_27808581
http://forum.debian.org.tr/arsiv-konu-391.0-debian-live-usb.html
http://alsamixer.wordpress.com/2012/10/07/installing-debian/
http://cv.uoc.edu/app/phpBB3/viewtopic.php?f=3511&t=20897
http://www.computerbase.de/forum/showthread.php?t=1208228&p=13936928

70. Bug in Debian version of Gnuplot 4.4.0

The symptom:

Gnuplot 4.4.0-1.1 (the current Debian version) can't handle numbers smaller than 10**(-9) properly
e.g.

me@beryllium:~/Dropbox$ gnuplot
G N U P L O T
Version 4.4 patchlevel 0
last modified March 2010
System: Linux 3.2.0-1-amd64
Copyright (C) 1986-1993, 1998, 2004, 2007-2010
Thomas Williams, Colin Kelley and many others
gnuplot home:     http://www.gnuplot.info
faq, bugs, etc:   type "help seeking-assistance"
immediate help:   type "help"
plot window:      hit 'h'
Terminal type set to 'wxt'
gnuplot> print 10**(-9)
1e-09
gnuplot> print 10**(-10)
7.09186959928599e-10
gnuplot> print 10**(-11)
8.22536045240377e-10

It's not related to set zero.

gnuplot> set zero 1e-20
gnuplot> print 10**(-11)
8.22536045240377e-10
gnuplot> print 10**(-10)
7.09186959928599e-10
gnuplot> print 10**(-9)
1e-09
gnuplot> 

Currently, I'm using version 4.4.0-1.1 -- which is used in all versions of Debian.

me@beryllium:~$ apt-cache showpkg gnuplot
Package: gnuplot
Versions:
4.4.0-1.1 (/var/lib/apt/lists/ftp.au.debian.org_debian_dists_testing_main_binary-amd64_Packages) (/var/lib/apt/lists/ftp.au.debian.org_debian_dists_stable_main_binary-amd64_Packages) (/var/lib/apt/lists/ftp.au.debian.org_debian_dists_unstable_main_binary-amd64_Packages) (/var/lib/dpkg/status)
..
Dependencies:
4.4.0-1.1 - gnuplot-nox (2 4.4.0-1.1) gnuplot-x11 (2 4.4.0-1.1) gnuplot-doc (2 4.4.0-1.1)
Provides:
4.4.0-1.1 -
Reverse Provides: 

The bug affects the output from the print statement as well as the internal handling of numbers:

gnuplot> plot 10**(-11)
Warning: empty y range [8.22536e-10:8.22536e-10], adjusting to [8.14311e-10:8.30761e-10]

gnuplot> plot 10**(-12)
Warning: empty y range [-1.3748e-09:-1.3748e-09], adjusting to [-1.36105e-09:-1.38855e-09]

gnuplot> plot 10**(-12)/10**(-10)
Warning: empty y range [-1.93855:-1.93855], adjusting to [-1.91917:-1.95794]

gnuplot> set xrange [10**(-9):10**(-12)]
gnuplot> plot x 

The bug is similar to this: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=360087

Integer overflows are not reported. A hint could be printed that real
(float) numbers should (could) be used to avaid this problem.
gnuplot> print 1000000*100000
1215752192
gnuplot> print 1000000**2
-727379968
gnuplot> print 100000**2
1410065408
gnuplot> a=2000000**2
gnuplot> print a
1385447424
gnuplot> print 10000**2 # OK
100000000

which ended up with "Added tag(s) wontfix. "
Ergo, if you're using debian and you are using gnuplot for serious purposes (research, work), compile your own version of gnuplot as per below.


ANNOYING: there are packages such as maxima which depend on gnuplot. Remove the debian version of gnuplot using apt-get and you lose maxima too. Octave, which one would think would be a heavier user of gnuplot, does not depend on gnuplot but merely recommends it.

Package: maxima                        
State: not installed
Version: 5.26.0-3
Priority: optional
Section: math
Maintainer: Camm Maguire <camm@debian.org>
Uncompressed Size: 47.8 M
Depends: libc6 (>= 2.3), libgmp10, libreadline6 (>= 6.0), libx11-6, gnuplot-x11

Maybe time to build your own maxima.deb? In the end you will end up with a very inelegant system with mixed packages.

---

Upstreams:
I've compiled and checked the current upstreams version:

me@beryllium:~$ sudo apt-get autoremove gnuplot gnuplot-nox
cd ~/temp
wget http://downloads.sourceforge.net/project/gnuplot/gnuplot/4.4.4/gnuplot-4.4.4.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fgnuplot%2Ffiles%2F&ts=1329439455&use_mirror=aarnet
 mv gnuplot-4.4.4.tar.gz\?r\=http\:%2F%2Fsourceforge.net%2Fprojects%2Fgnuplot%2Ffiles%2F gnuplot-4.4.4.tar.gz

tar -xvf gnuplot-4.4.4.tar.gz
./configure
make 
sudo checkinstall

me@beryllium:~/temp/gnuplot-4.4.4$ sudo dpkg -i gnuplot_4.4.4-1_amd64.deb
(Reading database ... 446323 files and directories currently installed.)
Preparing to replace gnuplot 4.4.4-1 (using gnuplot_4.4.4-1_amd64.deb) ...
Unpacking replacement gnuplot ...
Setting up gnuplot (4.4.4-1) ...
Processing triggers for man-db ...
me@beryllium:~/temp/gnuplot-4.4.4$ gnuplot
G N U P L O T
Version 4.4 patchlevel 4
last modified November 2011
System: Linux 3.2.0-1-amd64
Copyright (C) 1986-1993, 1998, 2004, 2007-2011
Thomas Williams, Colin Kelley and many others
gnuplot home:     http://www.gnuplot.info
faq, bugs, etc:   type "help seeking-assistance"
immediate help:   type "help"
plot window:      hit 'h'
Terminal type set to 'x11'
gnuplot> print 10**(-9)
1e-09
gnuplot> print 10**(-10)
1e-10
gnuplot> print 10**(-11)
1e-11
gnuplot> print 3.14*10**(-10)
3.14e-10
gnuplot> print 3.14*10**(-20)
3.14e-20
gnuplot> print 3.14*10**(-21)
3.14e-21
gnuplot> print 3*10**(-12)/(4*10**(-14))
75.0
gnuplot> 

Ergo, upstreams v 4.4.4 works.

Update: Here's my bug report:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=660299

69. Reverse VNC using vncviewer and tightvnc

Reverse VNC is a good way of helping people remotely. It requires that your IP is remotely accessible, but it does NOT require that the client's IP is public.

This is based on http://ubuntuforums.org/showthread.php?t=299489 and https://caedesnotes.wordpress.com/2010/01/08/remote-administrationtech-support-with-reverse-vnc/ but with screenshots. If both of you are using linux but you for some reason prefer vnc over ssh, have a look at that link for how to do (i.e. using x11vnc on the client)

Like a lot of people I have parents. Like most of those people, my parents aren't too interested in augmenting their computer skills. Which is fine. But trying to explain over the phone where to click etc. gets old really fast -- VNC is an advantage here if they are running windows. Parents also tend to appreciate lots of screenshots -- so I've provided that. It's running on a French version of XP -- it's my way of indirectly learning and keeping my French alive.

-- START HERE --

The situation:
You're running linux. The client is running Windows. You are you, the person you're helping is The Client.
Set up your system before having the client follow the instructions here. I put the client instructions first so you can send them to this page.

---

The Client:
They are running Windows. They don't have a public IP. Here's what they should do:

1. Download tightvncviewer from here: http://www.tightvnc.com/download.php

2. Installation of tightvnc

a) Install both client and server or only server. It doesn't matter -- the server is important here.

b) During setup, make sure that you check all the boxes as shown below

You will not need to share your password with the person helping you. Make sure not to leave these blank.

c) Tightvnc requires the administrator password to be able to accept connections.

Right-click on the icon in the taskbar tray, and click on Configuration...

Change main server port to 5500. We're doing this to be consistent.
You may de-select 'Serve Java Viewer to Web clients)

The steps above you will only need to do once.

The following steps you will need to do each time you want to get help:
1. Right-click on the vnc icon in the taskbar tray. Select Attach Listening Viewer...

2. In Hostname or IP address of the viewer, type the IP address given to you by the person helping you. In this example it's 192.168.1.102, but it can be anything. Append ::5500 to the IP address. In our example, the input is 192.168.1.102::5500

That's it. If something not described here is happening, make sure to tell the person helping you. Especially if messages about Blocked programmes or Firewalls come up.

---

You (linux person):
Your public IP is 192.168.1.102 in the example above. Your port 5500 is open to the world.

sudo apt-get install vncviewer

Start your listening session by

vncviewer -compresslevel 9 -listen 0

vncviewer -listen: Listening on port 5500
vncviewer -listen: Command line errors are not reported until a connection comes in.

Once the connection is started by the client you get:

Connected to RFB server, using protocol version 3.8
Enabling TightVNC protocol extensions
No authentication needed
Authentication successful
Desktop name "tantalum"
VNC server default format:
  16 bits per pixel.
  Least significant byte first in each pixel.
  True colour: max red 31 green 63 blue 31, shift red 11 green 5 blue 0
Using default colormap which is TrueColor.  Pixel format:
  32 bits per pixel.
  Least significant byte first in each pixel.
  True colour: max red 255 green 255 blue 255, shift red 16 green 8 blue 0

You can play around with the compresslevels which runs from 0 to 9. Make sure to keep an eye on your firewall log -- your client/parent may easily get frustrated if things aren't working -- and you really don't want to be the cause of it.

---

Notes:
iptables
I tested this by running windows xp in virtualbox on one linux box, 192.168.1.1, and piped the vnc connection through to another linux box at 192.168.1.102. The firewall settings I had to do were:

On 192.168.1.1

Open port 17500 to tcp traffic from 192.168.1.102 to 255.255.255.255:
sudo iptables -A INPUT -i eth1 -s 192.168.1.102 -d 255.255.255.255 -p tcp --dport 17500 -j ACCEPT

On 192.168.1.102
Open port 5500 to tcp traffic destined for 192.168.1.102:
sudo iptables -A INPUT -d 192.168.1.102 -p tcp --dport 5500 -j ACCEPT

68. Reverse ssh tunnel to access server behind firewall

This isn't so much a post to show how to do it as it is a post describing a specific example of it in use. I basically spent two minutes looking at http://www.howtoforge.com/reverse-ssh-tunneling and was up and running in no time. It's that simple.

The sitation:
I have a computer at work. Opening up a port to allow for remote access is a headache and a half, since it involves getting signatures from a range of people and drawing up an IT security plan etc. As an academic during grant season I don't have that kind of time. Nor do I want to put up with all that BS. I also  understand that opening up ports willy-nilly can leads to security threats.

Anyway, I have iinet at home and they leave port 22 open by default. I have a Linksys WRT54 running Tomato and I allow key-based ssh external access.

My IP address is not static but changes perhaps once per month at most.
On my main desktop at home I run  this as a cron job:


#!/bin/bash
ipaddr=`wget http://automation.whatismyip.com/n09230945.asp -O - -o /dev/null`
when=`date +%a' '%d' '%b' '%Y' '%H':'%M`
echo $when $ipaddr >>/home/me/Dropbox/currentip.dat
exit 0

That way I can easily look up the latest ip address in my dropbox folder.

I run debian testing on all boxes.

Connecting via ssh to my home router works flawlessly. The other way doesn't work at all.

The solution:

We'll pretend that my home ip is 124.54.34.23 and my work ip is 169.23.54.6

At work

While at work, I connect to my home router using

ssh -R 19999:localhost:22 root@124.54.34.23

This logs me in to my Tomato router. Once in, start

top -d 600

This will keep top running, updating every ten minutes. This is to prevent the connection from being dropped.

The alternative is of course to use autossh -- the basic usage is just to replace ssh.

Now, go home

At home

log in to your router from the local network, then connect to port 19999 on localhost:

me@niobium:~$ ssh root@192.168.2.1

Tomato v1.28.1816

BusyBox v1.14.4 (2010-06-27 20:11:16 PDT) built-in shell (ash)
Enter 'help' for a list of built-in commands.

# ssh me@localhost -p 19999
me@localhost's password:

Linux beryllium 3.2.0-1-amd64 #1 SMP Sun Feb 5 15:17:15 UTC 2012 x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Wed Feb 15 18:55:45 2012 from localhost

me@beryllium:~$ 

Simple as that

67. Neat trick using reverse proxy -- several http hosts behind a firewall with only one port open

The situation: I was running two wireless webcams (Airlink101 AIC 250W) in order to monitor my laboratory. Both of these were connected to a linksys router. Only port 22 and 80 were opened by the university. We were forwarding port 80 to a Debian box running apache.

The goal: We wanted to have a page, e.g.www.externalhostname.com/image.html, serve up images from both the webcams. Using apache.

The solution:
A friend came up with this neat solution.

The following is assumed:

• The external dns name is www.externalhostname.com
• The cameras have the LAN ips 192.168.1.121 and 192.168.1.122

First the html file -- image.html:

<html>
<head>
<title>Lab Webcams</title>
<META HTTP-EQUIV="REFRESH" CONTENT="5">
</head>
<body bgcolor="rgb(0,0,122)" text="white">
<table border="1">
<tr>
<td>
Cam 1480
</td>
<td>
Cam 1485
</td>
<tr>
<td>
<img src="http://www.externalhostname.com/cam1/image.jpg" width="320" height="240"/>
</td>
<td>
<img src="http://www.externalhostname.com/cam2/image.jpg" width="320" height="240"/>
</td>
</table>
</body>
</htm>

Next, configure apache using /etc/apache2/httpd.conf:

LoadModule proxy_module /usr/lib/apache2/modules/mod_proxy.so
LoadModule proxy_http_module /usr/lib/apache2/modules/mod_proxy_http.so
LoadModule proxy_connect_module /usr/lib/apache2/modules/mod_proxy_connect.so
ProxyRequests Off
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPass /cam1 http://192.168.1.121
ProxyPassReverse /cam1 http://192.168.1.121
ProxyPass /cam2 http://192.168.1.122
ProxyPassReverse /cam2 http://192.168.1.122

Finally, copy the following from /etc/apache2/mods-available to /etc/apache2/mods-enabled:
proxy.conf

<IfModule mod_proxy.c>
</IfModule>

proxy_http.load

# Depends: proxy
LoadModule proxy_http_module /usr/lib/apache2/modules/mod_proxy_http.so

proxy.load

LoadModule proxy_module /usr/lib/apache2/modules/mod_proxy.so

That's it.

66. Minor bug: evolution 3.2.2 crashes, google chrome sync going crazy on debian testing

Symptom:

• Evolution 3.2.2 crashes every few minutes -- presumably on retrieving new mail every ten minutes
• Evolution uses 150% of your resources and becomes unresponsive if you try to recover messages lost in the crash
• If you sync your Chrome/Chromium browser with google, you get a warning triangle saying that sync failed. On trying to log in again the log-in window keeps disappearing

Logging evolution using CAMEL_DEBUG=all evolution >&evo.log gives

(evolution:8112): evolution-mail-CRITICAL **: e_mail_folder_uri_from_folder: assertion `CAMEL_IS_FOLDER (folder)' failed
**
GLib-GIO:ERROR:/tmp/buildd/glib2.0-2.30.2/./gio/gdbusmessage.c:1986:append_value_to_blob: assertion failed: (g_utf8_validate (v, -1, &end) && (end == v + len))
[imapx:F] adding command, fmt = 'IDLE'
[imapx:F] completing command buffer is [4] 'IDLE'
[imapx:F] Starting command (active=1, literal) F00104 IDLE
[imapx:F] camel_imapx_write: 'F00104 IDLE
'
[imapx:F] camel_imapx_read: buffer is '+ idling
'
[imapx:F] token '+'
[imapx:F] token TOKEN 'idling'
[imapx:F] token '
'
[imapx:F] Got continuation response for IDLE
[imapx:F] ** Starting next command
[imapx:F] * no, no jobs

Here's the log from another crash:

(evolution:19158): evolution-mail-CRITICAL **: e_mail_folder_uri_from_folder: assertion `CAMEL_IS_FOLDER (folder)' failed
(evolution:19158): GLib-CRITICAL **: g_hash_table_lookup: assertion `hash_table != NULL' failed
(evolution:19158): evolution-mail-CRITICAL **: e_mail_folder_uri_from_folder: assertion `CAMEL_IS_FOLDER (folder)' failed
**
GLib-GIO:ERROR:/tmp/buildd/glib2.0-2.30.2/./gio/gdbusmessage.c:1986:append_value_to_blob: assertion failed: (g_utf8_validate (v, -1, &end) && (end == v + len))

Followed by instant Evolution disappearance

If I start evolution and disable all gmail related accounts, it stays stable. If I enable our university gmail-hosted account it crashes --

Icedove/thunderbird is not crashing.

The whole thing seems to be a combination of weird Google stuff and Evolution behaviour.


Solution - sort of:  
I solved the google chrome issue by signing out/disabling sync, then re-enabling again. I also removed my Online Accounts in gnome, then added them again.
I also installed libnss3-tools due to some errors chrome was throwing up, but it's probably unrelated.
Evolution was still unhappy though. I did the 'windows' thing and rebooted -- evolution crashed after about twenty minutes. After starting evolution again it ran without a hickup for 8 hours before I shut my system down.

65. LaTeX for 2013 ARC proposals

If you're in the limbo where you're not quite expert enough with LaTeX, but have happily said goodbye to MS Office, and you're an Australia scientist who annually has to fight the endless bureaucracy of the Australian Research Council (seriously -- 10 page limit about the science, followed by 70 pages about personal qualities?) you may find that you're in for a lot of grief.

If you collaborate with other people you will have to settle for a file format everyone can deal with ('everyone' except linux users), which is likely to be Word run in a virtual environment. If you're writing by yourself or together with more computer-savvy colleagues you will get away with using LaTeX for a fair number of the sections, while using gEdit for others.

I'm presuming that you will create a separate .tex file for each section.

The rules (http://www.arc.gov.au/ncgp/dp/dp_instructions.htm) says that you need

• Black type
• Single column
• A4
• 0.5 cm margin on all sides (top, bottom, right, left)
• A 'highly legible font', preferably 12 pt Times New Roman

Also, reading between the lines, you shouldn't/needn't include page numbers since this will be generated by the RMS.

\usepackage{nopageno}

with \pagestyle{plain} just after \begin{document} may do the trick.


Black type and single column are defaults in LaTeX, so need no action. Everything below goes into the preamble.

For A4 and 12 point font, set that in the documentclass

\documentclass[12pt,a4paper]{article}\usepackage[latin1]{inputenc}\usepackage{amsmath}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{mhchem}

For 0.5 cm margins on all sides 

\usepackage[margin=0.5cm]{geometry}

Additional for all sections:





\setlength\parindent{0pt} %no indent on first paragraph
\usepackage{fullpage} % an alternative if you don't need 0.5 cm margins
\author{} %we don't want author\date{} %we don't want date\begin{document}



Part C, section C1
Here you have several sections, such as AIMS AND BACKGROUND, RESEARCH PROJECT etc. To make sure that these DO NOT get numbered, use

\seccounter{secnumbdepth}{0}

Since the references are a separate document, set

\documentclass[10pt,a4paper]{article}

Part D is filled out online.

Part E, while a pain to write, uses regular 12 pt Times New Roman. If you don't want section numbering, use {secnumbdepth}{0}. If you want your sections to look like this

E1. Justification....

E2. Details of non-ARC contributions

Use

\seccounter{secnumbdepth}{1}\def\thesection{E\arabic{section}.}

As usual, this goes into the preamble, not the body. You should -- ideally -- never have to change the way you write. All changes should go into the documentclass style file or preamble.


Part F. Oh how we curse you.

F13.1 should be easy enough -- you can even skip sections and just use \title{F13.1. Justification} together with a \maketitle right after \begin{document}. Or you can use
\def\thesection{F13.\arabic{section.}


F14.2 offers potential for fun:

\setcounter{secnumdepth}{4}
\def\thesection{F14.2.\arabic{section}.}
\def\thesubsection{\alph{subsection})}
\title{F14.2 Recent significant publications (since 2007)}

This way if you put the following in the body:

\maketitle
\section{Research publications}
\subsection{Scholarly books}
\subsection{Scholarly book chapters}

you get

F14.2 Recent significant publications (since 2007)
F14.2.1. Research publications
a) Scholarly books 
b) Scholarly book chapters

For your reference list you may use 

25.* Blow, Joe; Doe, John; \textbf{Hill, Vera}. \emph{One page derivation of the GUT}, Phys. Rev., \textbf{2012}, \emph{108(3)}, 234-235 \\
For F14.3 use

\def\thesection{F\arabic{section}.}
\def\thesubsection{14.\arabic{subsection}}
without \maketitle but with 

\subsection{Ten career-best publications} 
in the body

G1 is difficult in latex if you want to do it properly. I gave up and used libreoffice for this one since I couldn't figure out how 



Put this in the preamble

\usepackage{rotating} %rotate text in narrow tables...\usepackage{array} % for vertical centering in tables...hacky...

and this in the body:
\section{Research support for Vera Hill}
\begin{table}[h]
 \begin{tabular}{b{4cm} p{0.5cm} p{0.5cm} l c  c c c c}
\textbf{ Description} & \begin{sideways}Same Research area \end{sideways}  &  \begin{sideways}Support type \end{sideways}& ARC Project ID & 2010 (k\$) & 2011 (k\$) & 2012 (k\$) & 2013 (k\$) & 2014 (k\$) \\
 \hline
 "Unifying general relativity with quantum mechanics" & N & C  & DP100202921 & 120 & 190 & 180 & 170 & 120 \\
 \hline
 \end{tabular}
\end{table}

It doesn't look quite right. You may also want to try e.g. m{0.5cm} for the rotated header captions The remaining sections should be fairly straightforward, formatting-wise. Ultimately, the ARC application process is one of the most ridiculous one I've had the misfortune to be subjected to.