22 April 2012

123. Adding python support to nwchem under debian

I've posted the general build instructions for nwchem 6.0 with mpi support here: http://verahill.blogspot.com.au/2012/03/nwchem-60-with-openmpi-support-on.html

However, those instructions don't include python support.

0. Download,  extract nwchem and install blas etc. as shown in http://verahill.blogspot.com.au/2012/03/nwchem-60-with-openmpi-support-on.html

1. Edit nwchem-6.0/src/config/makefile.h
Change line 1962 from
EXTRA_LIBS +=    -lnwcutil  -lpthread -lutil -ldl
to
EXTRA_LIBS +=    -lnwcutil  -lpthread -lutil -ldl -lz -lssl
2. Install python headers
sudo apt-get install python2.7-dev

3. Execute the following commands (one by one or by putting them in a shell script)
export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export PYTHONHOME=/usr
export PYTHONVERSION=2.7
export USE_MPI=y
export USE_MPIF=y
export MPI_LOC=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openmpi/lib
export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"
cd $NWCHEM_TOP/src
make clean
make nwchem_config
make FC=gfortran

It should work fine and after a long build you'll have python enabled binaries.

4. Testing
You can test whether there's python support by creating test.nw with

python
for n in range(1,6):
        print n, n*2,n**2
end
task python
and running it with 
mpirun -n 1 nwchem test.nw

which gives



  NWChem Input Module
                                -------------------


                               NWChem Python program
                               ---------------------
for n in range(1,6):
        print n, n*2,n**2
1 2 1
2 4 4
3 6 9
4 8 16
5 10 25
1 2 1
2 4 4
3 6 9
4 8 16
5 10 25
 Task  times  cpu:        0.0s     wall:        0.0s

Done.


Note:
for ROCKS/CENTOS it was not necessary to edit src/config/makefile.h


The relevant parts in the the build configuration are
export NWCHEM_MODULES="all python"
export PYTHONHOME=/opt/rocks
export PYTHONVERSION=2.4

Other than that, just follow http://verahill.blogspot.com.au/2012/03/building-nwchem-60-on-rocks-543centos.html



Error:
gfortran: error: /usr/include/python2.7/lib/python2.7/config/libpython2.7.a: No such file or directory
make: *** [all] Error 1
locate libpython2.7.a
/usr/lib/libpython2.7.a
/usr/lib/python2.7/config/libpython2.7.a

Reason:
export PYTHONHOME=/usr/include/python2.7

Solution:
export PYTHONHOME=/usr


Error:
In function `PyZlib_compress':
(.text+0x1540): undefined reference to `deflateEnd'

Solution:
http://www.emsl.pnl.gov/docs/nwchem/nwchem-support/2012/02/0065.Re:_NWCHEM_undocumented_compilation_flag

Edit nwchem-6.0/src/config/makefile.h
For LINUX64 look at lines 1960-1964

1960      ifeq ($(BUILDING_PYTHON),python)
1961 #   EXTRA_LIBS += -ltk -ltcl -L/usr/X11R6/lib -lX11 -ldl
1962      EXTRA_LIBS +=    -lnwcutil  -lpthread -lutil -ldl
1963   LDOPTIONS = -Wl,--export-dynamic
1964      endif

Change line 1962 to
EXTRA_LIBS +=    -lnwcutil  -lpthread -lutil -ldl -lz -lssl


20 April 2012

122. DVB-T in VLC on Debian Wheezy

For some reason me-tv has stopped working properly today  (20/4/2012)-- I can't change the channels, screen size or drop down the menus. In order to quit, you have to kill the application. Something's clearly amiss. (Edit 22/5/2012: seems to be this bug https://bugs.launchpad.net/ubuntu/+source/me-tv/+bug/958751 -- 09/01/2013: it's been fixed now ) kaffeine, although a KDE app, also works well, and is very simple to set up -- the EPG  also works better than vlc.

This is Kaffeine, not VLC
This is me-tv, not VLC


Oh well.

Given that it's Friday night I had to come up with a quick fix.

I've got a Leadtek 1000 DTS, which lspci makes show up as
01:06.0 Multimedia controller: Philips Semiconductors SAA7130 Video Broadcast Decoder (rev 01)

See item 7 in http://verahill.blogspot.com.au/2012/01/debian-testing-64-wheezy-small-fixes.html if your card isn't recognised (Hint: enable PnP in the bios)

So, where's what I did:
1. Install gnome-dvb-*
sudo apt-get install gnome-dvb-daemon gnome-dvb-client vlc

2. Set up your tv card:
gnome-dvb-setup

This bit's easy. Just click your way through. Eventually you'll realize that while gnome-dvb sets up channels ok, it only support recording -- not tv watching. And the whole totem + gnome-dvb-daemon? Not working yet it seems.

3. Start vlc
vlc ~/.config/gnome-dvb-daemon/channels_DVB-T.conf

If all went well you are now watching TV!
You should have a list over channels in your playlist (ctrl+L).
The EPG support could perhaps be better (me-tv is awesome) but you'll find rudimentary listings by going to Tools/Program Guide.

As for me-tv...it might be the same as this bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=667731


 Looking at /var/cache/apt/archives I don't see anything obvious:

-rw-r--r-- 1 root root  23232372 Apr 16 05:18 linux-image-3.2.0-2-amd64_3.2.15-1_amd64.deb
-rw-r--r-- 1 root root   3505210 Apr 16 05:18 linux-headers-3.2.0-2-common_3.2.15-1_amd64.deb
-rw-r--r-- 1 root root    572064 Apr 16 05:18 linux-headers-3.2.0-2-amd64_3.2.15-1_amd64.deb
-rw-r--r-- 1 root root    244210 Apr 17 07:32 libtag1-vanilla_1.7.1-2_amd64.deb
-rw-r--r-- 1 root root      9132 Apr 17 07:32 libtag1c2a_1.7.1-2_amd64.deb
-rw-r--r-- 1 root root    195492 Apr 18 03:33 libservlet2.5-java_6.0.35-3_all.deb
-rw-r--r-- 1 root root   1348610 Apr 18 08:18 netpbm_2%3a10.0-15+b1_amd64.deb
-rw-r--r-- 1 root root     94438 Apr 18 08:18 libnetpbm10_2%3a10.0-15+b1_amd64.deb

(I use my own kernel and not the debian one)

Here's my current channel_DVB-T.conf list (Melbourne, Victoria, Australia):

ABC News 24:226500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:2314:0:560
ABC1:226500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:512:650:563
ABC2 / ABC4:226500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:2307:2308:562
ABC3:226500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:2311:2312:564
ABC Jazz:226500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:0:2318:567
ABC Dig Music:226500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:0:2317:566
ABC1:226500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:512:650:561
GO!:191625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:517:700:1074
GEM:191625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:512:0:1073
Nine Digital:191625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:519:720:1072
EXTRA:191625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:520:730:1075
ONE:219500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:514:0:1585
TEN Digital:219500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:512:650:1589
ONE:219500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:514:0:1591
ELEVEN:219500000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_3_4:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_16:HIERARCHY_NONE:516:681:1592
SBS ONE:536625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_2_3:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:161:81:785
SBS HD:536625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_2_3:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:102:103:789
SBS TWO:536625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_2_3:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:162:83:786
SBS 3:536625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_2_3:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:161:81:787
SBS 4:536625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_2_3:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:161:81:788
SBS Radio 1:536625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_2_3:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:0:201:798
SBS Radio 2:536625000:INVERSION_AUTO:BANDWIDTH_7_MHZ:FEC_2_3:FEC_NONE:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:0:202:799

7:177500000:INVERSION_OFF:BANDWIDTH_7_MHZ:FEC_2_3:FEC_2_3:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:769:770:1328
7 Two:177500000:INVERSION_OFF:BANDWIDTH_7_MHZ:FEC_2_3:FEC_2_3:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:769:770:1330
7 Mate:177500000:INVERSION_OFF:BANDWIDTH_7_MHZ:FEC_2_3:FEC_2_3:QAM_64:TRANSMISSION_MODE_8K:GUARD_INTERVAL_1_8:HIERARCHY_NONE:769:770:1331

121. Connection to remote server via an intermediary server

We have three computers: A, B and C

A is our desktop. C is a remote computer we want to connect to. B is a server in the middle.

For various reasons we want to first connect from A to B, then from B to C. A scenario is where C is behind a firewall bridged by B.

A specific scenario is this:
I have a linksys router, e.g. Terbium.internet.com, which manages the network in a lab. There are several computers on the network in the lab behind Terbium: Sulfur, Phosphorous, Oxygen (192.168.1.106) etc.
My home computer is Niobium.

Terbium is a linksys router (W54G) running tomato and you have to log in as root using a keyfile.
Oxygen is a debian box where I have a personal account as user verahill.

The ugly method:
In the same terminal on computer Niobium do
ssh root@terbium.internet.com
ssh verahill@192.168.1.106

This works fine for working remotely on oxygen (192.168.1.106). However, file transfer is a pain

The pretty method:
In one terminal on Niobium, do

ssh root@terbium.internet.com -L 9999:192.168.1.106:22

In another terminal on Niobium, do
ssh verahill@localhost -p 9999

Basically, all traffic to port 9999 on niobium is forwarded to port 22 on Oxygen -- not terbium.

The cool thing? File transfer is a breeze (via sftp using e.g. filezilla)!

It really is that simple

Web tunnel
to browse the internet via a tunnel it's still easier to do
ssh -C -D 9889 root@terbium.internet.com
and set your browser to use a SOCKS proxy -- point it to localhost, port 9889.