10 January 2013

314. Briefly: Installing talkd on ROCKS 5.4.3

I was asked to set up talkd on our Rocks 5.4.3 cluster (Centos 5.4) . There's no talkd or talk-server packages in the repos on that server.

Note: The general consensus seems to be that talk is
1. insecure and
2. outdated.

To install:
wget http://mirror.centos.org/centos/5/os/x86_64/CentOS/talk-server-0.17-31.el5.x86_64.rpm
yum localinstall talk-server-0.17-31.el5.x86_64.rpm
sudo iptables -A INPUT -p udp --dport 517 -s 127.0.0.1 -j ACCEPT
sudo iptables -A INPUT -p udp --dport 518 -s 127.0.0.1 -j ACCEPT

Above I've added, but haven't yet tried, -s 127.0.0.1 to limit connections from the local computer (localhost). If it doesn't work with -s 127.0.0.1, then try without -- but then be aware of the security implications. These rules also aren't permanent and will be lost on reboot. To make them permanent, edit /etc/sysconfig/iptables.

I couldn't get talk to work before opening the ports and would get
Error on read from talk daemon : Connection refused
Note that talkd uses Xinet and not init -- it will not run as  memory resident daemon, but instead be launched by xinet only when talkd is needed (traffic is detected to the ports associated with talkd). Xinet.d is a bit like a concierge, waking up whomever seems to be the adressee.


Edit both /etc/xinetd.d/talk and /etc/xinetd.d/ntalk. Change to:
# default: off # description: The talk server accepts talk requests for chatting with users \ # on other systems. service talk { flags = IPv4 disable = no socket_type = dgram wait = yes user = nobody group = tty server = /usr/sbin/in.talkd }
Finally, restart xinet.d (doing chkconfig talk on wasn't enough):
sudo /etc/init.d/xinet.d restart



Note: If you or the other user have several terminals open you should figure out which terminal to use. If you're user2, do
ps
PID TTY TIME CMD 5455 pts/23 00:00:00 bash 9321 pts/23 00:00:00 ps
user1 can then do
[user1@host ~]$ talk user2@localhost pts/23

and user2 will see the following in that terminal:
Message from Talk_Daemon@host at 14:49 ... talk: connection requested by user1@localhost.localdomain. talk: respond with: talk user1@localhost.localdomain
If you are user1 and have no idea on what terminal user2 is logged on, you can try
w|grep user2
user2 pts/8 remote:S.0 04Nov12 59:00 0.04s 0.04s /bin/bash user2 pts/9 remote:S.1 04Nov12 17:28 0.03s 0.03s /bin/bash user2 pts/10 remote Mon12 2days 0.03s 0.00s ssh -X -v volde user2 pts/11 local Mon12 2days 0.29s 0.26s perl eccejobmon user2 pts/23 remote 14:30 7.00s 0.00s 0.00s -bash

09 January 2013

313. Which Office for linux users? Libreoffice, Openoffice, Lotus Symphony, Google Docs, Microsoft Live/Skydrive, Wine/Office2003 and Office2003 in Virtualbox, WPS Office

UPDATE 6 Apr 2013: I've added WPS Office

UPDATE 10 Feb 2013: I've tried Libreoffice 4.0 as well -- no improvement in terms of rendering vs Libreoffice 3.5.4 (below). The start-up times are greatly improved though.

UPDATE 4 Feb. 2013: Equation Editor in Office 2003 on Wine used to be supported -- however, while fixing the editor in office 2010 support for older office versions broke. This will (hopefully) be fixed in future versions and the codeweaver and wine devs are aware of the issue. Bug reports to that effect:
http://bugs.winehq.org/show_bug.cgi?id=32697 and http://www.codeweavers.com/support/tickets/browse/?ticket_id=931759

Most of my publications are via collaborations, and I'm more or less the only one who uses Linux. When I'm in charge, I use latex -- but you can be a zealot, or you can be a pragmatist. Life tends to be easier for the pragmatic. I draw the line at actually using a native installation of Windows though.

A few things are important: figures should render ok, endnote should render ok, I need to use track changes and I should be able to insert equations. Oh, and the templates from the major publishers (RSC, Wiley, ACS) should work.

So, once and for all (well, I should revise this for each major version I suppose) here's an evaluation of  where each solution on Linux falls short and where it does well.

I mocked up a simple article style doc file using Word 2003, Endnote X and Microsoft Equation Editor 3.0 and the Angewandte Chemie template, and then opened it in various alternative solutions.

I tried:

  • LibreOffice
  • OpenOffice
  • Lotus Symphony
  • Abiword
  • Google Docs/Drive
  • Microsoft Live/Skydrive


Here's what I found:

NOTE: when I say 'good' or 'bad' I mean how well they reproduce the look which is found in Word 2003. It's not a judgement on the quality of the word processor as such. I also recognise that for a developer it isnt much fun spending time working on reproducing someone else's crappy and poorly documented software, when you could be implementing new and wonderful things instead.

Outcome: all non-Word 2003 solutions at a minimum had the figure move slightly relative to the text. All solutions did the Endnote references ok, if not the location of them. Some solutions had issues with the greek letters in the equation. All non-MS solutions rendered the equation as an image which couldn't be edited.

See for yourself...


Office 2003/Virtualbox
This is the canonical version -- this is how it was created and how it should look.
Word 2003 in Virtualbox


Libreoffice 1:3.5.4+dfsg-4 
Looks fine, but
* The numbering for the equation (put it there using a tab stop) is too far left
* part of a paragraph now separates the figure from its caption -- the picture hasn't moved, but the first column on the first page seems to be one line shorter, which pushes the text of the previous paragraph past the figure.
* The last paragraph doesn't end on the first page, but in the first column of the second page..
* The second page also looks different, with the endnote bit moved to the second column.
LibreOffice


Openoffice 3.4.1-1
I grabbed the latest debs from http://www.openoffice.org/download/other.html#tested-full
On the second page the Endnote references are in the right place (c.f. Libreoffice).

* Greek letters in Equation replaced by squares (missing font?). (greek letters in text is ok)
* The numbering for the equation (put it there using a tab stop) is too far left
* part of a paragraph now separates the figure from its caption -- the picture hasn't moved, but the first column on the first page seems to be one line shorter, which pushes the text of the previous paragraph past the figure.
* The last paragraph doesn't end on the first page, but in the first column of the second page..


OpenOffice


Lotus Symphony 3.0.1
I downloaded the Lucid i386 deb and installed it in an Ubuntu Lucid virtual machine. Symphony is based on OpenOffice.

* Greek letters in Equation disappeared completely. (greek letters in text is ok)
* The numbering for the equation (put it there using a tab stop) is too far left
* part of a paragraph now separates the figure from its caption -- the picture hasn't moved, but the first column on the first page seems to be one line shorter, which pushes the text of the previous paragraph past the figure.
* The last paragraph doesn't end on the first page, but in the first column of the second page..
* The second page also looks different, with the endnote bit moved to the second column.


Lotus Symphony


Office 2003/Wine
See here for installation: http://verahill.blogspot.com.au/2013/01/307-installing-office-2003-in-wine-on.html
Everything looks fine with one pretty important exception:
* The equation is missing completely. Note that other people may have better luck installing Equation editor under wine -- I haven't managed to install it at all.
Word 2003 in Wine
Abiword
Plenty of issues:
* The author box has move to the top left corner.
* Can't find the table
* Greek letters missing in equation
* Greek letter in text missing
* Lots of stuff missing or misplaced (e.g. Keyword background picture)
Abiword


Google Docs
This was the big surprise -- I've always thought that google docs was essentially as 'good' as Microsoft Office. They have a financial incentive to be, after all. Turns out that the document was all but unrecognisable when opened in Google Docs. By far the worst solution. Note: the out-of-date warning is triggered by Chrome 18. Most similar to Abiword.
Google Docs

Microsoft Live 
Yes, that's right -- Microsoft Live works on Linux. This is the free version you get via Hotmail/Skydrive -- not their Office 365 or whatever they use now.
There are more surprises -- it doesn't render the document faithfully.

View:
* The numbering for the equation (put it there using a tab stop) is too far left. (like Libreoffice)
* part of a paragraph now separates the figure from its caption -- the picture hasn't moved, but the first column on the first page seems to be one line shorter, which pushes the text of the previous paragraph past the figure. (like Libreoffice)
* The last paragraph doesn't end on the first page, but in the first column of the second page. (like Libreoffice)
* The last paragraph doesn't end on the first page, but in the first column of the second page. (like Openoffice)
In other words -- Libreoffice/Openoffice do just as well as Microsoft Live


The interface for the View mode


full -- and no, you can't zoom in and out...


Editing:
In editing mode all it looks like a mess -- see figures.
In addition to the obvious formatting mess,
* greek letters in text show up as squares (problem with fonts on my computer or problems with live?)
* The equation cannot be edited.
However, once you go back to View mode the formatting is restored -- but it'd leave you flying blind while trying to make changes.

Editing mode


The full document -- yes, there's no Zoom...
WPS (Kingsoft)
I only discovered this by accident. Kingsoft (金山软件) is a legitimate Chinese company (Star Office was big in the 90s) and somehow there's a linux version (i386) of their Office suite: http://linux.wps.cn/. The bad news is that it's in Chinese, but that doesn't stop me from evaluating it -- it's such a close copy to Office 2007 that you can pretty much guess how to use it.

You can download it by clicking on this link: http://wdl.cache.ijinshan.com/wps/download/Linux/unstable/wps-office_8.1.0.3724~b1p2_i386.deb

It seems to be reproducing the file pretty well, with the exception of how it renders the equation and the missing stars in the author affiliation section. However, the location of the figure caption is correct, and the second page looks good.

And here's how it looks:




312. Tau + OpenMPI profiling on Debian Testing/Wheezy

Still searching for a way to easily look at the execution of parallel jobs I came across TAU: http://www.cs.uoregon.edu/Research/tau/home.php

You can download without registering, but please do register as the number of registered users tend to be important for funding and evaluation of software development in academia: http://www.cs.uoregon.edu/Research/tau/downloads.php

I'm not really sure about how to use PDT, and I've used Tau without it before without any problems.

The compilation order below is also important -- pdt won't build without libpdb.a which is generated by tau -- but you can't configure tau with -pdt if it doesn't exist.


Compiling
sudo mkdir /opt/tau
sudo chown $USER /opt/tau
cd /opt/tau

wget http://tu-dresden.de/die_tu_dresden/zentrale_einrichtungen/zih/forschung/software_werkzeuge_zur_unterstuetzung_von_programmierung_und_optimierung/otf/dateien/OTF-1.12.2salmon.tar.gz
tar xvf OTF-1.12.2salmon.tar.gz
cd /OTF-1.12.2salmon/
./configure --prefix=/opt/tau/OTF
make
make install
cd ../

wget http://tau.uoregon.edu/tau.tgz
tar xvf tau.tgz
cd tau-2.22-p1/
./configure -mpilib=/usr/lib/openmpi/lib -prefix=/opt/tau -openmp -TRACE -iowrapper -otf=/opt/tau/OTF -pthread
make install
cd ../

wget http://tau.uoregon.edu/pdt.tar.gz
tar xvf pdt.tar.gz
cd pdtoolkit-3.18.1/
./configure -prefix=/opt/tau/pdt
make
make install


cd ../tau-2.22-p1/
./configure -mpilib=/usr/lib/openmpi/lib -prefix=/opt/tau -openmp -TRACE -iowrapper -pthread -otf=/opt/tau/OTF -pdt=/opt/tau/pdt

make install


Testing
Time to try it out on something parallel.

First set the path

PATH=$PATH:/opt/tau/x86_64/bin

I used nwchem with this input file, co2.nw:
title "co nmr" geometry c 0 0 0 o 0 0 1.13 end basis * library "6-311+G*" end property shielding end dft direct grid fine mult 1 xc HFexch 0.05 slater 0.95 becke88 nonlocal 0.72 vwn_5 1 perdew91 0.81 end task dft property

and ran it using
mpirun -n 3 tau_exec nwchem co2.nw

which ends with
Total times cpu: 4.8s wall: 7.6s
It's obviously a bit too short, but will do for illustration purposes.

That generates a set of files, profile.*.0.0 -- one for each thread i.e. profile.1.0.0, profile.2.0.0 and profile.3.0.0 in this particular case. There are a lot of options for tracing, using hardware counters etc. -- see http://www.cs.uoregon.edu/Research/tau/docs/newguide/
pprof -s
Reading Profile files in profile.* FUNCTION SUMMARY (total): --------------------------------------------------------------------------------------- %Time Exclusive Inclusive #Call #Subrs Inclusive Name msec total msec usec/call --------------------------------------------------------------------------------------- 100.0 15,813 25,931 3 14276 8643959 .TAU application 18.8 4,870 4,870 10272 0 474 MPI_Barrier() 12.1 3,138 3,138 3 0 1046279 MPI_Init() 8.1 2,090 2,090 818 0 2556 MPI_Recv() 0.0 9 9 3 0 3173 MPI_Finalize() 0.0 3 3 24 0 128 MPI_Bcast() 0.0 2 2 6 0 463 MPI_Comm_dup() 0.0 1 1 790 0 2 MPI_Comm_size() 0.0 0.872 0.872 818 0 1 MPI_Send() 0.0 0.294 0.294 841 0 0 MPI_Comm_rank() 0.0 0.17 0.17 674 0 0 MPI_Get_count() 0.0 0.111 0.111 3 0 37 MPI_Comm_free() 0.0 0.026 0.026 3 0 9 MPI_Errhandler_set() 0.0 0.024 0.024 6 0 4 MPI_Group_rank() 0.0 0.02 0.02 6 0 3 MPI_Comm_compare() 0.0 0.015 0.015 4 0 4 MPI_Comm_group() 0.0 0.008 0.008 4 0 2 MPI_Group_size() 0.0 0.004 0.004 1 0 4 MPI_Group_translate_ranks() FUNCTION SUMMARY (mean): --------------------------------------------------------------------------------------- %Time Exclusive Inclusive #Call #Subrs Inclusive Name msec total msec usec/call --------------------------------------------------------------------------------------- 100.0 5,271 8,643 1 4758.67 8643959 .TAU application 18.8 1,623 1,623 3424 0 474 MPI_Barrier() 12.1 1,046 1,046 1 0 1046279 MPI_Init() 8.1 696 696 272.667 0 2556 MPI_Recv() 0.0 3 3 1 0 3173 MPI_Finalize() 0.0 1 1 8 0 128 MPI_Bcast() 0.0 0.926 0.926 2 0 463 MPI_Comm_dup() 0.0 0.436 0.436 263.333 0 2 MPI_Comm_size() 0.0 0.291 0.291 272.667 0 1 MPI_Send() 0.0 0.098 0.098 280.333 0 0 MPI_Comm_rank() 0.0 0.0567 0.0567 224.667 0 0 MPI_Get_count() 0.0 0.037 0.037 1 0 37 MPI_Comm_free() 0.0 0.00867 0.00867 1 0 9 MPI_Errhandler_set() 0.0 0.008 0.008 2 0 4 MPI_Group_rank() 0.0 0.00667 0.00667 2 0 3 MPI_Comm_compare() 0.0 0.005 0.005 1.33333 0 4 MPI_Comm_group() 0.0 0.00267 0.00267 1.33333 0 2 MPI_Group_size() 0.0 0.00133 0.00133 0.333333 0 4 MPI_Group_translate_ranks()

...which I can't pretend to understand. Reasonably, the first line would be the cpu time and the wall time (4.8 and 7.6 s vs 5,271 and 8,643 ms).

A visual representation can be had by launching paraprof:
paraprof


Now it's time to explore...

The one thing that doesn't seem to work is visualisation of the communication matrix...



Failed attempt to build with vampirtrace
sudo mkdir /opt/tau
sudo chown $USER /opt/tau
cd /opt/tau


wget http://tu-dresden.de/die_tu_dresden/zentrale_einrichtungen/zih/forschung/software_werkzeuge_zur_unterstuetzung_von_programmierung_und_optimierung/otf/dateien/OTF-1.12.2salmon.tar.gz
tar xvf OTF-1.12.2salmon.tar.gz
cd /OTF-1.12.2salmon/
./configure --prefix=/opt/tau/OTF
make
make install
cd ../


wget http://tu-dresden.de/die_tu_dresden/zentrale_einrichtungen/zih/forschung/software_werkzeuge_zur_unterstuetzung_von_programmierung_und_optimierung/vampirtrace/dateien/VampirTrace-5.14.1.tar.gz
tar xvf VampirTrace-5.14.1.tar.gz
cd VampirTrace-5.14.1/
./configure --prefix=/opt/tau/vampirtrace --with-mpi-dir=/usr/lib/openmpi/lib --with-extern-otf-dir=/opt/tau/OTF
make
make install


wget http://tau.uoregon.edu/tau.tgz
tar xvf tau.tgz
cd tau-2.22-p1/
./configure -mpilib=/usr/lib/openmpi/lib -prefix=/opt/tau -openmp -TRACE -iowrapper -otf=/opt/tau/OTF -vampirtrace=/opt/tau/vampirtrace
make install

It builds fine, but during execution of mpirun -n 2 tau_exec... I get
Error: No matching binding for 'mpi' in directory /opt/tau/x86_64/lib
Available bindings (/opt/tau/x86_64/lib):
Error: No matching binding for 'mpi' in directory /opt/tau/x86_64/lib
Available bindings (/opt/tau/x86_64/lib):
  /opt/tau/x86_64/lib/shared-disable
  /opt/tau/x86_64/lib/shared-disable