20 April 2012

120. Using truecrypt with dropbox


This is not some fancy, automatic solution. This is for people who may not be familiar with truecrypt and how to use it. To most people this will be obvious.

Basically, one way in which you can use truecrypt is to create a file which, when mounted, acts like a device/filesystem. This container is encrypted and the strength of encryption depends on your key or password.

As a solution it also suffers from slow initial syncing.

Setting it up is a piece of cake:
1. Install truecrypt.
Get the correct version e.g. "Standard x64" from http://www.truecrypt.org/downloads.


Install by extracting the file and executing it:
chmod +x truecrypt-7.1a-setup-x64
sudo ./truecrypt-7.1a-setup-x64

2. Start truecrypt and click on Create Volume

Select encrypted file container. The other option is more useful  if you have access to the hardware.

Using a Hidden truecrypt volume means you pay a space penalty, depending on how you distribute the space between the two volumes.

Select location and filename


Pick an algorithm. I don't know much about this, but using a cascade sounds reasonable. I guess there are performance penalties though.

Be aware that the container file will take up all this space -- whether it's empty or not.









 3. Use the container file.
Click on Select File, then mount. You typically need to supply both the container password and your admin password.


This is what the inside of the container file looks like.
And this is what the dropbox folder looks like

19 April 2012

119. Installing sun java/Oracle java in debian

Update 23 March 2013:  the java.com file isn't distributed as a .bin file anymore. Just replace all instances of .bin with .tar.gz and it works just as well.

This basically follows http://sylvestre.ledru.info/blog/sylvestre/2012/02/29/java_package_replacement_of_sun_java6

While a few applications (Grix and Grisu) I was playing with which require java 1.5 or better should work fine with openjdk-7 (1.6.0_24), they didnt. I had to run it using binary packages which I downloaded from java.com and then installed locally.

I'd like a better solution, and here it is:

1. sudo apt-get install java-package

2. Download the java files from java.com, e.g. jre-6u31-linux-x64.bin or jre-7u17-linux-x64.tar.gz
http://java.com/en/download/manual.jsp?locale=en

3. Make-jpkg
Run make-jpkg on the downloaded file, whether it's ending in .bin or .tar.gz:

make-jpkg jre-6u31-linux-x64.bin
Create debian package:
    dh_testdir
    dh_testroot
    dh_installchangelogs
    dh_installdocs
    dh_compress
    dh_fixperms
    dh_installdeb
    dh_shlibdeps
dpkg-shlibdeps: warning: Can't extract name and version from library name `libjvm.so'
[..]
dpkg-shlibdeps: warning: Can't extract name and version from library name `libjvm.so'
[..]
dpkg-shlibdeps: warning: Can't extract name and version from library name `libjli.so'
dpkg-shlibdeps: warning: package could avoid a useless dependency if /tmp/make-jpkg.9XTCruNvKM/install/usr/lib/jvm/j2re1.6-oracle/lib/amd64/native_threads/libhpi.so /tmp/make-jpkg.9XTCruNvKM/install/usr/lib/jvm/j2re1.6-oracle/lib/amd64/libjava.so /tmp/make-jpkg.9XTCruNvKM/install/usr/lib/jvm/j2re1.6-oracle/lib/amd64/libdt_socket.so /tmp/make-jpkg.9XTCruNvKM/install/usr/lib/jvm/j2re1.6-oracle/lib/amd64/libnet.so /tmp/make-jpkg.9XTCruNvKM/install/usr/lib/jvm/j2re1.6-oracle/bin/javaws /tmp/make-jpkg.9XTCruNvKM/install/usr/lib/jvm/j2re1.6-oracle/lib/amd64/libhprof.so were not linked against libnsl.so.1 (they use none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if /tmp/make-jpkg.9XTCruNvKM/install/usr/lib/jvm/j2re1.6-oracle/lib/amd64/libjawt.so was not linked against libmawt.so (it uses none of the library's symbols).
    dh_gencontrol
    dh_md5sums
    dh_builddeb
dpkg-deb: building package `oracle-j2re1.6' in `/tmp/make-jpkg.9XTCruNvKM/oracle-j2re1.6_1.6.0+update31_amd64.deb'.
    copy oracle-j2re1.6_1.6.0+update31_amd64.deb into directory /home/me/Downloads/
The Debian package has been created in the current directory. You can
install the package as root (e.g. dpkg -i oracle-j2re1.6_1.6.0+update31_amd64.deb). 
4. sudo dpkg -i oracle-j2re1.6_1.6.0+update31_amd64.deb 
Selecting previously unselected package oracle-j2re1.6.
(Reading database ... 561777 files and directories currently installed.)
Unpacking oracle-j2re1.6 (from oracle-j2re1.6_1.6.0+update31_amd64.deb) ...
Setting up oracle-j2re1.6 (1.6.0+update31) ...
update-alternatives: using /usr/lib/jvm/j2re1.6-oracle/bin/ControlPanel to provide /usr/bin/ControlPanel (ControlPanel) in auto mode.
update-alternatives: using /usr/lib/jvm/j2re1.6-oracle/lib/amd64/libnpjp2.so to provide /usr/lib/iceweasel/plugins/libjavaplugin.so (iceweasel-javaplugin.so) in auto mode.
update-alternatives: using /usr/lib/jvm/j2re1.6-oracle/lib/amd64/libnpjp2.so to provide /usr/lib/chromium/plugins/libjavaplugin.so (chromium-javaplugin.so) in auto mode.

5a. sudo update-alternatives --config java
There are 5 choices for the alternative java (providing /usr/bin/java).

  Selection    Path                                            Priority   Status------------------------------------------------------------* 0            /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java   1061      auto mode  1            /usr/bin/gij-4.4                                 1044      manual mode  2            /usr/bin/gij-4.6                                 1046      manual mode  3            /usr/lib/jvm/j2re1.6-oracle/bin/java             314       manual mode  4            /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java   1061      manual mode  5            /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java   1051      manual modePress enter to keep the current choice[*], or type selection number: 3update-alternatives: using /usr/lib/jvm/j2re1.6-oracle/bin/java to provide /usr/bin/java (java) in manual mode.
5b. 
 sudo update-alternatives --config javaws


There are 2 choices for the alternative javaws (providing /usr/bin/javaws).
  Selection    Path                                              Priority   Status
------------------------------------------------------------
* 0            /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/javaws   1061      auto mode
  1            /usr/lib/jvm/j2re1.6-oracle/bin/javaws             314       manual mode
  2            /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/javaws   1061      manual mode
Press enter to keep the current choice[*], or type selection number: 1
update-alternatives: using /usr/lib/jvm/j2re1.6-oracle/bin/javaws to provide /usr/bin/javaws (javaws) in manual mode.
update-alternatives: warning: skip creation of /usr/share/man/man1/javaws.1.gz because associated file /usr/lib/jvm/j2re1.6-oracle/man/man1/javaws.1.gz (of link group javaws) doesn't exist.



6.  Verification
java -version
java version "1.6.0_31"
Java(TM) SE Runtime Environment (build 1.6.0_31-b04)
Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode)
Go to http://java.com/en/download/installed.jsp


Iceweasel worked fine at this stage.

7. To get Chrome to work I did the following:
sudo updatedb
locate libjavaplugin.so
/usr/lib/chromium/plugins/libjavaplugin.so
/usr/lib/iceweasel/plugins/libjavaplugin.so
/usr/lib/mozilla/plugins/libjavaplugin.so
 ls -lah /usr/lib/chromium/plugins/libjavaplugin.so etc.
/usr/lib/chromium/plugins/libjavaplugin.so -> /etc/alternatives/chromium-javaplugin.so
/usr/lib/iceweasel/plugins/libjavaplugin.so -> /etc/alternatives/iceweasel-javaplugin.so
/usr/lib/mozilla/plugins/libjavaplugin.so -> /etc/alternatives/mozilla-javaplugin.so
ls -lah /etc/alternatives/chromium-javaplugin.so etc.
/etc/alternatives/chromium-javaplugin.so -> /usr/lib/jvm/j2re1.6-oracle/lib/amd64/libnpjp2.so
/etc/alternatives/iceweasel-javaplugin.so -> /usr/lib/jvm/j2re1.6-oracle/lib/amd64/libnpjp2.so
/etc/alternatives/mozilla-javaplugin.so -> /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64/IcedTeaPlugin.so
sudo rm /etc/alternatives/mozilla-javaplugin.so
sudo ln -s /usr/lib/jvm/j2re1.6-oracle/lib/amd64/libnpjp2.so /etc/alternatives/mozilla-javaplugin.so


And you're done.



Links to this post:
https://www.linuxquestions.org/questions/debian-26/which-java-for-wheezy-4175469043/

118. Solution to nwchem: SHMMAX too small

Update: also see this post: http://verahill.blogspot.com.au/2012/10/shmmax-revisited-and-shmall-shmmni.html

When running nwchem using mpirun I've occasionally encountered this error.

Error:
******************* ARMCI INFO ************************
The application attempted to allocate a shared memory segment of 44498944 bytes in size. This might be in addition to segments that were allocated succesfully previously. The current system configuration does not allow enough shared memory to be allocated to the application.

This is most often caused by:
1) system parameter SHMMAX (largest shared memory segment) being too small or
2) insufficient swap space.
Please ask your system administrator to verify if SHMMAX matches the amount of memory needed by your application and the system has sufficient amount of swap space. Most UNIX systems can be easily reconfigured to allow larger shared memory segments,
see http://www.emsl.pnl.gov/docs/global/support.html
In some cases, the problem might be caused by insufficient swap space.
*******************************************************
0:allocate: failed to create shared region : -1
(rank:0 hostname:boron pid:17222):ARMCI DASSERT fail. shmem.c:armci_allocate():1082 cond:0

Diagnosis:
Check the currently defined shmmax:
cat /proc/sys/kernel/shmmax
33554432
Well, 33554432<44498944, so it seems that it's caused by reason 1 above.

Solution:

Edit /etc/sysctl.conf
Add a line saying
kernel.shmmax=44498944
Save and reboot. The exact value is up to you -- I've set my shmmax to 128*1024*1024=134217728, while our production cluster has 6269961216.

Update: to change it on the fly do
sudo sysctl -w kernel.shmmax=6269961216