28 February 2013

348. Using the official firefox 19 binary on debian testing/wheezy

UPDATE 20/3/2013: installing a system from scratch I'm having issues with xulrunner-10.0:i386, since it pulls in libstartup-notification0:i386 which causes all sorts of headaches (uninstalls xulrunner-10.0 and everything it depends on)

This post made me realise that installing the pre-built firefox binary isn't as straight-forward as one might like. A small nuisance is that the getfirefox.com versoin is only 32 bit, but that's life.

Here we go.

Download the standard binary version of firefox from here:
http://www.mozilla.org/en-US/firefox/new/?from=getfirefox

You can install it wherever you want, but I'm going for a local installation. Put the firefox-19.0.tar.bz2 in your home folder and untar it:
tar xvf firefox-19.0.tar.bz2
cd ~/firefox/
./firefox

If you get
bash: ./firefox: No such file or directory

then you are missing 32 bit compatibility libraries:
sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install ia32-libs
sudo apt-get install xulrunner-10.0:i386

To get flash set up you'll need the 32 bit version. Download the 32 bit version from http://get.adobe.com/flashplayer/?no_redirect

mkdir ~/.mozilla/plugins -p
mv ~/Downloads/install_flash_player_11_linux.i386.tar.gz ~/firefox
cd ~/firefox/
tar xvf install_flash_player_11_linux.i386.tar.gz
cp libflashplayer.so ~/.mozilla/plugins/libflashplayer.so

And that should leave you with a working firefox versions. Note that running 'firefox' as a command will invoke iceweasel if installed. To get around that you can e.g. do

echo "alias ffox='$HOME/firefox/./firefox'" >> ~/.bashrc
source ~/.bashrc

after which you can launch firefox by running ffox form the terminal. A standard installation would be to put everything in /opt and set up a .desktop file -- but there's plenty of guides to how to do that.

347. Minor ECCE oddity when pasting basis sets from BSE: lines longer than 254 chars wreak havoc

Using lines longer than 254 chars when editing nwchem input in ECCE leads to the rest of the input being dropped.

I discovered this when pasting basis sets from bse.pnl.gov. If you paste something which has a line longer than 254 chars, such as the one starting with # H He and ending with valence below (345 chars), everything that comes after that line will be dropped.
# Def2-SVP EMSL Basis Set Exchange Library 2/27/13 8:08 PM # Elements References # -------- ---------- # H He Li Be B C N O F Ne Na Mg Al Si P S Cl Ar K Ca Sc Ti V Cr Mn Fe Co Ni Cu Zn Ga Ge As Se Br Kr Rb Sr Y Zr Nb Mo Tc Ru Rh Pd Ag Cd In Sn Sb Te I Xe Cs Ba La Hf Ta W Re Os Ir Pt Au Hg Tl Pb Bi Po At Rn : F. Weigend and R. Ahlrichs, Phys. Chem. Chem. Phys., Balanced basis sets of split valence, triple zeta valence and quadruple zeta valence # quality for H to Rn: Design and assessment of accuracy 7, 3297 (2005). # BASIS "ao basis" PRINT #BASIS SET: (4s,1p) -> [2s,1p] H S 13.0107010 0.19682158E-01 1.9622572 0.13796524 0.44453796 0.47831935 H S 0.12194962 1.0000000 H P 0.8000000 1.0000000 #BASIS SET: (7s,4p,1d) -> [3s,2p,1d] O S 2266.1767785 -0.53431809926E-02 340.87010191 -0.39890039230E-01 77.363135167 -0.17853911985 21.479644940 -0.46427684959 6.6589433124 -0.44309745172 O S 0.80975975668 1.0000000 O S 0.25530772234 1.0000000 [..]

To reproduce, set up a calculation. In the editor, click on 'Final Edit'. Now paste your basis set. Save and exit (it's vi/m, so that means using :wq). 

Everything seems to be fine

Now, either select the job and hit Ctrl+I to see the input, or open the editor and click on 'Final Edit' again.

Nothing below the line immediately preceding the long line will be saved. It's not a visualisation issue either -- if you launch the job and do ctrl+o to see what NWChem received as input, it mirrors what you see as input.

Pasting anything other than that overly long line works fine.




A more artificial example would be to try to save this
a
b
abcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyz1234
d
e

which works, vs this:
a
b
abcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyzabcdefghijklmnopqrstuvxyz12345
d
e

which doesn't. There's a difference of one character.

27 February 2013

346. Tripwire -- keeping an eye on important files

The whole UEFI/Secure Boot debacle lead me to this post: http://arstechnica.com/information-technology/2013/02/linus-torvalds-i-will-not-change-linux-to-deep-throat-microsoft/

In the comments section 'rhavenn' wrote this in passing: " ...by taking a SHA512 hash of your kernel, offloading that to a 3rd party and occasionally verifying the hash. It's nothing tripwire or any other software hasn't been doing for ages. "

Which lead me to check if tripwire was in the debian repos -- and it is.
UPDATE: You can also checkout 'aide' which is also in the repos.

Given how I use my work computers I'm more concerned with disk corruption than infection by malicious code, but as someone who goes through the occasional airport, and with it the potential risk of having to hand over my laptop (never happened so far), I do see a the use for tripwire. Also, in spite of the almost complete lack of viruses in the wild for linux, one should never be complacent (just look at OS X). Besides, learning new things is always a good thing.

Although to be fair, what tripwire does is what most (decent) antivirus programs also do.

Here's how to get set up with tripwire.


Installation

sudo apt-get install tripwire

You'll be asked to answer a series of questions:

Tripwire uses a pair of keys to sign various files, thus ensuring their unaltered state. By acceptin here, you will be prompted for the passphrase for the first of those keys, the site key, during the installation. You are also agreeing to create a site key if one doesn't exist already. Tripwire uses the site key to sign files that may be common to multiple systems, e.g. the configuration & policy files. See twfiles(5) for more information. Unfortunately, due to the Debian installation process, there is a period of time where this passpgrase exists in an unencrypted format. Were an attacker to have access to your machine during this period, he could possibly retrieve your passphrase and use it at some later point. If you would rather not have this exposure, decline here. You will then need to create a site key, configuration file & policy file by hand. See twadmin(8) for more information. Do you wish to create/use your site key passphrase during installation?
Tripwire keeps its configuration in a encrypted database that is generated, by default, from /etc/tripwire/twcfg.txt Any changes to /etc/tripwire/twcfg.txt, either as a result of a change in this package or due to administrator activity, require the regeneration of the encrypted database before they will take effect. Selecting this action will result in your being prompted for the site key passphrase during the post-installation process of this package. Rebuild Tripwire configuration file?
Tripwire uses two different keys for authentication and encryption of files. The site key is used to protect files that could be used across several systems. This includes the policy and configuration files. You are being prompted for this passphrase either because no site key exists at this time or because you have requested the rebuilding of the policy or configuration files. Remember this passphrase; it is not stored anywhere! Enter site-key passphrase:
Tripwire uses two different keys for authentication and encryption of files. The local key is used to protect files specific to the local machine, such as the Tripwire database. The local key may also be used for signing integrity check reports. You are being prompted for this passphrase because no local key file currently exists. Remember this passphrase; it is not stored anywhere! Enter local key passphrase:
Tripwire has been installed The Tripwire binaries are located in /usr/sbin and the database is located in /var/lib/tripwire. It is strongly advised that these locations be stored on write-protected media (e.g. mounted RO floppy). See /usr/share/doc/tripwire/README.Debian for details.

Initialise the database:
sudo tripwire -m i
Please enter your local passphrase: 
Parsing policy file: /etc/tripwire/tw.pol
Generating the database...
*** Processing Unix File System ***
[..]
### Warning: File system error.
### Filename: /proc/30401/task/30401/fdinfo/4
### No such file or directory
### Continuing...
[..]
The object: "/proc/fs/nfsd" is on a different file system...ignoring.
The object: "/proc/sys/fs/binfmt_misc" is on a different file system...ignoring.
[..]
Wrote database file: /var/lib/tripwire/beryllium.twd
The database was successfully generated.

Checking a single file:
sudo tripwire -m c /bin/ls
Integrity checking objects specified on command line...
Wrote report file: /var/lib/tripwire/report/beryllium-20130227-141840.twr


Open Source Tripwire(R) 2.4.2.2 Integrity Check Report

Report generated by:          root
Report created on:            Wed Feb 27 14:18:40 2013
Database last updated on:     Never

===============================================================================
Report Summary:
===============================================================================

Host name:                    beryllium
Host IP address:              192.168.1.1
Host ID:                      None
Policy file used:             /etc/tripwire/tw.pol
Configuration file used:      /etc/tripwire/tw.cfg
Database file used:           /var/lib/tripwire/beryllium.twd
Command line used:            tripwire -m c /bin/ls 

===============================================================================
Rule Summary: 
===============================================================================

-------------------------------------------------------------------------------
  Section: Unix File System
-------------------------------------------------------------------------------

  Rule Name                       Severity Level    Added    Removed  Modified 
  ---------                       --------------    -----    -------  -------- 
  Root file-system executables    100               0        0        0        
  (/bin)

Total objects scanned:  1
Total violations found:  0

===============================================================================
Object Summary: 
===============================================================================

-------------------------------------------------------------------------------
# Section: Unix File System
-------------------------------------------------------------------------------

No violations.

===============================================================================
Error Report: 
===============================================================================

No Errors

-------------------------------------------------------------------------------
*** End of report ***

Open Source Tripwire 2.4 Portions copyright 2000 Tripwire, Inc.
Integrity check complete.

As a cronjob:
You can also just run
sudo tripwire -m c
in which case your system will be checked.

I would guess that putting this as a cronjob to run once per week would be the intended way of using tripwire. Edit /etc/crontab and put
0 22 * * 7 root tripwire -m c

Update: Tripwire on debian puts a file in /etc/cron.daily

26 February 2013

345. Replacing gEdit with Kate

For some reason I can't get gEdit to auto-spellcheck as I type anymore. There's simply no option to do it. Funnily enough, my favourite editor vim does it without any issues -- but while I like vim and use it for most of my data processing, I prefer to edit e.g. html in gEdit. Habit, I suppose.

Anyway, while gEdit is great for everything else, it's given me an excuse to familiarise myself a bit more with Kate which I take is the KDE counterpart to gEdit. A counterpart that's on steroids like a lot of KDE applications (that's not necessarily a good thing -- see e.g. vim vs emacs -- but each to their own)

Another cool thing with kate is that you can run it in a vim compatibility mode. (Sure, there's also gVim for the real deal, but gVIM looks ugly on my computer and I'm a shallow person.)

1. Installation
sudo apt-get install kate

2. Associations
Edit ~/.local/share/applications/defaults.list
 text/plain=kate.desktop
 text/x-python=kate.desktop
 text/x-sql=kate.desktop
 text/html=kate.desktop
 text/dat=kate.desktop
 text/xml=kate.desktop

You can associate kate with as many formats are you want this way.

Create ~/.local/share/applications/kate.desktop
[Desktop Entry]
Name=kate
Comment=kate text editor
Exec=/usr/bin/kate
Type=Application
Categories=Office;Accessories;
Set dynamic word wrap, enable vim, override

3. Desktop
In gnome, either navigate through your menu to 'Main Menu' (under 'system tools') or, if you're running a stock gnome 3 without a menu, go to the overview mode (what you end up with when you put the mouse in the top left corner) and type in main menu.

Either way, go to accessories and tick the box next to Kate.

4. Set up kate
Start kate, go to settings, configure kate and set your defaults:
Enable vi mode

Set up automatic spell check

Activate Dynamic Word Wrap

And enjoy:



24 February 2013

344. Compile a kernel (3.8) without using kpkg

Note: There is a much better and easier way: http://verahill.blogspot.com.au/2013/02/342-compiling-kernel-38-on-debian.html . What follows is just done for the sake of exploration.

Post begins:
It is incredibly easy to compile your own kernel on Debian using kernel-package. See e.g. http://verahill.blogspot.com.au/2013/02/342-compiling-kernel-38-on-debian.html

In the interest of learning how to compile a kernel in a more generic way which is applicable to non-debian systems (arch, red hat etc.), here's a method which doesn't rely on kpkg (kernel-package).

The downside is that this method does not produce a set of .deb files, and that you can't uninstall it using apt.

I'll be brief since most things are covered e.g. here.  I looked at this post when writing this.

sudo apt-get install build-essential ncurses-bin
mkdir ~/tmp
cd ~/tmp
wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.8.tar.bz2
tar xvf linux-3.8.tar.bz2
cd linux-3.8/
cat /boot/config-`uname -r`>.config
make oldconfig

Once you're done answering the questions, do

time make -j3

This takes 29 minutes (AMD Athlon II X3).

time make -j3 modules

This takes 4 minutes. Finish by running

sudo make modules_install
sudo make headers_install INSTALL_HDR_PATH=/usr/src/linux-3.8.0
sudo make install

Note that the default path for the headers is /usr/include, which doesn't play well with most programs that check for the presence of headers before installation, so use INSTALL_HDR_PATH to specify the destination (at least on debian).

make install takes care of initramfs and grub-update as well and generates
/boot/config-3.8.0
/boot/initrd.img-3.8.0
/boot/System.map-3.8.0
/boot/vmlinuz-3.8.0
And that's really it -- compiling a kernel even without kernel-packages is pretty easy. Reboot and everything should be in working order.

23 February 2013

343. 'track changes' in LaTeX: collaborative writing

While I find latex superior to word processors in almost every single aspect, there is one thing I have to admit that MS/Libre/Open Office have over LaTeX: track changes. Since LaTeX is nothing but code you can obviously use something like git or svn to manage versions of .tex code. There are some differences though -- e.g. git would show you the diff output, but not necessarily the context of the changes, and I'm having a difficult enough time convincing people to use LaTeX without having to worry about setting up a git server as well (I can obviously cheat around it by running a local git server, downloading their file and push it to my git server etc, but then I would be the only one who'd be able to track the changes. Also, it's not elegant).

I have a student who is currently preparing a thesis, and while I've managed to give most of my feedback via the pdf annotation function in mendeley, it doesn't feel natural when it should be a simple matter of editing the .tex directly (yet in a way that leaves the student in control over whether to accept or reject the changes).

So here's an exploration of three different ways of passing comments and corrections back and forth when using LaTeX.

I'm also not the first one to ask this question. See e.g. http://tex.stackexchange.com/questions/3653/what-is-the-best-way-to-track-changes-with-non-computer-people

1. Make a pdf and annotate it
 This is a simple and straightforward method for passing comments back and forth. It is not a very convenient method for making any substantial changes though. On top of that, linux does not currently have any particularly useful program for annotating pdf files. See here: http://verahill.blogspot.com.au/2013/02/338-annotating-pdfs-in-linux-revisited.html for more information.

Quite apart from the lack of tool, I really want to emphasize that making small annotations is not a replacement for a tool that allows you to really stir up the text.

2. Using git
You can easily set up your own git server and use that for versioning. http://verahill.blogspot.com.au/2013/01/324-setting-up-private-git-server.html

You can see the version differences using a couple of methods, but gitk might be the easiest one. The downside with this is that you lose some of the context of the changes. Also, it shows you the code that was changed, the changes in the final document (e.g. formatting, figures etc.).

Still it's a pretty intuitive and easy way of tracking changes -- and in terms of versioning it is certainly the best solution. You can fork, roll back and generally have a perfect account of how the document has evolved.
git in particular makes it easy to see added code

In terms of small changes like fixing typos it is a lot more subtle though




3. latexdiff
latexdiff is an easy to use tool that's in the debian repos. Simply do
latexdiff version1.tex version2.tex > diff.tex

and then open diff.tex and compile it to see the changes.

Pleasantly suprised




4. Other solutions
there are plenty of solutions for introducing mark-up directly into the tex document, like trackchanges, changes etc. I've used them in the past but it lack elegance when doing heavier collaborative writing.

Thoughts

I can't help but think that the best solution is to combine git with latexdiff -- or even a special fork of git that uses latexdiff.

Googling shows that apparently I"m not the only one who thinks so: http://gitorious.org/git-latexdiff#more as well as http://tex.stackexchange.com/questions/1325/using-latexdiff-with-git
I haven't actually tested any of those methods yet, and it still more or less requires a common git server.

20 February 2013

342. Compiling Kernel 3.8 on Debian Testing/Wheezy

NOTE: It seems like series 3.8 has issues with intel (i915) graphics -- it occasionally generates kworker threads that causes unresponsiveness as seen by slow mouse and keyboard response when e.g. plugging or unplugging mains power. No issues on e.g. nvidia though.
http://verahill.blogspot.com.au/2013/03/368-slow-mouse-and-keyboard-triggered.html
http://forums.gentoo.org/viewtopic-p-7278760.html
https://bbs.archlinux.org/viewtopic.php?pid=1248190

Post:

Kernel 3.8 is out now. Not much to say -- the compilation works well using the standard method. The compressed kernel is about 81 Mb to download.

The approach below shows how to compile the kernel on Debian. If you're interested in a more generic approach, see this post: http://verahill.blogspot.com.au/2013/02/344-compile-kernel-38-without-using-kpkg.html

NOTE: kernel 3.8 -- in contrast to the 3.7 series -- now compiles fine on AMD FX 8150.

NOTE: kernel 3.8 plays well with nvidia dkms

Here we go:
sudo apt-get install kernel-package fakeroot build-essential ncurses-dev
mkdir ~/tmp
cd ~/tmp
wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.8.tar.bz2
tar xvf linux-3.8.tar.bz2
cd linux-3.8/
cat /boot/config-`uname -r`>.config
make oldconfig

You will be asked a lot of questions -- how many depends on what version you upgrade from. If in doubt, pick the default answer (i.e. hit enter). If really in doubt, use google.

Then continue:
make-kpkg clean

Do
make menuconfig

if you want to make any specific changes to the kernel (e.g. add support for certain devices)

Then continue:
time fakeroot make-kpkg -j4 --initrd kernel_image kernel_headers

As usual 4 is the number of threads you wish to launch -- make it equal to the number of cores that you have for optimum performance during compilation (more about that here).

The build takes around 20 minutes on a four-core intel i5-2400 with -j4, and 14 minutes on an fx-8150 with -j8 (96 minutes with -j1).

Install:
sudo dpkg -i ../linux-image-3.8.0_3.8.0-10.00.Custom_amd64.deb ../linux-headers-3.8.0_3.8.0-10.00.Custom_amd64.deb

New stuff/Questions:


Offload RCU callback processing from boot-selected CPUs (RCU_NOCB_CPU) [N/y/?] (NEW) *
Memory placement aware NUMA scheduler (NUMA_BALANCING) [N/y/?] (NEW) *
Enable to assign a node which has only movable memory (MOVABLE_NODE) [N/y/?] (NEW)
Allow for memory hot-add (MEMORY_HOTPLUG) [Y/n] y
Allow for balloon memory compaction/migration (BALLOON_COMPACTION) [Y/n/?] (NEW)
Set default setting of cpu0_hotpluggable (BOOTPARAM_HOTPLUG_CPU0) [N/y/?] (NEW
Debug CPU0 hotplug (DEBUG_HOTPLUG_CPU0) [N/y/?] (NEW)
ACPI tables can be passed via uncompressed cpio in initrd (ACPI_INITRD_TABLE_OVERRIDE) [N/y/?] (NEW)
Support multiple cpuidle drivers (CPU_IDLE_MULTIPLE_DRIVERS) [N/y/?] (NEW)
"NOTRACK" target support (DEPRECATED) (NETFILTER_XT_TARGET_NOTRACK) [N/m] (NEW

Default SCTP cookie HMAC encoding
  > 1. Enable optional MD5 hmac cookie generation (SCTP_DEFAULT_COOKIE_HMAC_MD5) (NEW)
    2. Enable optional SHA1 hmac cookie generation (SCTP_DEFAULT_COOKIE_HMAC_SHA1) (NEW)
    3. Use no hmac alg in SCTP cookie generation (SCTP_DEFAULT_COOKIE_HMAC_NONE) (NEW)
  choice[1-3?]:   Enable optional MD5 hmac cookie generation (SCTP_COOKIE_HMAC_MD5) [Y/?] (NEW) y

Enable optional SHA1 hmac cookie generation (SCTP_COOKIE_HMAC_SHA1) [N/y/?] (NEW) *
Enable optional MD5 hmac cookie generation (SCTP_COOKIE_HMAC_MD5) [Y/?] (NEW) y
Distributed ARP Table (BATMAN_ADV_DAT) [N/y/?] (NEW)
Kvaser CAN/USB interface (CAN_KVASER_USB) [N/m/?] (NEW)
LSI MPT Fusion SAS 3.0 Device Driver (SCSI_MPT3SAS) [N/m/?] (NEW)
Chelsio Communications FCoE support (SCSI_CHELSIO_FCOE) [N/m/?] (NEW) *
Marvell 88E6060 ethernet switch chip support (NET_DSA_MV88E6060) [N/m/y/?] (NEW)
Marvell 88E6085/6095/6095F/6131 ethernet switch chip support (NET_DSA_MV88E6131) [N/m/y/?] (NEW)
Marvell 88E6123/6161/6165 ethernet switch chip support (NET_DSA_MV88E6123_61_65) [N/m/y/?] (NEW) *
Cadence devices (NET_CADENCE) [Y/n/?] (NEW)
AT91RM9200 Ethernet support (ARM_AT91_ETHER) [N/m/y/?] (NEW)
Cadence MACB/GEM support (MACB) [N/m/y/?] (NEW)
Broadcom devices (NET_VENDOR_BROADCOM) [Y/?] y
Marvell MDIO interface support (MVMDIO) [N/m/y/?] (NEW)
CDC MBIM support (USB_NET_CDC_MBIM) [N/m/?] (NEW)
Atheros Wireless Cards (ATH_CARDS) [N/m/?] (NEW)
Atheros AR5523 wireless driver support (AR5523) [N/m/?] (NEW)
Wilocity 60g WiFi card wil6210 support (WIL6210) [N/m/?] (NEW) *
Realtek RTL8723AE PCIe Wireless Network Adapter (RTL8723AE) [N/m/?] (NEW)
ARC UART driver support (SERIAL_ARC) [N/m/y/?] (NEW) *
CBUS I2C driver (I2C_CBUS_GPIO) [N/m/?] (NEW)
TS-5500 DIO blocks and compatibles (GPIO_TS5500) [N/m/y/?] (NEW) 
TI BQ2415x battery charger driver (CHARGER_BQ2415X) [N/m/?] (NEW)
Board level reset or power off (POWER_RESET) [N/y/?] (NEW) *

 Default Thermal governor
  > 1. step_wise (THERMAL_DEFAULT_GOV_STEP_WISE) (NEW)
    2. fair_share (THERMAL_DEFAULT_GOV_FAIR_SHARE) (NEW)
    3. user_space (THERMAL_DEFAULT_GOV_USER_SPACE) (NEW)
  choice[1-3?]:   Fair-share thermal governor (FAIR_SHARE) [N/y/?] (NEW)

Step_wise thermal governor (STEP_WISE) [Y/?] (NEW) y
User_space thermal governor (USER_SPACE) [N/y/?] (NEW)
SSB GPIO driver (SSB_DRIVER_GPIO) [N/y/?] (NEW) *
BCMA GPIO driver (BCMA_DRIVER_GPIO) [N/y/?] (NEW)
Support for Realtek PCI-E card reader (MFD_RTSX_PCI) [N/m/y/?] (NEW)
TI ADC / Touch Screen chip support (MFD_TI_AM335X_TSCADC) [N/m/y/?] (NEW)
Support for Nano River Technologies Viperboard (MFD_VIPERBOARD) [N/m/?] (NEW)
Support for Retu multi-function device (MFD_RETU) [N/m/?] (NEW) *
Maxim MAX8973 voltage regulator  (REGULATOR_MAX8973) [N/m/?] (NEW)
TI TPS51632 Power Regulator (REGULATOR_TPS51632) [N/m/?] (NEW)
Siano SMS1xxx based MDTV receiver (SMS_USB_DRV) [N/m/?] (NEW)
Siano SMS1xxx based MDTV via SDIO interface (SMS_SDIO_DRV) [N/m/?] (NEW)   *
Stanton Control System 1 MIDI (SND_SCS1X) [N/m/?] (NEW) *
ION iCade arcade controller (HID_ICADE) [N/m/?] (NEW)
HID over I2C transport layer (I2C_HID) [N/m/?] (NEW) *
Renesas R-Car USB phy support (USB_RCAR_PHY) [N/m/?] (NEW)   *
SDHCI support for ACPI enumerated SDHCI controllers (MMC_SDHCI_ACPI) [N/m/?] (NEW)
NXP PCF8523 (RTC_DRV_PCF8523) [N/m/?] (NEW)
Philips PCF8563/Epson RTC8564 (RTC_DRV_PCF8563) [M/n/?] m
Userspace platform driver with generic irq and dynamic memory (UIO_DMEM_GENIRQ) [N/m/?] (NEW)
Microsoft Hyper-V Balloon driver (HYPERV_BALLOON) [N/m/?] (NEW) *
SystemBase PCI Multiport UART (SB105X) [N/m/y/?] (NEW)
TTY over Firewire (FIREWIRE_SERIAL) [N/m/?] (NEW) *
F2FS filesystem support (EXPERIMENTAL) (F2FS_FS) [N/m/y/?] (NEW) *
Enable CIFS debugging routines (CIFS_DEBUG) [Y/n/?] (NEW)
Simplified Mandatory Access Control Kernel Support (SECURITY_SMACK) [N/y/?] (NEW)
Camellia cipher algorithm (x86_64/AES-NI/AVX) (CRYPTO_CAMELLIA_AESNI_AVX_X86_64) [N/m/y/?] (NEW)



Links to this post:
http://forum.notebookreview.com/lenovo/575569-linux-x220-41.html
http://www.reddit.com/r/CrunchBang/comments/196zx5/whats_the_best_way_to_install_a_new_kernel_in/
http://www.lubuntu.ru/forum/viewtopic.php?t=1125
http://forums.linuxmint.com/viewtopic.php?f=143&p=711457
https://www.linux.org.ru/forum/linux-install/9128362
http://stackoverflow.com/questions/16573913/compile-new-kernel-on-debian-wheezy
http://crunchbang.org/forums/viewtopic.php?id=26825
http://mygoodluck.org/debian-obnovil-yadro-do-38

19 February 2013

341. Upgrading/installing BankID on 64 bit linux

Note: the post below is aimed at installing BankID on Debian (should be ok for ubuntu/mint too). For Arch Linux, see here (item 16)

There are a few ways to get around the rotten behaviour of bankid. This is one of them:

NOTE: to install nspluginwrapper you need to enable the stable/squeeze repos by e.g. adding
deb http://ftp.au.debian.org/debian/ squeeze main contrib non-free
to your /etc/apt/sources.list.

That's normally reasonably safe since apt by default pulls in the newest package and I haven't had any issues. Just be careful though.

You can also install nspluginwrapper by compiling it as shown here: http://verahill.blogspot.com.au/2013/03/366-nspluginwrapper-on-debian.html


Note that there's a FOSS alternative in Fribid (http://verahill.blogspot.se/2012/02/debian-testing-wheezy-64-fribid-as.html) which seems to be working perfectly -- and if you can use it, use it. The main limitation is that in practice you'll have to collect your certificate/ID with it, since newer versions of BankID saves the ID in an incompatible format. Like many foreigners, I don't have the opportunity to visit Sweden for the sole sake of picking up a new ID, so I'm stuck with BankID. But you may not be.



0. Things to install:
sudo apt-get install iceweasel nspluginwrapper ia32-libs

1. Download BankID and uninstall any previous installations
cd ~/Downloads
mkdir bankid
cd bankid
wget https://install.bankid.com/Download?defaultFileId=Linux -O bankid.tar.gz
tar xvf bankid.tar.gz
cd BISP-4.19.1.11663/
sudo sh install.4.19.1.11663.sh u

2. If you're upgrading, make sure to remove any previous libplugins.so
sudo updatedb && locate libplugins.so
/home/me/Downloads/bankid/BISP-4.19.1.11663/libplugins.so /usr/lib/mozilla/plugins/libplugins.so /usr/lib/mozilla/plugins/npwrapper.libplugins.so /usr/lib/nspluginwrapper/plugins/npwrapper.libplugins.so
sudo nspluginwrapper -r /usr/lib/mozilla/plugins/npwrapper.libplugins.so sudo rm /usr/lib/mozilla/plugins/libplugins.so sudo rm /usr/lib/nspluginwrapper/plugins/npwrapper.libplugins.so

3. Install the new version
sudo sh install.4.19.1.11663.sh i
Installing BankID Security Application ln: failed to create symbolic link `/usr/lib/firefox-addons/plugins': No such file or directory WARNING: Failed installing plugin for Firefox 3. Manually add symlink to libplugins.so in your Firefox 3 plugin directory if this browser is to be used. Installation complete.
sudo nspluginwrapper --install /usr/local/lib/personal/libplugins.so

4. Test your installation
Don't bother with test.bankid.com since the idiots won't let you test anything that identifies itself as 64 bit (more about that later). Instead
5. Testing against test.bankid.com
Everything is in working order but for some idiotic reason bankid.com won't even allow you to test you fancy new 64 bit installation -- and it all boils down the useragent string in iceweasel/firefox identifying itself as running on a 64 bit system (paradoxically, a real 32 bit browser running in a chroot won't work either since the kernel is 64 bit -- in schroot you can use personality=linux32 to get around it, but good luck dealing with the massive memory leaks).

Anyway,
  • open your iceweasel browser
  • type in about:config in the address bar
  • promise that you'll be careful
  • right-click on the page, select New, String
  • In the first box, type general.useragent.override
  • In the second box paste Mozilla/5.0 (X11; Linux i686; rv:10.0.12) Gecko/20100101 Firefox/10.0.12 Iceweasel/10.0.12
You can now go to https://test.bankid.com. I find this a bit humiliating though, and you can use bankid everywhere but bankid.com without having to set the useragent to identify your system as being i686.


If you're having issues, the first thing to check is about:plugins in firefox:


18 February 2013

340. Issues when compiling Nwchem 6.1.1 -- missing *.fh in src/include

The problem:
If you modify the nwchem sources, build, do a make realclean and then pack them up to export your patched sources to other nodes you might find that compiling doesn't work that well, yielding errors such as (I'll list them all to help google):

make[1]: *** No rule to make target `/opt/nwchem/nwchem-6.1.1-src_mod/src/include/stdio.fh', needed by `/opt/nwchem/nwchem-6.1.1-src_mod/lib/LINUX64/libnwcutil.a(basis.o)'.  Stop.
make: *** [libraries] Error 1

and

make[1]: *** No rule to make target `/opt/nwchem/nwchem-6.1.1-src_mod/src/include/nwc_const.fh', needed by `/opt/nwchem/nwchem-6.1.1-src_mod/lib/LINUX64/libnwcutil.a(basis.o)'.  Stop.
make: *** [libraries] Error 1
nwchem.F:3:0: fatal error: errquit.fh: No such file or directory
compilation terminated.
stubs.F:15:0: fatal error: errquit.fh: No such file or directory

and

make[1]: *** No rule to make target `/opt/nwchem/nwchem-6.1.1-src_mod/src/include/errquit.fh', needed by `/opt/nwchem/nwchem-6.1.1-src_mod/lib/LINUX64/libnwcutil.a(basis.o)'.  Stop.
make: *** [libraries] Error 1

and

make[1]: *** No rule to make target `/opt/nwchem/nwchem-6.1.1-src_mod/src/include/util.fh', needed by `/opt/nwchem/nwchem-6.1.1-src_mod/lib/LINUX64/libnwcutil.a(bas_input.o)'.  Stop.

and

/opt/nwchem/nwchem-6.1.1-src_mod/src/include/util.fh:1:0: fatal error: printlevels.fh: No such file or directory
compilation terminated.
make[1]: *** [/opt/nwchem/nwchem-6.1.1-src_mod/lib/LINUX64/libnwcutil.a(bas_input.o)] Error 1
make[1]: *** Waiting for unfinished jobs....

and

basisP.F: In function 'nbf_from_ucont':
basisP.F:427:0: warning: '__result_nbf_from_ucont' may be used uninitialized in this function [-Wmaybe-uninitialized]
make[2]: warning: -jN forced in submake: disabling jobserver mode.
make[1]: warning: -jN forced in submake: disabling jobserver mode.
make[1]: *** No rule to make target `/opt/nwchem/nwchem-6.1.1-src_mod/src/include/msgids.fh', needed by `/opt/nwchem/nwchem-6.1.1-src_mod/lib/LINUX64/libnwcutil.a(geom.o)'.  Stop.

and

make[1]: *** No rule to make target `/opt/nwchem/nwchem-6.1.1-src_mod/src/include/bitops.fh', needed by `/opt/nwchem/nwchem-6.1.1-src_mod/lib/LINUX64/libnwcutil.a(pstat_alloc.o)'.  Stop.

and

In file included from pstat_alloc.F:12:0:
/opt/nwchem/nwchem-6.1.1-src_mod/src/include/bitops.fh:11:0: fatal error: bitops_decls.fh: No such file or directory
compilation terminated.
make[1]: *** [/opt/nwchem/nwchem-6.1.1-src_mod/lib/LINUX64/libnwcutil.a(pstat_alloc.o)] Error 1

and

In file included from pstat_alloc.F:12:0:
/opt/nwchem/nwchem-6.1.1-src_mod/src/include/bitops.fh:12:0: fatal error: bitops_funcs.fh: No such file or directory
compilation terminated.
make[1]: *** [/opt/nwchem/nwchem-6.1.1-src_mod/lib/LINUX64/libnwcutil.a(pstat_alloc.o)] Error 1

and

make[2]: warning: -jN forced in submake: disabling jobserver mode.
make[2]: *** No rule to make target `/opt/nwchem/nwchem-6.1.1-src_mod/src/include/itri.fh', needed by `/opt/nwchem/nwchem-6.1.1-src_mod/lib/LINUX64/libnwcutil.a(sym_sh_pair.o)'.  Stop.
make[2]: *** Waiting for unfinished jobs....

and

make[2]: *** No rule to make target `/opt/nwchem/nwchem-6.1.1-src_mod/src/include/bgj.fh', needed by `/opt/nwchem/nwchem-6.1.1-src_mod/lib/LINUX64/libnwints.a(exactd_mem.o)'.  Stop.

and

make[1]: *** No rule to make target `/opt/nwchem/nwchem-6.1.1-src_mod/src/include/numerical_constants.fh', needed by `/opt/nwchem/nwchem-6.1.1-src_mod/lib/LINUX64/librimp2.a(rimp2_v_e2.o)'.  Stop.

and

make[2]: *** No rule to make target `/opt/nwchem/nwchem-6.1.1-src_mod/src/include/util_sgroup.fh', needed by `/opt/nwchem/nwchem-6.1.1-src_mod/lib/LINUX64/libdntmc.a(gibbs.o)'.  Stop.

and

nwchem.F:11:0: fatal error: bgj_common.fh: No such file or directory

Solution:
This has been mentioned before on the nwchem forum, but not in explicit enough detail.
The solution is to copy a series of files from src/util and to remove make realclean from your build instructions (or at least do the copying after the make realclean step).

Copy these files
cp src/util/stdio.fh src/include/
cp src/util/nwc_const.fh src/include/
cp src/util/errquit.fh src/include/
cp src/util/util.fh src/include/
cp src/util/printlevels.fh src/include/
cp src/util/msgids.fh src/include/
cp src/util/bitops.fh src/include/
cp src/util/bitops_decls.fh src/include/
cp src/util/bitops_funcs.fh src/include/
cp src/util/itri.fh src/include/
cp src/util/bgj.fh src/include/
cp src/util/numerical_constants.fh src/include/
cp src/util/util_sgroup.fh src/include/
cp src/util/bgj_common.fh src/include/

Then build, using e.g.
export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export PYTHONVERSION=2.7
export PYTHONHOME=/usr
export BLASOPT="-L/opt/openblas/lib -lopenblas"
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openmpi/lib:/opt/openblas/lib
export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"
cd $NWCHEM_TOP/src
make nwchem_config
make FC=gfortran 1>make.log 2>make.err
export FC=gfortran
cd ../contrib
./getmem.nwchem

15 February 2013

339. Compiling ncdu on ROCKS 5.4.3/Centos 5.6

du is nice, but ncdu gives a better overview. Nothing odd about building it though:

mkdir ~/tmp
cd ~/tmp
wget http://dev.yorhel.nl/download/ncdu-1.9.tar.gz
tar xvf ncdu-1.9.tar.gz
cd ncdu-1.9/
sudo mkdir /share/apps/tools/ncdu -p
sudo chown $USER /share/apps/tools/ncdu
./configure --prefix=/share/apps/tools/ncdu
make
make install
echo 'export PATH=$PATH:/share/apps/tools/ncdu/bin' >> ~/.bashrc
source ~/.bashrc

Start by running
ncdu

13 February 2013

338. Annotating PDFs in linux -- revisited. Still no obvious solution

Update 28/02/2014: I've had a look at I, Librarian and Master PDF Editor here: http://verahill.blogspot.com.au/2014/02/558-more-options-for-pdf-annotation.html

There are a a few main reasons why even platform agnostic people have trouble moving to linux. One of them is poor compatibility of open solutions with MS Office (incompatibility between MS Equations and libre/openoffice is my main gripe, and some may find the lack of native EndNote an issue), and one quite particular to academics is the lack of proper pdf annotation (traditionally you edit your galley proofs by making annotations on the pdf -- my latest paper from Wiley came back with a doc file though, which was a bit surprising, but promising)

I've gotten by in the past by using 'pdf x-change viewer', which is a windows program, under wine. However, there are a lot of things I don't like about that solution, and the search for a native solution has continued to the point where I'm willing to throw money at it.

Why the exercise is unfair
The problem is that our definition of what 'works' and what doesn't is based on what we expect it to look like -- and that's based on our experience. I have colleagues that consider libreoffice 'crap' because it behaves or looks different from MS Office. While I agree that it's not viable as a replacement to office when collaborating with Office users who refuse to use libreoffice, it works fine if libreoffice is all you use. Anyway, it's unfair. Same goes for pdf annotation -- we expect it to look and function like adobe acrobat, simply because that's what we're used to. Any deviation is akin to a bug.

So keep that in mind when I write off some of the alternatives that actually do work -- just not in the way I expect them to.

I've found a lot of people (online) swearing by e.g.  Xournal, so even if it doesn't work for me, you may decide differently.

Commercial software in academia

I don't normally like buying software which is critical to my work. There are many reasons for it, and cost is only a minor one (although I don't like spending tax payer money on overpriced software).

Instead, if the software is critical to the science that I'm doing, I prefer to write my own algorithms in octave or python, and if it's a bit more peripheral I'm still wary of becoming reliant on a piece of software that may one day disappear -- either because I'm forced to upgrade through planned obsolescence, or because the company goes bankrupt/discontinues the software without releasing it as open source.

I cannot recommend Mendeley. And that's for two reasons:
1. it installs a list file in /etc/apt/sources.list.d/ without asking. That's the way malware (and google...) behaves!
2. There's plenty of mention about how it's 'free', but the free version is very restrictive, and if you create a private group you're stuck with a nagging message saying that you "must" upgrade if you want to add more users/groups. A simple one-time message would suffice.

Maybe I'm overreacting, but I don't appreciate this behaviour at all. Beyond that the annotation function works fine, so you decide for yourself.

Anyway, here's a short list of programs I've considered:
(and FOSS= Free and Open Source Software)



Evince
Evince is FOSS and now (since when?) supports adding annotations. It's not working very well (sometimes doesn't save annotation, slow), and you can't delete annotations, so be careful what you're saving.
Evince


pdfedit
pdfedit is another standard linux package. It does highlighting well, but annotates by adding text on top of the document -- not as a sticky note. FOSS.
pdfedit


flpsed
Does annotation, but as text superimposed on the pdf document, not as a collapsible sticky note. FOSS..

Qoppa PDF Studio
This is java based, and runs on Linux. I must already now say that there's one aspect of it that I really don't like: it's available in a Standard and a Professional version. That's the kind of artificial crippling of software that Microsoft likes to engage in, but I though we were beyond that on Linux...the price, $89, is too steep for something that I'd only use for annotations. Note also that the trial version puts a big nasty watermark over everything -- but you can hardly fault them for that, since it's not free. Closed source. Commercial.
Other than that it works, although it's not as pretty as mendeley.

Qoppa PDF studio


Xournal
Xournal can export annotated PDFs, but it doesn't do annotation in the same way as the other programs i.e. using sticky notes. Instead you can simply add text on top of the pdf, and it doesn't really do it for me. FOSS..
Xournal -- the annotations are not easy to spot


Whyteboard
It draws on top of PDF using imagemagick. FOSS.

Mendeley
While it's meant mainly for reference management, this does proper PDF annotation as well, and is platform agnostic. However, it is closed source, requires you to log in (even if you're using the desktop client) every time you use it, and needs you to explicitly keep documents (at least their titles etc.) out of the shared web catalogue. Other than that the pdf annotation works beautifully. I get really annoyed by the requirement to log in even when working offline though. It's free in the sense of gratis though -- but only up to 100 Mb of shared document space, you can only have one private group, and it can only have two members (+you). If you want more you need to pay (see e.g. here).
In practical terms, it seems to use GMT to time stamp annotationsand I haven't found an obvious way
of changing that (without going online). Also, it installs a file into /etc/apt/sources.list.d/ without asking.
Mendeley


Misc
Other 'solutions' that pop up is Okular which work in a roundabout way -- i.e the annotations aren't stored as part of the pdf. Again, it looks pretty, but the annotations are not exported with the pdf.
UPDATE: Note that this doesn't seem to be an issue anymore -- see comments below and this post: https://groakat.wordpress.com/2013/08/27/annotating-pdf-with-okular/ -- note that the new version is NOT in either wheezy or jessie i.e. they won't work.

okular


I tried FoxIt reader as well which claims to do annotation but doesn't on Linux -- and the windows version is not functional under linux/wine.

PDF X-change viewer/wine pops up so often as a suggested solution that it's beginning to look like spam. It does work though:
PDF Xchange viewer


An online only option is http://www.pdfescape.com -- but the paranoid part of me doesn't like the idea of uploading documents that are meant to be private.

12 February 2013

337. Modifying Nwchem 6.1.1 to work with GabEdit

Karol Strutynski left the following comment on a post about NWChem and Gabedit:

Hello,
I have one important comment:
The vectors coefficients in the nwchem output are incomplete!
The default behaviour of nwchem is to print 10 first coefficients with value bigger than 0.15. For systems with many atoms it is not enough, usually its not even close.

This behaviour is hard-coded in the nwchem source.
To change this you must search each instance of movecs_print_anal in the source code and replace 0.15d0 for smaller value in appropriate calls.
Furthermore you must change one loop in the src/ddscf/movecs_pr_anal.F file and around 200 line there will be loop:
do klo = 0, min(n-1,9), 2
You must increase the range of this loop, for something more reasonable like:
do klo = 0, min(n-1,199), 2

After recompiling the nwchem will print more coefficients and the gabedit will produce more reliable orbitals.

Best regards,
Karol Strutynski

So let's modify NWChem. I'll be modifying the 27th of June release of NWChem 6.1.1, which you'll obtain as Nwchem-6.1.1-src.2012-06-27.tar.gz from http://www.nwchem-sw.org/index.php/Download.


Change the number in red to something smaller (I tried 0.01d0) in the following files:
 /src/ddscf/uhf.F
 146  9611    continue
 147          call movecs_print_anal(basis, ilo, ihi, 0.15d0, g_movecs,
 148      $        'UHF Final Alpha Molecular Orbital Analysis',
 149      $        .true., dbl_mb(k_eval), oadapt, int_mb(k_irs),
 150      $        .true., dbl_mb(k_occ))
 151          call movecs_print_anal(basis, ilo, ihi, 0.15d0, g_movecs(2),
 152      $        'UHF Final Beta Molecular Orbital Analysis',
 153      $        .true., dbl_mb(k_eval+nbf), oadapt, int_mb(k_irs+nmo),
 154      $        .true., dbl_mb(k_occ+nbf)

/src/ddscf/scf_vec_guess.F
506          if (scftype.eq.'RHF' .or. scftype.eq.'ROHF') then
507             call movecs_print_anal(basis, 1,
508      &           nprint, 0.15d0, g_movecs,
509      &           'ROHF Initial Molecular Orbital Analysis',
510      &           .true., dbl_mb(k_eval), oadapt, int_mb(k_irs),
511      &           .true., dbl_mb(k_occ))
512          else
513             nprint = min(nalpha+20,nmo)
514             call movecs_print_anal(basis, max(1,nbeta-20),
515      &           nprint, 0.15d0, g_movecs,
516      &           'UHF Initial Alpha Molecular Orbital Analysis',
517      &           .true., dbl_mb(k_eval), oadapt, int_mb(k_irs),
518      &           .true., dbl_mb(k_occ))
519             call movecs_print_anal(basis, max(1,nbeta-20),
520      &           nprint, 0.15d0, g_movecs(2),
521      &           'UHF Initial Beta Molecular Orbital Analysis',
522      &           .true., dbl_mb(k_eval+nbf), oadapt, int_mb(k_irs+nmo),
523      &           .true., dbl_mb(k_occ+nbf))

/src/ddscf/rohf.F
155          endif
156          call movecs_print_anal(basis, ilo, ihi, 0.15d0, g_movecs,
157      $        'ROHF Final Molecular Orbital Analysis',
158      $        .true., dbl_mb(k_eval), oadapt, int_mb(k_irs),
159      $        .true., dbl_mb(k_occ))

/src/mcscf/mcscf.F
680       if (util_print('final vectors analysis', print_default))
681      $     call movecs_print_anal(basis,
682      $     max(1,nclosed-10), min(nbf,nclosed+nact+10),
683      $     0.15d0, g_movecs, 'Analysis of MCSCF natural orbitals',
684      $     .true., dbl_mb(k_evals), .true., int_mb(k_sym),
685      $     .true., dbl_mb(k_occ))
686 c

/src/nwdft/scf_dft_cg/dft_cg_solve.F
166           call movecs_fix_phase(g_movecs(ispin))
167           call movecs_print_anal(basis, ilo, ihi, 0.15d0,
168      &         g_movecs(ispin),blob,
169      &         .true., dbl_mb(k_eval+(ispin-1)*nbf),
170      &         oadapt, int_mb(k_irs+(ispin-1)*nbf),
171      &         .true., dbl_mb(k_occ+(ispin-1)*nbf))
172         enddo

/src/nwdft/scf_dft/dft_scf.F
1736             call movecs_print_anal(ao_bas_han, ilo, ihi, 0.15d0,
1737      &           g_movecs(ispin),
1738      &           blob,
1739      &           .true., dbl_mb(k_eval(ispin)), oadapt,
1740      &           int_mb(k_ir+(ispin-1)*nbf_ao),
1741      &           .true., dbl_mb(k_occ+(ispin-1)*nbf_ao))

/src/nwdft/scf_dft/dft_mxspin_ovlp.F
186       call movecs_print_anal(basis,int_mb(k_non),int_mb(k_non)
187      & ,0.15d0,g_alpha,'Alpha Orbitals without Beta Partners',
188      &   .false., 0.0 ,.false., 0 , .false., 0 )
189 c
190       if (nct.GE.2) then
191       do i = 2,nct
192       ind = int_mb(k_non+i-1)
193       call movecs_print_anal(basis,ind,ind
194      & ,0.15d0,g_alpha,' ',
195      &   .false., 0.0 ,.false., 0 , .false., 0 )
196       enddo
197       endif

352 c
353        call movecs_print_anal(basis, 1, nalp, 0.15d0, g_ualpha,
354      & 'Alpha Orb. w/o Beta Partners (after maxim. alpha/beta overlap)',
355      &   .false., 0.0 ,.false., 0 , .false., 0 )
356 c
Otherwise once could presumably edit the header in ./src/ddscf/movecs_pr_anal.F directly and substitute thresh. At a minimum you should edit that file according to Karol's instructions: change the number in red below to e.g. 199.

/src/ddscf/movecs_pr_anal.F
198             do klo = 0, min(n-1,9), 2
199                khi = min(klo+1,n-1)
200                write(LuOut,2) (
201      $              int_mb(k_list+k)+1,
202      $              dbl_mb(k_vecs+int_mb(k_list+k)),
203      $              (byte_mb(k_tags+int_mb(k_list+k)*16+m),m=0,15),
204      $              k = klo,khi)
205  2             format(1x,2(i5,2x,f12.6,2x,16a1,4x))
206             enddo

Compilation
At this point you should be able to follow post 242. Briefly: Compiling NWChem 6.1.1 with Python on Debian Testing (Wheezy) and compile nwchem with python etc. Don't forget to edit /src/config/makefile.h for python support as shown in that post. Once you're done with that you can compare the GabEdit plots with and without the modification.

Alternatively, if you're simply making changes to a copy of nwchem that you've compiled before, you can speed thing up by a factor of ca 300 by following this post:
http://verahill.blogspot.com.au/2013/04/380-modifying-nwchem-code-without-full.html



The difference:
I ran a job on benzene as described in post 281. Visualising NWChem output with GabEdit. I chose to run use the ELF (electron localisation function) on output from the unmodified and modified nwchem binaries. It's a pretty big difference:

Original

Modified

08 February 2013

336. Compiling ATLAS, netblas, lapack and openblas on Arch Linux

Here's another Arch post.

I was a bit surprised to find that there's no ATLAS in the standard Arch repositories (it is in AUR though), so here's how to build some of the more common math libraries for yourself:


ATLAS

pacman -S wget base-devel gcc-fortran cpupower
sudo systemctl enable cpupower

To build ATLAS you should set the governor for your CPU to performance to get the best optimization:

cpupower frequency-set -g performance
sudo cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor

Basically copy the scaling_governor to all cpus (cpu0, cpu1, cpu2 ...) as shown in the last line above. When you set the governor back to e.g. ondemand, follow the same steps.

sudo mkdir /opt/ATLAS
chown ${USER} /opt/ATLAS
mkdir -p ~/tmp/atlas
cd ~/tmp/atlas
wget http://www.netlib.org/lapack/lapack-3.4.2.tgz
wget http://downloads.sourceforge.net/project/math-atlas/Stable/3.10.1/atlas3.10.1.tar.bz2
tar xvf atlas3.10.1.tar.bz2
mkdir build/
cd build/
../ATLAS/./configure --prefix=/opt/ATLAS -Fa alg '-fPIC' --with-netlib-lapack-tarfile=$HOME/tmp/atlas/lapack-3.4.2.tgz --shared
make
make install

Simple as that. You can now change the governor back
cpupower frequency-set -g ondemand
sudo cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
sudo cp /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
...


netlib BLAS and lapack
pacman -S wget base-devel gcc-fortran cmake
sudo mkdir /opt/netlib
sudo chown $USER /opt/netlib
mkdir /opt/netlib/blas/lib -p
mkdir -p ~/tmp/blas
cd ~/tmp/blas
wget http://www.netlib.org/blas/blas.tgz
tar xvf blas.tgz
cd BLAS/

Edit make.inc
OPTS = -O3 -shared -m64 -march=native -fPIC
make all
gfortran -shared -Wl,-soname,libnetblas.so -o libblas.so.1.0.1 *.o -lc
ln -s libblas.so.1.0.1 libnetblas.so
cp lib*blas* /opt/netlib/blas/lib
cd ../
wget http://www.netlib.org/lapack/lapack-3.4.2.tgz
tar xvf lapack-3.4.2.tgz
mkdir /opt/netlib/lapack
mkdir build/
cd build/
ccmake ../lapack-3.4.2/ -DCMAKE_INSTALL_PREFIX=/opt/netlib/lapack -DBUILD_SHARED_LIBS=ON -DUSE_OPTIMIZED_BLAS=ON 

Hit c twice to configure, then g to generate.
Edit CMakeCache.txt and add the following lines at the beginning:
########################
# EXTERNAL cache entries
########################
 BLAS_FOUND:STRING=TRUE
 BLAS_GENERIC_FOUND:BOOL=TRUE
 BLAS_GENERIC_blas_LIBRARY:FILEPATH=/opt/netlib/blas/lib/libnetblas.so
 BLAS_LIBRARIES:PATH=/opt/netlib/blas/lib/libnetblas.so

Do
ccmake ../lapack-3.4.2/

again, then hit c once, then g.
Next,
make
make install

Done.

Openblas
Copied from here: http://verahill.blogspot.com.au/2013/02/334-compiling-nwchem-with-openmpi-and.html

Download from http://github.com/xianyi/OpenBLAS/tarball/v0.1.1

pacman -S wget base-devel gcc-fortran
sudo mkdir /opt/openblas
sudo chown $USER /opt/openblas
tar xvf xianyi-OpenBLAS-v0.1.1-0-g5b7f443.tar.gz
cd xianyi-OpenBLAS-e6e87a2/
make all BINARY=64 CC=/usr/bin/gcc FC=/usr/bin/gfortran USE_THREAD=0 INTERFACE64=1 1> make.log 2>make.err
make PREFIX=/opt/openblas install
cp lib*.*  /opt/openblas/lib


07 February 2013

335. Compiling sinfo in Arch linux

Note: I'm not abandoning Debian. I'm just going on a roadtrip with Arch to see whether we can all happily co-exist in the same machine park.

sinfo is available in AUR, but it's broken and won't build. Luckily it's very easy to build it yourself.

(see here for debian and here for centos)

mkdir ~/tmp
cd ~/tmp
sudo pacman -S wget base-devel boost asio
wget http://www.ant.uni-bremen.de/whomes/rinas/sinfo/download/sinfo-0.0.47.tar.gz
tar xvf sinfo-0.0.47.tar.gz
cd sinfo-0.0.47/
sed -i 's/\-mt//g' sinfo/Makefile.*
sed -i 's/\-mt//g' sinfod/Makefile.*
./configure
make
sudo make install


Not sure about a suitable systemd script (I'll update somewhere down the line). To start sinfod manually do
sudo sinfod --quiet 

if you have multiple ifaces you can set the --bcastaddress.

To start sinfo itself, well, run sinfo
sinfo

Not much fun with only one node...

334. Compiling nwchem with openmpi and python on Arch linux

Here's the reason why I gave my virtual machine 30 Gb in post 333 -- to be able to evaluate whether I can figure out how to build all the software that I need on Arch.

Behold my surprise when I realised that there's no need for separate -dev packages, as is the case on Debian i.e. the headers are generally installed together with the package (so e.g. python is enough -- you don't need python-dev as well).

While debian is probably the best choice for my nodes (I want stability -- not the latest flashiest stuff), nwchem is a good test case since I've been playing with it for years, and it's not available in the pacman or AUR repos.

Your mileage with openblas will vary depending on your hardware. ACML is an alternative on e.g. FX-8150. ATLAS doesn't seem to work with NWChem when I try it, but I'm not sure what I'm doing wrong. See the original post for examples on how to link to other math libs.

I'm mainly looking at this post: http://verahill.blogspot.com.au/2012/09/briefly-compiling-nwchem-611-with.html


Dependencies:
pacman -S wget base-devel gcc-fortran tcsh openmpi

Openblas:
Download from http://github.com/xianyi/OpenBLAS/tarball/v0.1.1

sudo mkdir /opt/openblas
sudo chown $USER /opt/openblas
tar xvf xianyi-OpenBLAS-v0.1.1-0-g5b7f443.tar.gz
cd xianyi-OpenBLAS-e6e87a2/
make all BINARY=64 CC=/usr/bin/gcc FC=/usr/bin/gfortran USE_THREAD=0 INTERFACE64=1 1> make.log 2>make.err
make PREFIX=/opt/openblas install
cp lib*.*  /opt/openblas/lib

Nwchem:
sudo mkdir /opt/nwchem
sudo chown $USER /opt/nwchem
cd /opt/nwchem
wget http://www.nwchem-sw.org/images/Nwchem-6.1.1-src.2012-06-27.tar.gz
tar xvf Nwchem-6.1.1-src.2012-06-27.tar.gz
cd nwchem-6.1.1-src/

Edit nwchem-6.1.1-src/src/config/makefile.h and edit line 1957 as shown in this post.

Then continue:
export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export PYTHONVERSION=2.7
export PYTHONHOME=/usr
export BLASOPT="-L/opt/openblas/lib -lopenblas"
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/usr/lib/openmpi
export MPI_INCLUDE=/usr/include
export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openmpi:/opt/openblas/lib
export LIBMPI="-L/usr/lib/openmpi -lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"
cd $NWCHEM_TOP/src
make clean
make nwchem_config
make FC=gfortran 1> make.log 2>make.err
export FC=gfortran
cd ../contrib
./getmem.nwchem

Note that some of the locations are a little bit different from debian.

Edit your ~/.bashrc and add:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/openblas/lib export PATH=$PATH:/opt/nwchem/nwchem-6.1.1-src/bin/LINUX64
You can now test your new binary by running a job, e.g. co.nw:
title "co nmr" geometry c 0 0 0 o 0 0 1.13 end basis * library "6-311+g*" end dft direct grid fine mult 1 xc HFexch 0.05 slater 0.95 becke88 nonlocal 0.72 vwn_5 1 perdew91 0.81 end task dft optimize
Run:

nwchem co.nw

or

mpirun -n 2 nwchem co.nw

06 February 2013

333. Arch linux with Gnome in virtualbox

Not too different from what I've posted before (330, 331), but I'm just so excited that I can't help myself, so here we go:

I'll presume that you've got virtualbox installed.


0. Get the Arch iso:
If you're allowed to use bittorrent:

sudo apt-get install bittorrent squashfs-tools
cd ~/Downloads/
btdownloadcurses https://www.archlinux.org/iso/2013.01.04/archlinux-2013.01.04-dual.iso.torrent

If you're not allowed to use bittorrent:

cd ~/Downloads
wget http://mirror.aarnet.edu.au/pub/archlinux/iso/2013.01.04/archlinux-2013.01.04-dual.iso

Note that the links will change with time.



1. Setting up the machine
Start virtualbox and click on New. Select Type=Linux and Version=Arch Linux.

Give the machine at least 512 Mb, although even that is probably a bit on the low end for Gnome.

Create a virtual harddisk. 8 Gb is enough for testing purposes, but if you're actually planning on using the virtual machine you're better off giving it something like 15-20 Gb. VDI is fine when it comes to format, and 'Dynamically allocated' is a safe choice for a normal desktop.








There's a little bit of tinkering left to do. Click on Settings, go to Storage, and mount your iso:





2. First Boot
Click on Star which will boot off of the CD.


You can pick whatever makes you happy here. For the sake of simplicity will pick the first option -- x86_64.

You're now met with the first indication that this ain't Kansas (or Ubuntu):

Remember -- terminals/consoles should give you warm and fuzzy feeling because there is nothing which you can't accomplish in a terminal. A one-click installation option would be nice for checking things out though, but it's actually pretty easy to get started.


Preparing the harddrive
Our first step is to format the harddrive. Remember, it's an imaginary virtual harddrive -- nothing will be touched on your physical harddrive.

Do

fdisk /dev/sda

Hit n to create a new partition. p for primary, 1 to make it partition number 1, make it start at sector 2048, and make it e.g. 2 Gb by having it stop at +2G -- this will be our swap. Hit n again, p again, and make it partition number 2, and accept the default sectors that it suggests. This will be our  /(root) partition.

Hit a, then 2, to make the partition bootable.

Hit t, then 1, then 82 to make a swap disk.

Hit w to finalise the changes.

Do
mkswap /dev/sda1
mkfs.ext4 /dev/sda2

to set up the file systems.

Do
mount /dev/sda2 /mnt

to mount your new ext4 partition.

Install a basic system:
Do
pacman-key --init
pacman-key --populate archlinux

to get set up with pacman. Edit pacman.conf if you want a specific architecture (i686 or x86_64), or the current architecture will be used.

Time to install packages:
pacstrap /mnt base gnome xorg-xinit xorg-server xorg-utils xorg-server-utils virtualbox-guest-modules virtualbox-guest-utils vim mlocate wicd wicd-gtk openssh grub-bios upower

Generate a proto-fstab:
genfstab -p /mnt > /mnt/etc/fstab

Then enter a chroot:
arch-chroot /mnt

Edit /etc/fstab and add a line for swap:
/dev/sda1    none    swap    sw    0 0

Set up your locale and timezone:
ln -s /usr/share/zoneinfo/Australia/Melbourne /etc/localtime
echo 'LANG="en_US.UTF-8"'>/etc/locale
echo 'KEYMAP=us'> /etc/vconsole.conf
echo 'en_US.UTF-8 UTF-8' >> /etc/locale.gen
locale-gen
echo "spinebill" >/etc/hostname

Set up a user:
pacman -S sudo
useradd -p `openssl passwd -1 mypassword` -m verahill
echo "verahill ALL=(ALL) ALL">> /etc/sudoers

Edit /home/verahill/.xinitrc and uncomment
exec gnome-session

Set up grub and booting:
mkinitcpio -p linux
grub-mkconfig -o /boot/grub/grub.cfg
grub-install /dev/sda

You're now ready to start your Arch system. First you need to shut down your current system though:
exit
umount /mnt
shutdown -h now

Once the machine is off, go to Settings, Storage, and remove disk from virtual drive.



Booting:
Boot by clicking on Start.
You might get the odd error message while booting, but everything will be fine.

Log in as the user you created, then enabled the ssh daemon and the wicd daemon:
sudo systemctl enable sshd
sudo systemctl enable wicd

Start wicd-curses
wicd-curses

and wait 10-20 seconds for wicd to detect the network card and starting it


Quit by hitting 'Q'.

To launch gnome, type
startx
evening gnome?
It'll take a little while to start up. This is a pretty ugly Gnome though, but that's the stock background which changes during the day.
daytime gnome?