15 April 2013

388. NWChem, PSPW and Fortran runtime error

I'm following this post: http://www.emsl.pnl.gov/docs/nwchem/nwchem-support/2012/12/0024.Re:_NWCHEM_errors_in_running_nwchem-6.1.

The issue:
If, on debian testing/wheezy 64 bit with gcc 4.7,  you run the following using NWChem 6.1.1

Title "boric acid" Start boric echo charge 0 geometry autosym units angstrom B 0.00000 0.00000 0.00000 O -4.93432e-17 1.55000 0.00000 H 1.06537 1.92667 0.00000 O 1.34234 -0.775000 0.00000 H 1.13586 -1.88597 0.00000 O -1.34234 -0.775000 0.00000 H -2.20123 -0.0406974 0.00000 end ecce_print ecce.out nwpw mult 1 xc pbe96 cutoff 90.0 np_dimensions -1 -1 tolerances 1e-7 1e-7 car-parrinello nose-hoover 1.000000e+03 2.981500e+02 1.000000e+03 2.981500e+02 time_step 0.500000e+00 fake_mass 1.000000e+02 loop 10 100 scaling 1.000000e+00 1.000000e+00 end end task pspw energy task pspw car-parrinello
You'll end up with
**************************************************** * * * NWPW PSPW Calculation * * * * [ (Grassman/Stiefel manifold implementation) ] * * * * [ NorthWest Chemistry implementation ] * * * * version #5.10 06/12/02 * * * * This code was developed by Eric J. Bylaska, * * and was based upon algorithms and code * * developed by the group of Prof. John H. Weare * * * **************************************************** >>> JOB STARTED AT Mon Apr 15 10:05:42 2013 <<< ================ input data ======================== library name resolved from: compiled reference NWCHEM_NWPW_LIBRARY set to: Generating 1d pseudopotential for B At line 649 of file psp_generator_input.F (unit = 99, file = './junk.inp') Fortran runtime error: Sequential READ or WRITE not allowed after EOF marker, possibly use REWIND or BACKSPACE -------------------------------------------------------------------------- mpirun has exited due to process rank 0 with PID 28100 on node neon exiting without calling "finalize". This may have caused other processes in the application to be terminated by signals sent by mpirun (as reported here). -------------------------------------------------------------------------


The fix
/opt/nwchem/nwchem-6.1.1-src is the NWCHEM_TOP of my particular nwchem source. See e.g. here for build instructions in general and not that e.g. BLASOPT settings etc. correspond to what I need on my system. Modify as necessary.

cd /opt/nwchem/nwchem-6.1.1-src/
wget http://www.nwchem-sw.org/images/Makefile.h.gcc46.patch.gz
gunzip Makefile.h.gcc46.patch.gz
patch -p0 < Makefile.h.gcc46.patch
cd src/nwpw/
touch `egrep -l GCC46 */*/*F`

export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=/opt/nwchem/nwchem-6.1.1-src
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all"
export PYTHONHOME=/usr
export BLASOPT="-L/opt/acml/acml5.2.0/gfortran64_fma4_int64/lib -lacml"
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export LIBRARY_PATH="$LIBRARY_PATH:/usr/lib/openmpi/lib:/opt/acml/acml5.2.0/gfortran64_fma4_int64/lib"
export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"
export FC=gfortran
make 
cd ../
make link
./getmem.nwchem

Because we just relink instead of recompiling from scratch, recompiling is fairly quick (2-3 minutes in total)

Testing

It now continues as it should:
================ input data ======================== library name resolved from: compiled reference NWCHEM_NWPW_LIBRARY set to: Generating 1d pseudopotential for B

and everything is good from there-on.


NOTE: you need to delete the output files from any previous run, or you might end up with errors during the Car-Parrinello portion (e.g.
At line 170 of file MOTION.F (unit = 19, file = './boric.ion_motion')
Fortran runtime error: Sequential READ or WRITE not allowed after EOF marker, possibly use REWIND or BACKSPACE
)



Stay tuned...
What remains for me is to start collecting the patches that are needed for nwchem to work properly (from my perspective -- I only have to support myself and my debian boxes, not a heterogeneous set of users with equally heterogeneous hardware, so my job is easier than that of the nwchem devs) so that installing and upgrading nwchem becomes less painful (yes, I've been patching by hand up till now...).

The issues that need to be patched are:
* including libz and libssl on debian when building with python support (e.g. step 1 in this post)
* changing the details in the output to support gabEdit (e.g. this post)
* patching for GCC 4.7 (i.e. what you've just read)

Update: The post is here now: http://verahill.blogspot.com.au/2013/04/389-patches-for-nwchem-611-on-debian.html

387. Compiling Wine 1.5.28 in a i386 chroot on Debian Wheezy amd64

UPDATE 16 May 2013: See here for Wine 1.5.30: http://verahill.blogspot.com.au/2013/05/416-wine-1530-in-chroot.html

UPDATE: for those who don't want to compile and are willing to take a minor risk* there are pre-built debs for mepis here: http://main.mepis-deb.org/mepiscr/testrepo/pool/test/w/wine/
They are provided by Steve Pusser, who is an active member at the debian forums.

* I presume, but don't actually know, that they are compatible with debian due to 1) Steve being active on the debian forums and 2) Mepis being based on debian stable (currently Squeeze)

Original post

New version of wine out now: 1.5.28.

Not much to get excited about from what I can tell, but wine now relies on GnuTLS instead of openssl.

Here's how to build it -- a lot of it is shamelessly recycled from earlier posts on this blog.

Getting started:
If you set up a chroot to build 1.5.27 before, you don't need to set up a new chroot to build 1.5.28. In that case, skip the set-up step below and instead re-enter your existing chroot like this:
sudo mount -o bind /proc wine32/proc
sudo cp /etc/resolv.conf wine32/etc/resolv.conf
sudo chroot wine32
su sandbox

Setting up the Chroot
mkdir $HOME/tmp/architectures/wine32
cd $HOME/tmp/architectures
sudo debootstrap --arch i386 wheezy $HOME/tmp/architectures/wine32 http://ftp.au.debian.org/debian/
sudo mount -o bind /proc wine32/proc
sudo cp /etc/resolv.conf wine32/etc/resolv.conf
sudo chroot wine32

You're now in the chroot:
apt-get update
apt-get install locales sudo vim
echo 'export LC_ALL="C"'>>/etc/bash.bashrc
echo 'export LANG="C"'>>/etc/bash.bashrc
echo '127.0.0.1 localhost beryllium' >> /etc/hosts
source /etc/bash.bashrc
adduser sandbox
usermod -g sudo sandbox
echo 'Defaults !tty_tickets' >> /etc/sudoers
su sandbox
cd ~/

Building Wine
While still in the chroot, continue (the i386 is ok; don't worry about it -- you don't actually need it):

sudo apt-get install libx11-dev:i386 libfreetype6-dev:i386 libxcursor-dev:i386 libxi-dev:i386 libxxf86vm-dev:i386 libxrandr-dev:i386 libxinerama-dev:i386 libxcomposite-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libdbus-1-dev:i386 libgnutls-dev:i386 libncurses-dev:i386 libsane-dev:i386 libv4l-dev:i386 libgphoto2-2-dev:i386 liblcms-dev:i386 libgstreamer-plugins-base0.10-dev:i386 libcapi20-dev:i386 libcups2-dev:i386 libfontconfig-dev:i386 libgsm1-dev:i386 libtiff-dev:i386 libpng-dev:i386 libjpeg-dev:i386 libmpg123-dev:i386 libopenal-dev:i386 libldap-dev:i386 libxrender-dev:i386 libxml2-dev:i386 libxslt-dev:i386 libhal-dev:i386 gettext:i386 prelink:i386 bzip2:i386 bison:i386 flex:i386 oss4-dev:i386 checkinstall:i386 ocl-icd-libopencl1:i386 opencl-headers:i386 libasound2-dev:i386 build-essential
mkdir ~/tmp
cd ~/tmp
wget http://prdownloads.sourceforge.net/wine/wine-1.5.28.tar.bz2
tar xvf wine-1.5.28.tar.bz2
cd wine-1.5.28/
./configure
time make -j2
sudo checkinstall --install=no
checkinstall 1.6.2, Copyright 2009 Felipe Eduardo Sanchez Diaz Duran This software is released under the GNU GPL. The package documentation directory ./doc-pak does not exist. Should I create a default set of package docs? [y]: Preparing package documentation...OK Please write a description for the package. End your description with an empty line or EOF. >> wine 1.5.28 >> ***************************************** **** Debian package creation selected *** ***************************************** This package will be built according to these values: 0 - Maintainer: [ root@beryllium ] 1 - Summary: [ wine 1.5.28 ] 2 - Name: [ wine ] 3 - Version: [ 1.5.28 ] 4 - Release: [ 1 ] 5 - License: [ GPL ] 6 - Group: [ checkinstall ] 7 - Architecture: [ i386 ] 8 - Source location: [ wine-1.5.28 ] 9 - Alternate source location: [ ] 10 - Requires: [ ] 11 - Provides: [ wine ] 12 - Conflicts: [ ] 13 - Replaces: [ ]

Compilation took ca 19 minutes. Checkinstall takes a little while (In particular this step: 'Copying files to the temporary directory...').

Installing Wine
Exit the chroot
sandbox@beryllium:~/tmp/wine-1.5.28$ exit
exit
root@beryllium:/# exit
exit
me@beryllium:~/tmp/architectures$ 

On your host system
Enable multiarch and install ia32-libs, since you've built a proper 32 bit binary:
sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install ia32-libs

Copy the .deb package and install it
sudo cp wine32/home/sandbox/tmp/wine-1.5.28/wine_1.5.28-1_i386.deb .
sudo chown $USER wine_1.5.28-1_i386.deb
sudo dpkg -i wine_1.5.28-1_i386.deb

Links to this post:
http://forum.winehq.org/viewtopic.php?f=8&t=18924

12 April 2013

386. Briefly: Lenovo Thinkpad SL410 mute volume key

I'm a bit tired of the mute button on my keyboard not working (anymore?) on Debian Testing/Wheezy, so it was time to have a look at what could be done.

The issue
On an SL410 there are four buttons to the left of the keyboard: mute mic, volume up, volume down and mute.

Neither of them show up when using xev or showkey. The Volume Up and Down keys work, but not the mute mic or mute volume ones.

You can get the keycodes by monitoring dmesg -- pushing Mute volume three times I get
[ 2392.912282] hda-intel: spurious response 0x0:0x0, last cmd=0x1439000
[ 2393.447050] hda-intel: spurious response 0x0:0x0, last cmd=0x1439000
[ 2393.930206] hda-intel: spurious response 0x0:0x0, last cmd=0x1439000

The problem is that the keycode keeps changing -- the mute key has a different value depending on what the volume is.
mute mic    ?????
vol up/down 0x1439080,0x339080,00,09,13,1b,21,25,29,2d,30,33
mut vol     0x1439000, 339029

Anyway.

Solution

You'll probably find that thinkpad_acpi is loaded:
lsmod|grep thinkpad_acpi

Unload it:
sudo rmmod thinkpad_acpi

Now press the mute button -- did an orange light show under it? That means it's working.

I think the issue is that SL410 isn't a true thinkpad -- it's really an ideapad and the firmware doesn't match.

To make the change permanent,
edit /etc/modprobe.d/thinkpad-blacklist.conf
blacklist thinkpad_acpi

thinkpad_acpi shouldn't load on boot anymore.

Mute mic still doesn't work.

11 April 2013

385. OTPW -- connecting from an insecure computer using one-time passwords

I got inspired to look into this by this post: http://www.linuxjournal.com/content/configuring-one-time-password-authentication-otpw

The idea of one time passwords  is fairly simple -- even if someone manages to keylog your password (or see you type it in) it will be useless to them, as it can only be used once.

The implementation is also fairly simple: if you divide a password in half, and only let one half be static (the prefix) and the other half come from a pre-agreed list (the suffix), you can have a secure way of changing your password in a practical way.

Let's say that I decide that the static, prefix part should be 'ice' and that we make a pre-agreed list of suffices that we can use to make passwords:
001. box
002. flower
003. icle
004. skating
005. fishing

You should print this list and keep it reasonably safe. Without the prefix it is not useful. Likewise, any keylogger on a compromised computer will find out the prefix, but it will be useless without the suffix list. So keep that it in mind -- make sure that no-one gets hold of both.

You then try to log in via ssh from a remote client, and you get prompted for password 004 -- this means that the password you need to use is iceskating.

In real-world applications the suffixes are random, and not at all related to the prefix. Also, once the list has been generated it is not stored (so you better print it) -- only the hashes or the complete passwords are.

Note that the choice isn't really between OTPW and regular static password -- you can use both, just like you can use SSH with both key and password.



Set up
http://www.linuxjournal.com/content/configuring-one-time-password-authentication-otpw?page=0,2 deals with a number of different use scenarios. I'm only interested in enabling OTPW for remote SSH at this stage i.e. local login is using static passwords by default.

(if you're travelling with a laptop you may consider using OTPW for everything)

You need libpam-otpw. In addition, otpw-bin is useful for generating the suffix/hash list.
sudo apt-get install libpam-otpw otpw-bin

Generate a list of hashes and suffices:
otpw-gen|tee otpw.list

Print the list and delete it afterwards.

You now have a ~/.otpw file filled with hashes.

Create /etc/pam.d/ssh-otpw.
auth sufficient pam_otpw.so session optional pam_otpw.so

and edit /etc/pam.d/sshd -- include the ssh-otpw file immediately before common-auth to set the order of log in methods.
auth required pam_env.so # [1] auth required pam_env.so envfile=/etc/default/locale @include ssh-otpw @include common-auth account required pam_nologin.so @include common-account @include common-session session optional pam_motd.so motd=/run/motd.dynamic noupdate session optional pam_motd.so # [1] session optional pam_mail.so standard noenv # [1] session required pam_limits.so @include common-password
This way, if there's a ~/.otpw file you'll first be prompted for a one-time password. Otherwise that option will be skipped, so users with and without OTPW can co-exist happily.
In /etc/ssh/sshd_config, set
ChallengeResponseAuthentication yes
and make sure that UsePrivilegeSeparation and UsePAM are also set to yes (should be on Debian Wheezy).
sudo service ssh restart

and that's it!

Testing
To test, use a different computer and try to log in. I've got all my local computers set up to use keys to log in, so I had to to
ssh -o PubKeyAuthentication=no me@beryllium
Password 267:
In my list, 267 was given as
dfuF XE+L
so (pretending that my prefix was ice) my password was icedfuFXE+L.
If you answer that challenge wrong, you'll be asked for your static password instead. If you want to keep that secret, then interrupt the connection and retry.

Easy!

Once you start running out of password, run otpw-gen again for a new list.

384. Another (vaguely) TeX-related headache

The error
ps2pdf wasn't working on one of my .ps files (generated from a .tex file by texmaker). When I try to print the .ps file, it also shows up as having 197 pages, instead of 85 pages, which is what is has and what is seen in e.g. evince.

ps2pdf gives
Error: /syntaxerror in -file-
Operand stack:

Execution stack:
   %interp_exit   .runexec2   --nostringval--   --nostringval--   --nostringval--   2   %stopped_push   --nostringval--   --nostringval--   --nostringval--   false   1   %stopped_push   1910   1   3   %oparray_pop   1909   1   3   %oparray_pop   1893   1   3   %oparray_pop   1787   1   3   %oparray_pop   --nostringval--   %errorexec_pop   .runexec2   --nostringval--   --nostringval--   --nostringval--   2   %stopped_push
Dictionary stack:
   --dict:1164/1684(ro)(G)--   --dict:4/20(G)--   --dict:101/200(L)--   --dict:215/300(L)--   --dict:90/200(L)--   --dict:57/75(L)--   --dict:211/313(L)--   --dict:72/140(L)--   --dict:0/10(G)--   --dict:0/10(L)--   --dict:6/50(ro)(G)--   --dict:56/71(L)--
Current allocation mode is local
Last OS error: No such file or directory
Current file position is 86139456
GPL Ghostscript 9.05: Unrecoverable error, exit code 1

And I have no idea where to start troubleshooting...the file position error is not terribly helpful given that the file only has 2010146 lines. So I commented everything, then slowly uncommented everything.

And it was a bit more complex than I thought -- this is the smallest section of code I could use to reproduce the error (note that the two frames are about 30 pages apart in the original):


[preamble]
\begin{document}
\begin{frame}{A frame}
\begin{table}
 \begin{tabular}{l r}
 \includegraphics[width=4cm]{images/figure_1.eps} & \pause
 \includegraphics[width=3.5cm]{img/folder_1/figure_2.eps} 
 \end{tabular}
\end{table}
\small{
\begin{itemize}
\item Blah blah blah
\item Blah Blah?
\end{itemize}
}
\end{frame}

\begin{frame}{Another frame}
 \small{Blah blahdi blah}\\
 \begin{center}
 \begin{table}
 \begin{tabular}{c c}
 \includegraphics[width=5cm]{img/folder_1/figure_3.eps} &
 \includegraphics[width=5cm]{img/folder_1/figure_4.eps} \\
 {\sc Caption} & Caption
 \end{tabular}
 \end{table}
 \end{center}
\end{frame}

\end{document}

The error was only apparent if both figure_2.eps and figure_3.eps were requested -- somehow those two files didn't like each other. I had made figure_2.eps myself (povray -> png -> eps), but a collaborator had sent me figure_3.eps, which was made using Bruker Topspin (a stacked plot of NMR spectra)

The fix:
I fixed it by opening figure_3.eps in inkscape and saving it again

bad.eps is figure_3 before re-saving it, good.eps is figure_3 after re-saving it:

me@beryllium:~/work/manuscripts/Talks/Australie/img/folder_1$ head -n 20 bad.eps 
���� X�+x�+ ��%!PS-Adobe-3.1 EPSF-3.0 %ADO_DSC_Encoding: Windows Roman %%Title: 72_tempbw.eps %%Creator: Adobe Illustrator(R) 15.1 %%For: Collaborator %%CreationDate: 9/12/2012 %%BoundingBox: 0 0 633 359 %%HiResBoundingBox: 0 0 632.8331 358.5000 %%CropBox: 0 0 632.8331 358.5000 %%LanguageLevel: 2 %%DocumentData: Clean7Bit %ADOBeginClientInjection: DocumentHeader "AI11EPS" %ADO_ContainsXMP: MainFirstustrator(R) 15.1.0 x39 R agm 4.5189 ct 5.1875 %ADOEndClientInjection: DocumentHeader "AI11EPS" %%Pages: 1 %%DocumentNeededResources: %%DocumentSuppliedResources: procset Adobe_AGM_Image 1.0 0 %%+ procset Adobe_CoolType_Utility_T42 1.0 0 %%+ procset Adobe_CoolType_Utility_MAKEOCF 1.23 0 %%+ procset Adobe_CoolType_Core 2.31 0
me@beryllium:~/work/manuscripts/Talks/Australia/img/folder_1$ head -n 20 good.eps
%!PS-Adobe-3.0 EPSF-3.0 %%Creator: cairo 1.12.2 (http://cairographics.org) %%CreationDate: Thu Apr 11 11:36:28 2013 %%Pages: 1 %%DocumentData: Clean7Bit %%LanguageLevel: 2 %%BoundingBox: 0 -1 633 359 %%EndComments %%BeginProlog save 50 dict begin /q { gsave } bind def /Q { grestore } bind def /cm { 6 array astore concat } bind def /w { setlinewidth } bind def /J { setlinecap } bind def /j { setlinejoin } bind def /M { setmiterlimit } bind def /d { setdash } bind def /m { moveto } bind def
I had another couple of files which caused the same issue -- all of which originally came from Bruker Topspin (XWinNMR). Re-saving them in inkscape solved the issues.

I still don't understand why figure_1 had to be present though.

09 April 2013

382. Briefly: fixing pdfstudio's leftover crud

I didn't like pdfstudio (http://verahill.blogspot.com.au/2013/02/338-annotating-pdfs-in-linux-revisited.html), not least because it's stuck in a microsoft world of 'basic' and 'pro' versions i.e. artificial crippling. I promptly got rid of it, but it has been lingering on my system like a bad relationship:
dpkg: warning: parsing file '/var/lib/dpkg/available' near line 94935 package 'pdfstudio': missing description

The simplest, most radical fix is to do
sudo dpkg --clear-avail
sudo apt-get update

08 April 2013

381. Encrypting chat, voice, video-- revisited

About a year ago I published this post:
128. Encrypting your email, chat and VOIP in linux (Debian Wheezy)

While I'm using gajim for encrypted chatting on a daily basis, always sign my emails with PGP/GPG in thunderbird and evolution, and keep a truecrypt container in my dropbox account, I've noticed that I don't ever use twinkle, which was the voice encryption solution I looked at in post 128.

The time is thus ripe to explore alternatives. Also, seahorse and evolution have changed enough that updated screenshots may be warranted. So here's Encryption for 2013. You will find that the most difficult thing  about encryption is to convince other people to use it as well.

The tl;dr version is: use gajim. It's awesome.


--- Beginning of the boring bits ---

Why?
I'm not actually paranoid, but what about

*'If you don't have anything to hide, then what are you worrying about?' ? The issue with that kind of reasoning is that it can twisted into meaning that if you do worry, then you do have something to hide, and  mere possession of  will imply guilt. Presumably, if people in general are using e.g. PGP, the risk of it being banned or restricted (or being enforceable) is smaller. See e.g. this if you're living in the US.

* What happens if you gmail account gets hacked? Could someone glean enough information from your conversations and email to steal your identity?

* Do you need to send someone your PIN, bank card info or SSN but don't necessarily feel

* Do you need to discuss something but keep it confidential?

* It's fun! Being able to encrypt gives you a bit of a power trip.

What does annoy me are people who think that they aren't targets because 'why would anyone want to hack me'? Low-profile individuals are rarely targets in their own right, but what about the company you are working for? What about friends of yours? What about the possibility of opportunistic drive-by hacking where targets aren't selected by who they are, but by lapses in security?

In my case, I work for a large university -- gaining access to the intranet for an outsider would only be the first step in potentially compromising thousands of computers or accessing the personal information of tens of thousands of people (has happened at every uni I've worked). I manage servers at two universities -- if they get compromised their access to high speed internet could be used for spam routing (has also happened).

 I have friends who work with classified projects presumably involving nuclear weapons (nat'l lab in the US) -- I might be an intermediary target.

 I have a credit card, and a credit history. What if someone steals my identity and takes out a loan in my name? (has happened)

Or what if I have confidential data on a USB stick and I lose it -- if it's encrypted damage control is a lot easier (has happened at my uni).

Again, I'm not really paranoid, but there is always a tiny risk that someone, for whatever reason, is targeting you.

OTR and GPG/PGP
Note that I am NOT an expert on this. I've used PGP a lot, but have only recently been introduced to OTR. Also, merely being a user of something doesn't make an expert at it. What I write might be wrong.

PGP (pretty good privacy)/GPG (it's open sourced implementation) is a proven encryption method which uses asymmetric encryption with a private and a public key. You use someone else's public key to encrypt messages to them, and only their private key can decrypt it. Likewise, you give your public key to someone who wants to email you -- they use that to encrypt messages to you, and you -- and only you -- can decrypt it with your private key. PGP also supports signing to verify authorship -- you can sign (not encrypt) a message using your private key, and then anyone with your public key can decrypt it.

It's worth emphasizing this: if you encrypt with your private key, anyone in the world can decrypt it.

Whenever you generate a new key pair, you can publish the public key on a range of online repositories, such as http://pgp.mit.edu/ I think seahorse (gnome, evolution) and enigmail (thunderbird) can automatically search public key servers for public keys.

PGP is in principle a very strong encryption method using a 2048 byte (or larger) key. However, it suffers from a major drawback:  anyone who can get their hands on your private key can decrypt everything. That basically means that anyone who gets access to your unencrypted harddrive can copy your key and eavesdrop. The longer you use the same key, the more likely it is that this will already have happened, especially if you have traveled through the US and been forced to give them access to your laptop. Leaving your laptop in your hotel room -- especially in a place like China (although it goes for anywhere) -- is probably also not advisable. Since you can't keep an eye on your hardware 24/7, this is a serious issue.

Another potential issue is that if you sign conversation with a key, then that conversation can be attributed to you i.e. a third, eavesdropping party may be able to ascertain who the conversing parties are.

Again, emphasis: your conversation may be safe now, but if in three months your key is lost all your conversations for years will be decrypted.

You can deal with this to some extent by either encrypting your entire harddrive, or by keeping your private key on an encrypted USB stick (be mindful of the increased risk of misplacing your usb stick and other people gaining access to it -- any encryption can generally be broken, given enough time and resources).

My main use of PGP/GPG is to have candid discussions via chat, and to sign my email.

OTR (Off the Record) is a 'new' (since 2004) alternative and is described in (accessible) detail here: http://www.cypherpunks.ca/otr/otr-wpes.pdf

It aims to solve a lot of the issues with PGP by doing away with a long-lived key. It also aims to make it impossible for a third party to identify the conversing parties. From what I understand it works more or less like this:

If we decide to chat, I generate a key,and you generate a key. Once we're done talking, we dispose of the keys. No-one --- not even the conversing parties -- can decrypt the conversation, which also ensures that nothing will surface by accident at a later date. PGP can obviously be used in the same manner, but it's the implementation of it which has to be convenient. Obviously, in practice this works a whole lot better for chat than email.

Anyway, from my reading of it OTR works pretty much the same as PGP (using large primes and public/private keys) -- it's just the actual implementation which differs. On starting the chat you click on a button to generate your key pair, and the public key gets sent to your chat partner and vice versa. Exit the chat session and your keys are gone. OTR also uses authentication of messages to ensure authorship i.e. even though a man in the middle can encrypt messages to you using your public key and pretend that they are written by your conversation partner, the lack of the correct authentication signature will alert you to the fact that someone is spoofing the messages. PGP does the same by signing with the private key. There's no way to prove authorship based on the authentication signature though -- all you can do is determine that a series of messages came from the same author, but there's nothing which intrinsically proves who that person is. With PGP, on the other hand, if the assumption is that the private keys of the conversing parties have not been compromised, then they both are who they claim they are (of course you might've been duped into using someone else's public key and then all bets are off).

Anyway, OTR improves on PGP and works well for chat. PGP is probably still the easier solution for email.

SRTP (secure real-time transport protocol)
I don't know much about this, and am heavily relying on the Wikipedia page.
SRTP is an encryption method for streamed data, which I read to include voice and video. It needs a key management protocol, such as ZRTP. The way keys are managed depend on the protocol. As far as I understand ZRTP uses temporary asymmetric keys which are exchanged at the beginning of the encryption part of the conversation, similar to OTR, but includes one major difference: a call begins with a verbal exchange of a specific value displayed in the clients of the two parties conversing. This is to avoid that there's a man in the middle attack -- you will presumably recognise the voice of the person you're talking to and be able to verify the identity that way.

I won't cover data encryption using e.g. Truecrypt, in particular, since I've covered it in detail before, e.g. here: http://verahill.blogspot.com.au/2012/04/using-truecrypt-with-dropbox.html

Solutions that I won't consider:
While e.g. skype uses encryption, you're not in control. Also, it's not open source, so who knows how it works

While you using https and google talk in a browser encrypts you video/voice calls and chat, you're not in control -- google can decrypt your conversation any time.

--- End of the boring bits ---


PGP/GPG keypair
The easiest way to do it is probably via the seahorse GUI.
sudo apt-get install gnupg seahorse

Start seahorse (known as Passwords and Keys in Gnome 3).


Once you've made a key pair, synchronise it with a server so that other people can get your public key, e.g.


Alternatively, you can do everything on the command line:
gpg --gen-key
gpg (GnuPG) 1.4.12; Copyright (C) 2012 Free Software Foundation, Inc. [..] Please select what kind of key you want: (1) RSA and RSA (default) (2) DSA and Elgamal (3) DSA (sign only) (4) RSA (sign only) Your selection? 1 RSA keys may be between 1024 and 4096 bits long. What keysize do you want? (2048) Requested keysize is 2048 bits Please specify how long the key should be valid. 0 = key does not expire = key expires in n days w = key expires in n weeks m = key expires in n months y = key expires in n years Key is valid for? (0) 0 Key does not expire at all Is this correct? (y/N) Y You need a user ID to identify your key; the software constructs the user ID from the Real Name, Comment and Email Address in this form: "Heinrich Heine (Der Dichter) " Real name: I Lindqvist Email address: i.lindqvist@email.net Comment: fake address You selected this USER-ID: "I Lindqvist (fake address) " Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O You need a Passphrase to protect your secret key.
You'll get asked for a passphrase twice. Then:
We need to generate a lot of random bytes. It is a good idea to perform some other action (type on the keyboard, move the mouse, utilize the disks) during the prime generation; this gives the random number generator a better chance to gain enough entropy. Not enough random bytes available. Please do some other work to give the OS a chance to collect more entropy! (Need 283 more bytes) ..+++++ ...+++++ We need to generate a lot of random bytes. It is a good idea to perform some other action (type on the keyboard, move the mouse, utilize the disks) during the prime generation; this gives the random number generator a better chance to gain enough entropy. +++++ Not enough random bytes available. Please do some other work to give the OS a chance to collect more entropy! (Need 109 more bytes) .....................+++++ gpg: key 2B4C5636 marked as ultimately trusted public and secret key created and signed. gpg: checking the trustdb gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u pub 2048R/2B4C5636 2012-05-02 Key fingerprint = 5B71 C3F1 0C2D E008 B299 21A8 019F 907E 2B4C 5636 uid I Lindqvist (fake address) sub 2048R/78F9B6C1 2012-05-02
gpg --list-key
/home/me/.gnupg/pubring.gpg ----------------------------- pub 2048R/2B4C5636 2012-05-02 uid I Lindqvist (fake address) sub 2048R/78F9B6C1 2012-05-02
To add more email addresses, do
gpg --edit-key 2B4C5636
>adduid


and follow the prompts. You do not want to add subkeys. Do
>trust

to set the trust level. Ultimate for your own key, full for your pals.

Implementation

Chat:

Gajim 
Gajim supports both PGP (natively) and OTR (via a plug in). It's also easy to use and written in python, which is a plus.

To enable PGP/GPG, go to Edit, Accounts, Personal Information, Choose Key:

You need to assign the public key to the contacts you want to chat with. First get the public key -- either from an online repo or by email from your contact. In e.g. seahorse, go to Remote, Find Remote Keys and search:




Then, in gajim, do


To enable OTR, go to Edit, Plugins, click on the Available tab and wait a little while for the list to populate:





To use OTR, do the following:
Start chat, click on the settings icon, select off-the-record, and start

It's now encrypted

To verify the identity of the other party, click on Authenticate contact

Ask a question

And the other person answers

Authentication achieved

Alternatives
mcabber also support PGP/GPG, and is a curses based program. See here for configuration.

pidgin and jitsi both support OTR, but not PGP.

Pidgin and mcabber are in the debian repos.


Email:

Evolution:
Enabling PGP/GPG is very easy -- edit your account, and go to the Security tab:
To encrypt, start a new message and click on Options. You might want to have html turned off.
Thunderbird:
You'll need to install enigmail.

 You can select your key etc. under key management.

Alternatives
Mutt does PGP/GPG and is a neat curses-based program.


Z/SRTP in Voice

There are a couple of solutions for 'secure' voice and video. I looked at Twinkle before, and while it's a capable SIP program, the downside is that it requires access to a SIP service (e.g. Ekiga, or via your ISP). What is really needed are clients which can encrypt communication going via e.g. google talk/jabber and the likes. Jitsi does that, and is open source.

I won't consider Skype for the usual reasons -- it's not free, open or transparent.

Jitsi
Get jitsi by doing
wget https://download.jitsi.org/jitsi/debian/jitsi_1.0-latest_amd64.deb
sudo dpkg -i jitsi_2.0-latest_amd64.deb

Start Jitsi and set up an account. You can add more accounts later by going to Options and clicking the Accounts tab.

Anyway, start Jitsi, and go to Options, Security to get an overview. Here's the ZRTP tab. Basically you CAN change the parameters, but you're advised not to:

Jitsi also does OTR chat:

And you can set a master password, so that people can't start jitsi if you leave your desktop unprotected.

You don't need to do anything to enable ZRTP as it will automatically be used if both your and your partner's clients support it, so here's OTR in Jitsi:





380. Modifying NWChem code without a full recompile

This isn't a secret trick or anything, but may still not be immediately obvious to most people.

A full compilation of NWChem can easily take 20-40 minutes, depending on your build machine.

Sometimes you need to make changes to the source code, e.g. if you want to use GabEdit to analyse output -- ECCE is a fantastic piece of software and is great for managing computations, but GabEdit has implemented some pretty interesting routines for orbital analyses. In order for you to be able to reliably import NWChem output into GabEdit you need to modify a handful of fortran files.

See e.g. http://verahill.blogspot.com.au/2013/02/3xx-modifying-nwchem-611-to-work-with.html for patching NWChem, and http://verahill.blogspot.com.au/2012/11/visualising-nwchem-output-with-gabedit.html for how to run your nwchem jobs and how to visualize them in gabEdit (you need to explicitly define your basis sets).

Angelo Rossi made this comment:
Hello:

Thank you so much for this valuable information.

But under the heading of "Compilation" above, the directions lead to a pointer to compile the entire NWChem source. But only one or two subroutines are modified. Shouldn't there be a more surgical way of proceeding after making the suggested changes? Actually this would make a great separate post. That is, provide a procedure to recompile and link when small changes to NWChem are made. I've done this, but I can't remember.

Kind regards,

Angelo

The answer was on this page: http://xray.isc.kharkov.com/ext_docs/NWChem/prog/node12.html
So here we go:

Partial recompile/relinking of NWChem

1. Environmental variables
Define your environmental variable like you would during a normal compile
export LARGE_FILES=TRUE export TCGRSH=/usr/bin/ssh export NWCHEM_TOP=`pwd` export NWCHEM_TARGET=LINUX64 export NWCHEM_MODULES="all python" export PYTHONVERSION=2.7 export PYTHONHOME=/usr export BLASOPT="-L/opt/openblas/lib -lopenblas" export USE_MPI=y export USE_MPIF=y export USE_MPIF4=y export MPI_LOC=/usr/lib/openmpi/lib export MPI_INCLUDE=/usr/lib/openmpi/include export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openmpi/lib:/opt/acml/acml5.2.0/gfortran64_int64/lib:/opt/openblas/lib export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread" export FC=gfortran
2. Run make in modified directories
In the GabEdit example we made changes to files in src/ddscf, src/nwdft/scf_dft, and src/mcscf
cd $NWCHEM_TOP/src/ddscf
make
cd $NWCHEM_TOP/src/nwdft/scf_dft
make
cd $NWCHEM_TOP/src/mcscf
make

3. Run make link in the src directory
cd $NWCHEM_TOP/src
make link

4. Do your usual post-compilation memory magic
cd $NWCHEM_TOP/contrib
./getmem.nwchem

Done!

This all in all took around 6 seconds instead of 30-odd minutes.

06 April 2013

379. [Solved] Debian Wheezy -- Thinkpad SL410: reboot instead of shutdown when on battery

Solved: it was down to laptop-mode-tools, which I installed as part of this post two weeks ago. I uninstalled that package and my laptop is shutting down properly again.

Original post:
Since today[*], with Debian Wheezy stock kernel (3.2), my Thinkpad SL410 reboots instead of shutting down when it's running on battery. Works fine when on mains power. During shutdown it says 'Will now halt', then reboots.

Similar to what's seen in this thread: http://forums.debian.net/viewtopic.php?f=7&t=86316&start=15, but shutting down X seems to have no effect. Googling seems to yield a lot of hits for thinkpads and ideapads.

Annoying and no idea why.

The support for SL410 under Debian Wheezy has actually gotten worse during the past year -- now 'mute' doesn't work anymore, and from today it doesn't shutdown on issuing e.g. sudo shutdown -h now.Finally, recently the behaviour of the kernel (3.8) has lead to slowdowns on computer with i915 (intel) graphics, which would include a majority of laptops (a fix is on the way though).

Looking at /var/log/apt/history.log my kernel was updated (not a new version, so a bit confused), two days ago. Nothing else looks important enough.

[*] Given that removing laptop-mode-tools solved it, and that I installed that on the 23rd of March, I must've been running my laptop off of mains power for the past two weeks...alternatively, newer kernels fix this (supposed to be due to the combination of laptop-mode-tools and an apci bug in the kernel) since I've been running 3.7.10 for the past two weeks, and only switched back to 3.2 yesterday.


378. Briefly: Windows/Dialogues that are too large for a laptop screen

Short post which is probably just revealing something which a lot of people are already familiar with.

A really annoying thing with Evolution (and ECCE as well, actually) is that it doesn't play well with notebook screen. I have a Lenovo Thinkpad SL410 which has a 1366x768 screen and this is what I see when e.g. editing an account in Evolution:

Note the lack of buttons to click -- the Close and Apply dialogues don't show on the screen

I might be imagining, but I get the impression that GNOME projects don't like you to resize windows -- and you can't resize this dialogue.

The workaround:
In gnome, use alt+click and drag to move the window so that the buttons are revealed.


05 April 2013

377. Wine 1.5.27 and Office 2003: Equation editor now supported!

I prefer using TeX, but collaboration is a reality of academia, and sometimes it's just easier to shut up and use Word.

To install Office 2003 under Wine on Debian, see here: http://verahill.blogspot.com.au/2013/01/307-installing-office-2003-in-wine-on.html


In post 376 I compiled and installed wine 1.5.27. Starting word under wine I noticed that an old mathtype install was now recognised (if not completely functional), and that gave me hope that Equation Editor would be installable.




Equation Editor
Here's how to get it up and running

1. Control panel
If you've already installed Office under Wine, try using add/remove programs:
wine control

and click on Add/Remove programs, then modify your Office installation. That didn't work for me since my Office install wasn't listed for some reason.

If that doesn't work, then
2. (Re-)Install
sudo mount -o loop OFFICE11.iso /mnt
wine msiexec /i /mnt/pro11.msi

And start the installation.






Start word, go to Insert/Objects




376. Wine 1.5.27 on Debian: i386 chroot

UPDATE 16 May 2013: See here for Wine 1.5.30: http://verahill.blogspot.com.au/2013/05/416-wine-1530-in-chroot.html

Note: Please, if you do have issues actually using the wine binaries built this way, post a comment -- everything is working fine on my systems but I'd like to know exactly what libraries you need to install on the target system to make Wine built in the chroot run to satisfaction.


Preamble: Building wine on Debian Wheezy amd 64 is a PITA at the moment, since some amd64 -dev packages are not co-installable with their i386 counterpart. Such an example is libfreetype6-dev:i386.


If I'm understanding this post correctly I may be confusing the idea behind multiarch vs cross-compiling:

"multiarch provides support for cross-compiling, but we don't have an amd64->i386 cross-compiler - only a biarch (multilib) compiler."

which I interpret to mean that while we generally can compile against 32 bit libs in wheezy using multiarch, it's not necessarily the way it is intended to work (I've never bother to check whether the binaries are 32 bit, or just linked against 32 bit libs). Instead, an alternative way to look at multiarch is that it supports the use of packages linked against 32 bit libs (and puts libraries in arch dependent locations), but not by necessity the compilation of 32 bit binaries (if it does, great). Or maybe I'm wrong. The bottom line is that we might have to accept compiling in a chroot, and then install the 32 bit libs on the target machine to support execution.
I've built wine using chroot in the past, but it was always with the intention of hammering out dependencies.


Getting started:

The Chroot
mkdir $HOME/tmp/architectures/wine32
cd $HOME/tmp/architectures
sudo debootstrap --arch i386 wheezy $HOME/tmp/architectures/wine32 http://ftp.au.debian.org/debian/

sudo mount -o bind /proc wine32/proc
sudo cp /etc/resolv.conf wine32/etc/resolv.conf
sudo chroot wine32

You're now in the chroot:
apt-get update
apt-get install locales sudo vim
echo 'export LC_ALL="C"'>>/etc/bash.bashrc
echo 'export LANG="C"'>>/etc/bash.bashrc
echo '127.0.0.1 localhost beryllium' >> /etc/hosts
source /etc/bash.bashrc
adduser sandbox
usermod -g sudo sandbox
echo 'Defaults !tty_tickets' >> /etc/sudoers
su sandbox
cd ~/

Building Wine
While still in the chroot, continue (the i386 is ok; don't worry about it):
sudo apt-get install libx11-dev:i386 libfreetype6-dev:i386 libxcursor-dev:i386 libxi-dev:i386 libxxf86vm-dev:i386 libxrandr-dev:i386 libxinerama-dev:i386 libxcomposite-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libdbus-1-dev:i386 libgnutls-dev:i386 libncurses-dev:i386 libsane-dev:i386 libv4l-dev:i386 libgphoto2-2-dev:i386 liblcms-dev:i386 libgstreamer-plugins-base0.10-dev:i386 libcapi20-dev:i386 libcups2-dev:i386 libfontconfig-dev:i386 libgsm1-dev:i386 libtiff-dev:i386 libpng-dev:i386 libjpeg-dev:i386 libmpg123-dev:i386 libopenal-dev:i386 libldap-dev:i386 libxrender-dev:i386 libxml2-dev:i386 libxslt-dev:i386 libhal-dev:i386 libcurl4-openssl-dev:i386 gettext:i386 prelink:i386 bzip2:i386 bison:i386 flex:i386 oss4-dev:i386 checkinstall:i386 ocl-icd-libopencl1:i386 opencl-headers:i386 libasound2-dev:i386 build-essential
mkdir ~/tmp
cd ~/tmp
wget http://prdownloads.sourceforge.net/wine/wine-1.5.27.tar.bz2
cd wine-1.5.27/
./configure
time make -j2
sudo checkinstall --install=no
checkinstall 1.6.2, Copyright 2009 Felipe Eduardo Sanchez Diaz Duran This software is released under the GNU GPL. The package documentation directory ./doc-pak does not exist. Should I create a default set of package docs? [y]: Preparing package documentation...OK Please write a description for the package. End your description with an empty line or EOF. >> wine 1.5.27 >> ***************************************** **** Debian package creation selected *** ***************************************** This package will be built according to these values: 0 - Maintainer: [ root@beryllium ] 1 - Summary: [ wine 1.5.27 ] 2 - Name: [ wine ] 3 - Version: [ 1.5.27 ] 4 - Release: [ 1 ] 5 - License: [ GPL ] 6 - Group: [ checkinstall ] 7 - Architecture: [ i386 ] 8 - Source location: [ wine-1.5.27 ] 9 - Alternate source location: [ ] 10 - Requires: [ ] 11 - Provides: [ wine ] 12 - Conflicts: [ ] 13 - Replaces: [ ]

Compilation took ca 20 minutes. Checkinstall takes a little while (In particular this step: 'Copying files to the temporary directory...').

Installing Wine
Exit the chroot
sandbox@beryllium:~/tmp/wine-1.5.27$ exit
exit
root@beryllium:/# exit
exit
me@beryllium:~/tmp/architectures$ 

On your host system
Enable multiarch and install ia32-libs, since you've built a proper 32 bit binary:
sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install ia32-libs

Copy the .deb package and install it
sudo cp wine32/home/sandbox/tmp/wine-1.5.27/wine_1.5.27-1_i386.deb .
sudo chown $USER wine_1.5.27-1_i386.deb
sudo dpkg -i wine_1.5.27-1_i386.deb

The tricky bit
The question is what 32 bit libs you need on the host system, if any.
ldd /usr/local/bin/wine
linux-gate.so.1 => (0x55573000) libwine.so.1 => /usr/local/bin/../lib/libwine.so.1 (0x55577000) libpthread.so.0 => /lib/i386-linux-gnu/i686/cmov/libpthread.so.0 (0x556f5000) libc.so.6 => /lib/i386-linux-gnu/i686/cmov/libc.so.6 (0x5570e000) libdl.so.2 => /lib/i386-linux-gnu/i686/cmov/libdl.so.2 (0x55871000) /lib/ld-linux.so.2 (0x55555000)
So far everything is working fine with ia32-libs.

Links to this post:
http://forums.bistudio.com/showthread.php?149892-ARMA-III-on-Linux-servers-via-WINE%2Fpage4
http://forum.winehq.org/viewtopic.php?f=8&t=19010

04 April 2013

375. Bisecting the kernel -- looking for the commit behind the kworker/i915 issue.

This is an attempt to trace this issue: http://verahill.blogspot.com.au/2013/03/368-slow-mouse-and-keyboard-triggered.html.

See also
http://forums.gentoo.org/viewtopic-p-7278760.html
https://bbs.archlinux.org/viewtopic.php?pid=1248190

Update: I'm revising this post. Not quite done yet.

Another Update: Someone else has just bisected and ended up with a different commit:
https://bbs.archlinux.org/viewtopic.php?pid=1254285#p1254285
The patch they found makes a lot more sense than any of the ones I ended up with.

However, I do still see issues with the kernel I end up with, so I'm either starting from the wrong 'good', or there are several bad commits.

A third update: I just experienced slow-down with 1k interrupts per second on 3.7.10, which may indicate that I picked a bad starting point for bisecting. I'll leave the post up since it can serve as a guide for how to bisect in general.

A fourth update:
A fix is on the way to kernel 3.9 or 3.10.
https://patchwork.kernel.org/patch/2400621/
https://patchwork.kernel.org/patch/2402211/


Original post:

I didn't really want to do this, for several reasons:

* the bug only manifests itself on one of my computers, with intel graphics -- and that's a laptop (I'm not a huge fan of laptops, especially not with budding carpal tunnel syndrome, so testing the bisected version is a bit of a pain -- literally)

* the bug isn't consistently triggered, so you need to test the new kernel for an hour or more, and only by the absence of a specific behaviour can you see whether all is good i.e. testing is a bit of hit and miss.

* compiling the kernel takes a long time, and I don't currently have a suitable computer for it at home where my laptop (the one with the i915) is, so I'll build the kernel on a work computer, then install it at home, then repeat each day.

On the other hand, it's a learning experience, so here we go.

I'm looking at https://www.kernel.org/pub/software/scm/git/docs/git-bisect.html

I'm compiling the kernel debian style since it generates .deb files which are easy to install on other machines. Again, there are posts on how to compile the kernel using a more generic approach, or specifically for Arch, elsewhere on this blog.

NOTE: there's no sure-fire way of triggering this issue, but it seems to occur more frequently when the fan is on but approaching the thermal cut-off point where it will turn off. This makes troubleshooting somewhat more difficult.


1. Download/checkout
First get the sources -- it pulls everything from 2.6.12, so it's bigger than a normal kernel source download. I'll skip explicitly telling you what packages you need to have in order to compile the kernel or use git, since if you don't know, you're probably not really ready for this anyway. Besides, that information is available in other posts on this blog.

mkdir -p ~/tmp/kernel_bisect
cd ~/tmp/kernel_bisect/
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git linux-git
Cloning into 'linux-git'... remote: Counting objects: 2942099, done. remote: Compressing objects: 100% (443946/443946), done. remote: Total 2942099 (delta 2471580), reused 2940624 (delta 2470325) Receiving objects: 100% (2942099/2942099), 608.66 MiB | 4.58 MiB/s, done. Resolving deltas: 100% (2471580/2471580), done. Checking out files: 100% (42425/42425), done.
cd linux-git/


2A. Bisect
I know that 3.8.5 is bad and that 3.7.10 may be good (seem to remember having issues with the Arch version of 3.7.10, but no issues with my own build on debian. Not sure I can trust my memory).

I know for a fact that 3.2 is good. I don't want to bisect everything from 3.8.5 to 3.2 though, so I'll take a leap of faith and presume that 3.7.10 is good (which I'll regret if that's not true) and 3.8.0 is bad.

I know that the issue is present on amd64.

I think it's an issue to do with i915.

git bisect start -- arch/amd64 drivers/gpu/drm/i915
git bisect bad v3.8
git bisect good v3.7
Bisecting: 160 revisions left to test after this (roughly 7 steps) [28d491df4c6b00f9148a9885dba1f36a078535dc] drm/i915: Bad pixel formats can't reach the sprite code
2B. Compile
I'm being a bit lazy here as well, possibly at the cost of not being able to identify my issue: I'll use silentoldconfig to accept all the default settings.
make oldconfig
time fakeroot make-kpkg -j3 --initrd kernel_image kernel_headers

where 3 in -j3 is the number of cores (see here and here and here for more about the proper value for -j) -- I have a triple core AMD. It took around 43 minutes to build.

This will generate two .deb files in the parent folder -- one kernel image and one with headers.
Install these on your testing system.


sudo dpkg -i ../linux-image-3.7.0-rc2+_3.7.0-rc2+-10.00.Custom_amd64.deb ../linux-headers-3.7.0-rc2+_3.7.0-rc2+-10.00.Custom_amd64.deb

Since you installed .deb packages it's pretty easy to roll back any changes later.

3. Testing
What happens now depends on whether the kernel is plagued by the issue you're trying to bisect for:

3A. The problem persists
You've now narrowed it down to half as many commits.
git bisect bad
make-kpkg clean
make oldconfig
time fakeroot make-kpkg -j3 --initrd kernel_image kernel_headers
sudo dpkg -i ../linux-image-3.7.0-rc2+_3.7.0-rc2+-10.00.Custom_amd64.deb ../linux-headers-3.7.0-rc2+_3.7.0-rc2+-10.00.Custom_amd64.deb


3B. The problem is not present
You've now narrowed it down to half as many commits.
git bisect good
make-kpkg clean
make oldconfig
time fakeroot make-kpkg -j3 --initrd kernel_image kernel_headers
sudo dpkg -i ../linux-image-3.7.0-rc4+_3.7.0-rc4+-10.00.Custom_amd64.deb ../linux-headers-3.7.0-rc4+_3.7.0-rc4+-10.00.Custom_amd64.deb

Repeat step 3 until you've isolated the commit that caused the issue.

For me
* the first bisect didn't cause kworker slow downs (bisect good)
* the second bisect immediately led to slowdown (bisect bad)
* the third one was fine (bisect good)
* the fourth one had no issues (bisect good)
* the fifth one was good (bisect good)
* the sixth one was definitely bad (bisect bad)

powertop gives the following when it's slowing down
 0 mW    423.5 ms/s       4.5        kWork          i915_hotplug_work_func

and watch cat /proc/interrupts:
The number of interrupts increases very, very rapidly for '49:  PCI-MSI-edge i915' during mouse slowdown. Normal rate is around 70 per two seconds.
This is where it gets less clear -- from what I can see:
* the seventh one was a little bit bad (bisect bad).
* the eight one had minor issues when the fan went off (bisect bad).

But the first time I did the bisect I came to the exact opposite conclusion for the last two bisects. I'm suspecting that it's really down to two or more commits that together cause bad behavior, but on their own are merely annoying.

Anyway, I've ended up with this commit as the current culprit (again, the first time around I ended up with a different commit):
607a6f7a6621f65706ff536b2615ee65b5c2f575 is the first bad commit
commit 607a6f7a6621f65706ff536b2615ee65b5c2f575
Author: Daniel Vetter 
Date:   Wed Nov 14 17:47:39 2012 +0100

    drm/i915: drop buggy write to FDI_RX_CHICKEN register
    
    Jani Nikula noticed that the parentheses are wrong and we & the bit
    with the register address instead of the read-back value. He sent a
    patch to correct that.
    
    On second look, we write the same register in the previous line, and
    the w/a seems to be to set FDI_RX_PHASE_SYNC_POINTER_OVR to enable the
    logic, then keep always set FDI_RX_PHASE_SYNC_POINTER_OVR and toggle
    FDI_RX_PHASE_SYNC_POINTER_EN before/after enabling the pc transcoder.
    
    So the right things seems to be to simply kill the 2nd write.
    
    Cc: Jani Nikula 
    Reviewed-by: Chris Wilson 
    [danvet: Dropped a bogus ~ from the commit message that somehow crept
    in.]
    Signed-off-by: Daniel Vetter 

:040000 040000 f789c6c199c9db5c9d0d7961760574b5f0b1ede9 9e0cd2a09cab610b437164b1a74f620e43686ef1 M      drivers

It's just really difficult to reproduce the issue consistently with the last couple of kernels. I am in no way confident that the above commit is what's causing all this.

The last confirmed troublesome bisect (#6):

Here's the log:
git bisect start '--' 'arch/amd64' 'drivers/gpu/drm/i915'
# bad: [19f949f52599ba7c3f67a5897ac6be14bfcb1200] Linux 3.8
git bisect bad 19f949f52599ba7c3f67a5897ac6be14bfcb1200
# good: [29594404d7fe73cd80eaa4ee8c43dcc53970c60e] Linux 3.7
git bisect good 29594404d7fe73cd80eaa4ee8c43dcc53970c60e
# good: [28d491df4c6b00f9148a9885dba1f36a078535dc] drm/i915: Bad pixel formats can't reach the sprite code
git bisect good 28d491df4c6b00f9148a9885dba1f36a078535dc
# bad: [b4a98e57fc27854b5938fc8b08b68e5e68b91e1f] drm/i915: Flush outstanding unpin tasks before pageflipping
git bisect bad b4a98e57fc27854b5938fc8b08b68e5e68b91e1f
# good: [12f3382bc0262e981a2e58aca900cbbdbbe66825] drm/i915: implement WaDisablePSDDualDispatchEnable on IVB & VLV
git bisect good 12f3382bc0262e981a2e58aca900cbbdbbe66825
# good: [b9e0bda3cd325b55f336efb751736163f62abded] drm/i915: Always calculate 8xx WM values based on a 32-bpp framebuffer
git bisect good b9e0bda3cd325b55f336efb751736163f62abded
# good: [1c8b46fc8c865189f562c9ab163d63863759712f] drm/i915: Use LRI to update the semaphore registers
git bisect good 1c8b46fc8c865189f562c9ab163d63863759712f

and here are the remaining commits:


commit b4a98e57fc27854b5938fc8b08b68e5e68b91e1f
Author: Chris Wilson 
Date:   Thu Nov 1 09:26:26 2012 +0000

    drm/i915: Flush outstanding unpin tasks before pageflipping
    
    If we accumulate unpin tasks because we are pageflipping faster than the
    system can schedule its workers, we can effectively create a
    pin-leak. The solution taken here is to limit the number of unpin tasks
    we have per-crtc and to flush those outstanding tasks if we accumulate
    too many. This should prevent any jitter in the normal case, and also
    prevent the hang if we should run too fast.
    
    Note: It is important that we switch from the system workqueue to our
    own dev_priv->wq since all work items on that queue are guaranteed to
    only need the dev->struct_mutex and not any modeset resources. For
    otherwise if we have a work item ahead in the queue which needs the
    modeset lock (like the output detect work used by both polling or
    hpd), this work and so the unpin work will never execute since the
    pageflip code already holds that lock. Unfortunately there's no
    lockdep support for this scenario in the workqueue code.
    
    Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=46991
    Reported-and-tested-by: Tvrtko Ursulin 
    Signed-off-by: Chris Wilson 
    [danvet: Added note about workqueu deadlock.]
    Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=56337
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/intel_display.c |   22 ++++++++++++++++------
 drivers/gpu/drm/i915/intel_drv.h     |    4 +++-
 2 files changed, 19 insertions(+), 7 deletions(-)

commit a726915cef1daab57aad4c5b5e4773822f0a4bf8
Author: Daniel Vetter 
Date:   Tue Nov 20 14:50:08 2012 +0100

    drm/i915: resurrect panel lid handling
    
    But disabled by default. This essentially reverts
    
    commit bcd5023c961a44c7149936553b6929b2b233dd27
    Author: Dave Airlie 
    Date:   Mon Mar 14 14:17:55 2011 +1000
    
        drm/i915: disable opregion lid detection for now
    
    but leaves the autodetect mode disabled. There's also the explicit lid
    status option added in
    
    commit fca874092597ef946b8f07031d8c31c58b212144
    Author: Chris Wilson 
    Date:   Thu Feb 17 13:44:48 2011 +0000
    
        drm/i915: Add a module parameter to ignore lid status
    
    Which overloaded the meaning for the panel_ignore_lid parameter even
    more. To fix up this mess, give the non-negative numbers 0,1 the
    original meaning back and use negative numbers to force a given state.
    So now we have
    
    1  - disable autodetect, return unknown
    0  - enable autodetect
    -1 - force to disconnected/lid closed
    -2 - force to connected/lid open
    
    v2: My C programmer license has been revoked ...
    
    v3: Beautify the code a bit, as suggested by Chris Wilson.
    
    Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=27622
    Tested-by: Andreas Sturmlechner 
    Reviewed-by: Chris Wilson 
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/i915_drv.c    |    6 +++---
 drivers/gpu/drm/i915/intel_panel.c |   25 +++++++++++--------------
 2 files changed, 14 insertions(+), 17 deletions(-)

commit 8fed6193736bf22e0e44c03ee783761e9cc37238
Author: Takashi Iwai 
Date:   Mon Nov 19 18:06:51 2012 +0100

    drm/i915: Enable DP audio for Haswell
    
    This patch adds the missing code to send ELD for Haswell DisplayPort,
    based on Xingchao's original patch.
    
    A test was performed with HSW-D machine and NEC EA232Wmi DP monitor.
    
    Cc: Xingchao Wang 
    Signed-off-by: Takashi Iwai 
    Reviewed-by: Paulo Zanoni 
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/intel_ddi.c |    9 +++++++++
 1 file changed, 9 insertions(+)

commit c9839303d186d6270f570ff3c5f56c2327958086
Author: Chris Wilson 
Date:   Tue Nov 20 10:45:17 2012 +0000

    drm/i915: Pin the object whilst faulting it in
    
    In order to prevent reaping of the object whilst setting it up to
    handle the pagefault, we need to mark it as pinned. This has the nice
    side-effect of eliminating some special cases from the pagefault handler
    as well!
    
    Signed-off-by: Chris Wilson 
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/i915_gem.c |   29 +++++++++--------------------
 1 file changed, 9 insertions(+), 20 deletions(-)

commit fbdda6fb5ee5da401af42226878880069a6b8615
Author: Chris Wilson 
Date:   Tue Nov 20 10:45:16 2012 +0000

    drm/i915: Guard pages being reaped by OOM whilst binding-to-GTT
    
    In the circumstances that the shrinker is allowed to steal the mutex
    in order to reap pages, we need to be careful to prevent it operating on
    the current object and shooting ourselves in the foot.
    
    Signed-off-by: Chris Wilson 
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/i915_gem.c |    9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

commit be7cb6347e0c3aa1956748a860a2465a7ea128c4
Author: Chris Wilson 
Date:   Mon Nov 19 15:30:42 2012 +0000

    drm/i915: Remove bogus test for a present execbuffer
    
    The intention of checking obj->gtt_offset!=0 is to verify that the
    target object was listed in the execbuffer and had been bound into the
    GTT. This is guarranteed by the earlier rearrangement to split the
    execbuffer operation into reserve and relocation phases and then
    verified by the check that the target handle had been processed during
    the reservation phase.
    
    However, the actual checking of obj->gtt_offset==0 is bogus as we can
    indeed reference an object at offset 0. For instance, the framebuffer
    installed by the BIOS often resides at offset 0 - causing EINVAL as we
    legimately try to render using the stolen fb.
    
    Signed-off-by: Chris Wilson 
    Reviewed-by: Eric Anholt 
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/i915_gem_execbuffer.c |    9 ---------
 1 file changed, 9 deletions(-)

commit b92fa839015f27ba0f5c7ef9812eba9ecff538c2
Author: Chris Wilson 
Date:   Fri Nov 16 11:43:21 2012 +0000

    drm/i915: Remove save/restore of physical HWS_PGA register
    
    Now that we always restore the HWS registers (both physical and GTT
    virtual addresses) when re-initialising the rings, we can eliminate the
    superfluous save/restore of the register across suspend and resume.
    
    Signed-off-by: Chris Wilson 
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/i915_drv.h     |    1 -
 drivers/gpu/drm/i915/i915_suspend.c |    8 --------
 2 files changed, 9 deletions(-)

commit d09105c66eb813ab3f57ba5e738f477f6ff92dec
Author: Ben Widawsky 
Date:   Thu Nov 15 12:06:09 2012 -0800

    drm/i915: Fix warning in i915_gem_chipset_flush
    
    drivers/gpu/drm/i915/i915_drv.h:1545:2: warning: '______f' is static but
    declared in inline function 'i915_gem_chipset_flush' which is not static
    
    Reported-by: kbuild test robot 
    dri-devel-Reference: <50a4d41c data-blogger-escaped-.586vhmwghpukzbkb="" data-blogger-escaped-fengguang.wu="" data-blogger-escaped-intel.com="">
    Signed-off-by: Ben Widawsky 
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/i915_drv.h |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

commit 42d42e7e4220753bab3eb7b857721f203a4cd821
Author: Damien Lespiau 
Date:   Wed Oct 31 19:23:16 2012 +0000

    drm/i915: Only check for valid PP_{ON, OFF}_DELAYS on pre ILK hardware
    
    ILK+ have this register on the PCH. This check was triggering unclaimed
    writes.
    
    Signed-off-by: Damien Lespiau 
    Reviewed-by: Paulo Zanoni 
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/intel_bios.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

commit 607a6f7a6621f65706ff536b2615ee65b5c2f575
Author: Daniel Vetter 
Date:   Wed Nov 14 17:47:39 2012 +0100

    drm/i915: drop buggy write to FDI_RX_CHICKEN register
    
    Jani Nikula noticed that the parentheses are wrong and we & the bit
    with the register address instead of the read-back value. He sent a
    patch to correct that.
    
    On second look, we write the same register in the previous line, and
    the w/a seems to be to set FDI_RX_PHASE_SYNC_POINTER_OVR to enable the
    logic, then keep always set FDI_RX_PHASE_SYNC_POINTER_OVR and toggle
    FDI_RX_PHASE_SYNC_POINTER_EN before/after enabling the pc transcoder.
    
    So the right things seems to be to simply kill the 2nd write.
    
    Cc: Jani Nikula 
    Reviewed-by: Chris Wilson 
    [danvet: Dropped a bogus ~ from the commit message that somehow crept
    in.]
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/intel_display.c |    3 ---
 1 file changed, 3 deletions(-)