11 April 2013

384. Another (vaguely) TeX-related headache

The error
ps2pdf wasn't working on one of my .ps files (generated from a .tex file by texmaker). When I try to print the .ps file, it also shows up as having 197 pages, instead of 85 pages, which is what is has and what is seen in e.g. evince.

ps2pdf gives
Error: /syntaxerror in -file-
Operand stack:

Execution stack:
   %interp_exit   .runexec2   --nostringval--   --nostringval--   --nostringval--   2   %stopped_push   --nostringval--   --nostringval--   --nostringval--   false   1   %stopped_push   1910   1   3   %oparray_pop   1909   1   3   %oparray_pop   1893   1   3   %oparray_pop   1787   1   3   %oparray_pop   --nostringval--   %errorexec_pop   .runexec2   --nostringval--   --nostringval--   --nostringval--   2   %stopped_push
Dictionary stack:
   --dict:1164/1684(ro)(G)--   --dict:4/20(G)--   --dict:101/200(L)--   --dict:215/300(L)--   --dict:90/200(L)--   --dict:57/75(L)--   --dict:211/313(L)--   --dict:72/140(L)--   --dict:0/10(G)--   --dict:0/10(L)--   --dict:6/50(ro)(G)--   --dict:56/71(L)--
Current allocation mode is local
Last OS error: No such file or directory
Current file position is 86139456
GPL Ghostscript 9.05: Unrecoverable error, exit code 1

And I have no idea where to start troubleshooting...the file position error is not terribly helpful given that the file only has 2010146 lines. So I commented everything, then slowly uncommented everything.

And it was a bit more complex than I thought -- this is the smallest section of code I could use to reproduce the error (note that the two frames are about 30 pages apart in the original):


[preamble]
\begin{document}
\begin{frame}{A frame}
\begin{table}
 \begin{tabular}{l r}
 \includegraphics[width=4cm]{images/figure_1.eps} & \pause
 \includegraphics[width=3.5cm]{img/folder_1/figure_2.eps} 
 \end{tabular}
\end{table}
\small{
\begin{itemize}
\item Blah blah blah
\item Blah Blah?
\end{itemize}
}
\end{frame}

\begin{frame}{Another frame}
 \small{Blah blahdi blah}\\
 \begin{center}
 \begin{table}
 \begin{tabular}{c c}
 \includegraphics[width=5cm]{img/folder_1/figure_3.eps} &
 \includegraphics[width=5cm]{img/folder_1/figure_4.eps} \\
 {\sc Caption} & Caption
 \end{tabular}
 \end{table}
 \end{center}
\end{frame}

\end{document}

The error was only apparent if both figure_2.eps and figure_3.eps were requested -- somehow those two files didn't like each other. I had made figure_2.eps myself (povray -> png -> eps), but a collaborator had sent me figure_3.eps, which was made using Bruker Topspin (a stacked plot of NMR spectra)

The fix:
I fixed it by opening figure_3.eps in inkscape and saving it again

bad.eps is figure_3 before re-saving it, good.eps is figure_3 after re-saving it:

me@beryllium:~/work/manuscripts/Talks/Australie/img/folder_1$ head -n 20 bad.eps 
���� X�+x�+ ��%!PS-Adobe-3.1 EPSF-3.0 %ADO_DSC_Encoding: Windows Roman %%Title: 72_tempbw.eps %%Creator: Adobe Illustrator(R) 15.1 %%For: Collaborator %%CreationDate: 9/12/2012 %%BoundingBox: 0 0 633 359 %%HiResBoundingBox: 0 0 632.8331 358.5000 %%CropBox: 0 0 632.8331 358.5000 %%LanguageLevel: 2 %%DocumentData: Clean7Bit %ADOBeginClientInjection: DocumentHeader "AI11EPS" %ADO_ContainsXMP: MainFirstustrator(R) 15.1.0 x39 R agm 4.5189 ct 5.1875 %ADOEndClientInjection: DocumentHeader "AI11EPS" %%Pages: 1 %%DocumentNeededResources: %%DocumentSuppliedResources: procset Adobe_AGM_Image 1.0 0 %%+ procset Adobe_CoolType_Utility_T42 1.0 0 %%+ procset Adobe_CoolType_Utility_MAKEOCF 1.23 0 %%+ procset Adobe_CoolType_Core 2.31 0
me@beryllium:~/work/manuscripts/Talks/Australia/img/folder_1$ head -n 20 good.eps
%!PS-Adobe-3.0 EPSF-3.0 %%Creator: cairo 1.12.2 (http://cairographics.org) %%CreationDate: Thu Apr 11 11:36:28 2013 %%Pages: 1 %%DocumentData: Clean7Bit %%LanguageLevel: 2 %%BoundingBox: 0 -1 633 359 %%EndComments %%BeginProlog save 50 dict begin /q { gsave } bind def /Q { grestore } bind def /cm { 6 array astore concat } bind def /w { setlinewidth } bind def /J { setlinecap } bind def /j { setlinejoin } bind def /M { setmiterlimit } bind def /d { setdash } bind def /m { moveto } bind def
I had another couple of files which caused the same issue -- all of which originally came from Bruker Topspin (XWinNMR). Re-saving them in inkscape solved the issues.

I still don't understand why figure_1 had to be present though.

09 April 2013

382. Briefly: fixing pdfstudio's leftover crud

I didn't like pdfstudio (http://verahill.blogspot.com.au/2013/02/338-annotating-pdfs-in-linux-revisited.html), not least because it's stuck in a microsoft world of 'basic' and 'pro' versions i.e. artificial crippling. I promptly got rid of it, but it has been lingering on my system like a bad relationship:
dpkg: warning: parsing file '/var/lib/dpkg/available' near line 94935 package 'pdfstudio': missing description

The simplest, most radical fix is to do
sudo dpkg --clear-avail
sudo apt-get update

08 April 2013

381. Encrypting chat, voice, video-- revisited

About a year ago I published this post:
128. Encrypting your email, chat and VOIP in linux (Debian Wheezy)

While I'm using gajim for encrypted chatting on a daily basis, always sign my emails with PGP/GPG in thunderbird and evolution, and keep a truecrypt container in my dropbox account, I've noticed that I don't ever use twinkle, which was the voice encryption solution I looked at in post 128.

The time is thus ripe to explore alternatives. Also, seahorse and evolution have changed enough that updated screenshots may be warranted. So here's Encryption for 2013. You will find that the most difficult thing  about encryption is to convince other people to use it as well.

The tl;dr version is: use gajim. It's awesome.


--- Beginning of the boring bits ---

Why?
I'm not actually paranoid, but what about

*'If you don't have anything to hide, then what are you worrying about?' ? The issue with that kind of reasoning is that it can twisted into meaning that if you do worry, then you do have something to hide, and  mere possession of  will imply guilt. Presumably, if people in general are using e.g. PGP, the risk of it being banned or restricted (or being enforceable) is smaller. See e.g. this if you're living in the US.

* What happens if you gmail account gets hacked? Could someone glean enough information from your conversations and email to steal your identity?

* Do you need to send someone your PIN, bank card info or SSN but don't necessarily feel

* Do you need to discuss something but keep it confidential?

* It's fun! Being able to encrypt gives you a bit of a power trip.

What does annoy me are people who think that they aren't targets because 'why would anyone want to hack me'? Low-profile individuals are rarely targets in their own right, but what about the company you are working for? What about friends of yours? What about the possibility of opportunistic drive-by hacking where targets aren't selected by who they are, but by lapses in security?

In my case, I work for a large university -- gaining access to the intranet for an outsider would only be the first step in potentially compromising thousands of computers or accessing the personal information of tens of thousands of people (has happened at every uni I've worked). I manage servers at two universities -- if they get compromised their access to high speed internet could be used for spam routing (has also happened).

 I have friends who work with classified projects presumably involving nuclear weapons (nat'l lab in the US) -- I might be an intermediary target.

 I have a credit card, and a credit history. What if someone steals my identity and takes out a loan in my name? (has happened)

Or what if I have confidential data on a USB stick and I lose it -- if it's encrypted damage control is a lot easier (has happened at my uni).

Again, I'm not really paranoid, but there is always a tiny risk that someone, for whatever reason, is targeting you.

OTR and GPG/PGP
Note that I am NOT an expert on this. I've used PGP a lot, but have only recently been introduced to OTR. Also, merely being a user of something doesn't make an expert at it. What I write might be wrong.

PGP (pretty good privacy)/GPG (it's open sourced implementation) is a proven encryption method which uses asymmetric encryption with a private and a public key. You use someone else's public key to encrypt messages to them, and only their private key can decrypt it. Likewise, you give your public key to someone who wants to email you -- they use that to encrypt messages to you, and you -- and only you -- can decrypt it with your private key. PGP also supports signing to verify authorship -- you can sign (not encrypt) a message using your private key, and then anyone with your public key can decrypt it.

It's worth emphasizing this: if you encrypt with your private key, anyone in the world can decrypt it.

Whenever you generate a new key pair, you can publish the public key on a range of online repositories, such as http://pgp.mit.edu/ I think seahorse (gnome, evolution) and enigmail (thunderbird) can automatically search public key servers for public keys.

PGP is in principle a very strong encryption method using a 2048 byte (or larger) key. However, it suffers from a major drawback:  anyone who can get their hands on your private key can decrypt everything. That basically means that anyone who gets access to your unencrypted harddrive can copy your key and eavesdrop. The longer you use the same key, the more likely it is that this will already have happened, especially if you have traveled through the US and been forced to give them access to your laptop. Leaving your laptop in your hotel room -- especially in a place like China (although it goes for anywhere) -- is probably also not advisable. Since you can't keep an eye on your hardware 24/7, this is a serious issue.

Another potential issue is that if you sign conversation with a key, then that conversation can be attributed to you i.e. a third, eavesdropping party may be able to ascertain who the conversing parties are.

Again, emphasis: your conversation may be safe now, but if in three months your key is lost all your conversations for years will be decrypted.

You can deal with this to some extent by either encrypting your entire harddrive, or by keeping your private key on an encrypted USB stick (be mindful of the increased risk of misplacing your usb stick and other people gaining access to it -- any encryption can generally be broken, given enough time and resources).

My main use of PGP/GPG is to have candid discussions via chat, and to sign my email.

OTR (Off the Record) is a 'new' (since 2004) alternative and is described in (accessible) detail here: http://www.cypherpunks.ca/otr/otr-wpes.pdf

It aims to solve a lot of the issues with PGP by doing away with a long-lived key. It also aims to make it impossible for a third party to identify the conversing parties. From what I understand it works more or less like this:

If we decide to chat, I generate a key,and you generate a key. Once we're done talking, we dispose of the keys. No-one --- not even the conversing parties -- can decrypt the conversation, which also ensures that nothing will surface by accident at a later date. PGP can obviously be used in the same manner, but it's the implementation of it which has to be convenient. Obviously, in practice this works a whole lot better for chat than email.

Anyway, from my reading of it OTR works pretty much the same as PGP (using large primes and public/private keys) -- it's just the actual implementation which differs. On starting the chat you click on a button to generate your key pair, and the public key gets sent to your chat partner and vice versa. Exit the chat session and your keys are gone. OTR also uses authentication of messages to ensure authorship i.e. even though a man in the middle can encrypt messages to you using your public key and pretend that they are written by your conversation partner, the lack of the correct authentication signature will alert you to the fact that someone is spoofing the messages. PGP does the same by signing with the private key. There's no way to prove authorship based on the authentication signature though -- all you can do is determine that a series of messages came from the same author, but there's nothing which intrinsically proves who that person is. With PGP, on the other hand, if the assumption is that the private keys of the conversing parties have not been compromised, then they both are who they claim they are (of course you might've been duped into using someone else's public key and then all bets are off).

Anyway, OTR improves on PGP and works well for chat. PGP is probably still the easier solution for email.

SRTP (secure real-time transport protocol)
I don't know much about this, and am heavily relying on the Wikipedia page.
SRTP is an encryption method for streamed data, which I read to include voice and video. It needs a key management protocol, such as ZRTP. The way keys are managed depend on the protocol. As far as I understand ZRTP uses temporary asymmetric keys which are exchanged at the beginning of the encryption part of the conversation, similar to OTR, but includes one major difference: a call begins with a verbal exchange of a specific value displayed in the clients of the two parties conversing. This is to avoid that there's a man in the middle attack -- you will presumably recognise the voice of the person you're talking to and be able to verify the identity that way.

I won't cover data encryption using e.g. Truecrypt, in particular, since I've covered it in detail before, e.g. here: http://verahill.blogspot.com.au/2012/04/using-truecrypt-with-dropbox.html

Solutions that I won't consider:
While e.g. skype uses encryption, you're not in control. Also, it's not open source, so who knows how it works

While you using https and google talk in a browser encrypts you video/voice calls and chat, you're not in control -- google can decrypt your conversation any time.

--- End of the boring bits ---


PGP/GPG keypair
The easiest way to do it is probably via the seahorse GUI.
sudo apt-get install gnupg seahorse

Start seahorse (known as Passwords and Keys in Gnome 3).


Once you've made a key pair, synchronise it with a server so that other people can get your public key, e.g.


Alternatively, you can do everything on the command line:
gpg --gen-key
gpg (GnuPG) 1.4.12; Copyright (C) 2012 Free Software Foundation, Inc. [..] Please select what kind of key you want: (1) RSA and RSA (default) (2) DSA and Elgamal (3) DSA (sign only) (4) RSA (sign only) Your selection? 1 RSA keys may be between 1024 and 4096 bits long. What keysize do you want? (2048) Requested keysize is 2048 bits Please specify how long the key should be valid. 0 = key does not expire = key expires in n days w = key expires in n weeks m = key expires in n months y = key expires in n years Key is valid for? (0) 0 Key does not expire at all Is this correct? (y/N) Y You need a user ID to identify your key; the software constructs the user ID from the Real Name, Comment and Email Address in this form: "Heinrich Heine (Der Dichter) " Real name: I Lindqvist Email address: i.lindqvist@email.net Comment: fake address You selected this USER-ID: "I Lindqvist (fake address) " Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O You need a Passphrase to protect your secret key.
You'll get asked for a passphrase twice. Then:
We need to generate a lot of random bytes. It is a good idea to perform some other action (type on the keyboard, move the mouse, utilize the disks) during the prime generation; this gives the random number generator a better chance to gain enough entropy. Not enough random bytes available. Please do some other work to give the OS a chance to collect more entropy! (Need 283 more bytes) ..+++++ ...+++++ We need to generate a lot of random bytes. It is a good idea to perform some other action (type on the keyboard, move the mouse, utilize the disks) during the prime generation; this gives the random number generator a better chance to gain enough entropy. +++++ Not enough random bytes available. Please do some other work to give the OS a chance to collect more entropy! (Need 109 more bytes) .....................+++++ gpg: key 2B4C5636 marked as ultimately trusted public and secret key created and signed. gpg: checking the trustdb gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u pub 2048R/2B4C5636 2012-05-02 Key fingerprint = 5B71 C3F1 0C2D E008 B299 21A8 019F 907E 2B4C 5636 uid I Lindqvist (fake address) sub 2048R/78F9B6C1 2012-05-02
gpg --list-key
/home/me/.gnupg/pubring.gpg ----------------------------- pub 2048R/2B4C5636 2012-05-02 uid I Lindqvist (fake address) sub 2048R/78F9B6C1 2012-05-02
To add more email addresses, do
gpg --edit-key 2B4C5636
>adduid


and follow the prompts. You do not want to add subkeys. Do
>trust

to set the trust level. Ultimate for your own key, full for your pals.

Implementation

Chat:

Gajim 
Gajim supports both PGP (natively) and OTR (via a plug in). It's also easy to use and written in python, which is a plus.

To enable PGP/GPG, go to Edit, Accounts, Personal Information, Choose Key:

You need to assign the public key to the contacts you want to chat with. First get the public key -- either from an online repo or by email from your contact. In e.g. seahorse, go to Remote, Find Remote Keys and search:




Then, in gajim, do


To enable OTR, go to Edit, Plugins, click on the Available tab and wait a little while for the list to populate:





To use OTR, do the following:
Start chat, click on the settings icon, select off-the-record, and start

It's now encrypted

To verify the identity of the other party, click on Authenticate contact

Ask a question

And the other person answers

Authentication achieved

Alternatives
mcabber also support PGP/GPG, and is a curses based program. See here for configuration.

pidgin and jitsi both support OTR, but not PGP.

Pidgin and mcabber are in the debian repos.


Email:

Evolution:
Enabling PGP/GPG is very easy -- edit your account, and go to the Security tab:
To encrypt, start a new message and click on Options. You might want to have html turned off.
Thunderbird:
You'll need to install enigmail.

 You can select your key etc. under key management.

Alternatives
Mutt does PGP/GPG and is a neat curses-based program.


Z/SRTP in Voice

There are a couple of solutions for 'secure' voice and video. I looked at Twinkle before, and while it's a capable SIP program, the downside is that it requires access to a SIP service (e.g. Ekiga, or via your ISP). What is really needed are clients which can encrypt communication going via e.g. google talk/jabber and the likes. Jitsi does that, and is open source.

I won't consider Skype for the usual reasons -- it's not free, open or transparent.

Jitsi
Get jitsi by doing
wget https://download.jitsi.org/jitsi/debian/jitsi_1.0-latest_amd64.deb
sudo dpkg -i jitsi_2.0-latest_amd64.deb

Start Jitsi and set up an account. You can add more accounts later by going to Options and clicking the Accounts tab.

Anyway, start Jitsi, and go to Options, Security to get an overview. Here's the ZRTP tab. Basically you CAN change the parameters, but you're advised not to:

Jitsi also does OTR chat:

And you can set a master password, so that people can't start jitsi if you leave your desktop unprotected.

You don't need to do anything to enable ZRTP as it will automatically be used if both your and your partner's clients support it, so here's OTR in Jitsi:





380. Modifying NWChem code without a full recompile

This isn't a secret trick or anything, but may still not be immediately obvious to most people.

A full compilation of NWChem can easily take 20-40 minutes, depending on your build machine.

Sometimes you need to make changes to the source code, e.g. if you want to use GabEdit to analyse output -- ECCE is a fantastic piece of software and is great for managing computations, but GabEdit has implemented some pretty interesting routines for orbital analyses. In order for you to be able to reliably import NWChem output into GabEdit you need to modify a handful of fortran files.

See e.g. http://verahill.blogspot.com.au/2013/02/3xx-modifying-nwchem-611-to-work-with.html for patching NWChem, and http://verahill.blogspot.com.au/2012/11/visualising-nwchem-output-with-gabedit.html for how to run your nwchem jobs and how to visualize them in gabEdit (you need to explicitly define your basis sets).

Angelo Rossi made this comment:
Hello:

Thank you so much for this valuable information.

But under the heading of "Compilation" above, the directions lead to a pointer to compile the entire NWChem source. But only one or two subroutines are modified. Shouldn't there be a more surgical way of proceeding after making the suggested changes? Actually this would make a great separate post. That is, provide a procedure to recompile and link when small changes to NWChem are made. I've done this, but I can't remember.

Kind regards,

Angelo

The answer was on this page: http://xray.isc.kharkov.com/ext_docs/NWChem/prog/node12.html
So here we go:

Partial recompile/relinking of NWChem

1. Environmental variables
Define your environmental variable like you would during a normal compile
export LARGE_FILES=TRUE export TCGRSH=/usr/bin/ssh export NWCHEM_TOP=`pwd` export NWCHEM_TARGET=LINUX64 export NWCHEM_MODULES="all python" export PYTHONVERSION=2.7 export PYTHONHOME=/usr export BLASOPT="-L/opt/openblas/lib -lopenblas" export USE_MPI=y export USE_MPIF=y export USE_MPIF4=y export MPI_LOC=/usr/lib/openmpi/lib export MPI_INCLUDE=/usr/lib/openmpi/include export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openmpi/lib:/opt/acml/acml5.2.0/gfortran64_int64/lib:/opt/openblas/lib export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread" export FC=gfortran
2. Run make in modified directories
In the GabEdit example we made changes to files in src/ddscf, src/nwdft/scf_dft, and src/mcscf
cd $NWCHEM_TOP/src/ddscf
make
cd $NWCHEM_TOP/src/nwdft/scf_dft
make
cd $NWCHEM_TOP/src/mcscf
make

3. Run make link in the src directory
cd $NWCHEM_TOP/src
make link

4. Do your usual post-compilation memory magic
cd $NWCHEM_TOP/contrib
./getmem.nwchem

Done!

This all in all took around 6 seconds instead of 30-odd minutes.

06 April 2013

379. [Solved] Debian Wheezy -- Thinkpad SL410: reboot instead of shutdown when on battery

Solved: it was down to laptop-mode-tools, which I installed as part of this post two weeks ago. I uninstalled that package and my laptop is shutting down properly again.

Original post:
Since today[*], with Debian Wheezy stock kernel (3.2), my Thinkpad SL410 reboots instead of shutting down when it's running on battery. Works fine when on mains power. During shutdown it says 'Will now halt', then reboots.

Similar to what's seen in this thread: http://forums.debian.net/viewtopic.php?f=7&t=86316&start=15, but shutting down X seems to have no effect. Googling seems to yield a lot of hits for thinkpads and ideapads.

Annoying and no idea why.

The support for SL410 under Debian Wheezy has actually gotten worse during the past year -- now 'mute' doesn't work anymore, and from today it doesn't shutdown on issuing e.g. sudo shutdown -h now.Finally, recently the behaviour of the kernel (3.8) has lead to slowdowns on computer with i915 (intel) graphics, which would include a majority of laptops (a fix is on the way though).

Looking at /var/log/apt/history.log my kernel was updated (not a new version, so a bit confused), two days ago. Nothing else looks important enough.

[*] Given that removing laptop-mode-tools solved it, and that I installed that on the 23rd of March, I must've been running my laptop off of mains power for the past two weeks...alternatively, newer kernels fix this (supposed to be due to the combination of laptop-mode-tools and an apci bug in the kernel) since I've been running 3.7.10 for the past two weeks, and only switched back to 3.2 yesterday.


378. Briefly: Windows/Dialogues that are too large for a laptop screen

Short post which is probably just revealing something which a lot of people are already familiar with.

A really annoying thing with Evolution (and ECCE as well, actually) is that it doesn't play well with notebook screen. I have a Lenovo Thinkpad SL410 which has a 1366x768 screen and this is what I see when e.g. editing an account in Evolution:

Note the lack of buttons to click -- the Close and Apply dialogues don't show on the screen

I might be imagining, but I get the impression that GNOME projects don't like you to resize windows -- and you can't resize this dialogue.

The workaround:
In gnome, use alt+click and drag to move the window so that the buttons are revealed.


05 April 2013

377. Wine 1.5.27 and Office 2003: Equation editor now supported!

I prefer using TeX, but collaboration is a reality of academia, and sometimes it's just easier to shut up and use Word.

To install Office 2003 under Wine on Debian, see here: http://verahill.blogspot.com.au/2013/01/307-installing-office-2003-in-wine-on.html


In post 376 I compiled and installed wine 1.5.27. Starting word under wine I noticed that an old mathtype install was now recognised (if not completely functional), and that gave me hope that Equation Editor would be installable.




Equation Editor
Here's how to get it up and running

1. Control panel
If you've already installed Office under Wine, try using add/remove programs:
wine control

and click on Add/Remove programs, then modify your Office installation. That didn't work for me since my Office install wasn't listed for some reason.

If that doesn't work, then
2. (Re-)Install
sudo mount -o loop OFFICE11.iso /mnt
wine msiexec /i /mnt/pro11.msi

And start the installation.






Start word, go to Insert/Objects




376. Wine 1.5.27 on Debian: i386 chroot

UPDATE 16 May 2013: See here for Wine 1.5.30: http://verahill.blogspot.com.au/2013/05/416-wine-1530-in-chroot.html

Note: Please, if you do have issues actually using the wine binaries built this way, post a comment -- everything is working fine on my systems but I'd like to know exactly what libraries you need to install on the target system to make Wine built in the chroot run to satisfaction.


Preamble: Building wine on Debian Wheezy amd 64 is a PITA at the moment, since some amd64 -dev packages are not co-installable with their i386 counterpart. Such an example is libfreetype6-dev:i386.


If I'm understanding this post correctly I may be confusing the idea behind multiarch vs cross-compiling:

"multiarch provides support for cross-compiling, but we don't have an amd64->i386 cross-compiler - only a biarch (multilib) compiler."

which I interpret to mean that while we generally can compile against 32 bit libs in wheezy using multiarch, it's not necessarily the way it is intended to work (I've never bother to check whether the binaries are 32 bit, or just linked against 32 bit libs). Instead, an alternative way to look at multiarch is that it supports the use of packages linked against 32 bit libs (and puts libraries in arch dependent locations), but not by necessity the compilation of 32 bit binaries (if it does, great). Or maybe I'm wrong. The bottom line is that we might have to accept compiling in a chroot, and then install the 32 bit libs on the target machine to support execution.
I've built wine using chroot in the past, but it was always with the intention of hammering out dependencies.


Getting started:

The Chroot
mkdir $HOME/tmp/architectures/wine32
cd $HOME/tmp/architectures
sudo debootstrap --arch i386 wheezy $HOME/tmp/architectures/wine32 http://ftp.au.debian.org/debian/

sudo mount -o bind /proc wine32/proc
sudo cp /etc/resolv.conf wine32/etc/resolv.conf
sudo chroot wine32

You're now in the chroot:
apt-get update
apt-get install locales sudo vim
echo 'export LC_ALL="C"'>>/etc/bash.bashrc
echo 'export LANG="C"'>>/etc/bash.bashrc
echo '127.0.0.1 localhost beryllium' >> /etc/hosts
source /etc/bash.bashrc
adduser sandbox
usermod -g sudo sandbox
echo 'Defaults !tty_tickets' >> /etc/sudoers
su sandbox
cd ~/

Building Wine
While still in the chroot, continue (the i386 is ok; don't worry about it):
sudo apt-get install libx11-dev:i386 libfreetype6-dev:i386 libxcursor-dev:i386 libxi-dev:i386 libxxf86vm-dev:i386 libxrandr-dev:i386 libxinerama-dev:i386 libxcomposite-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libdbus-1-dev:i386 libgnutls-dev:i386 libncurses-dev:i386 libsane-dev:i386 libv4l-dev:i386 libgphoto2-2-dev:i386 liblcms-dev:i386 libgstreamer-plugins-base0.10-dev:i386 libcapi20-dev:i386 libcups2-dev:i386 libfontconfig-dev:i386 libgsm1-dev:i386 libtiff-dev:i386 libpng-dev:i386 libjpeg-dev:i386 libmpg123-dev:i386 libopenal-dev:i386 libldap-dev:i386 libxrender-dev:i386 libxml2-dev:i386 libxslt-dev:i386 libhal-dev:i386 libcurl4-openssl-dev:i386 gettext:i386 prelink:i386 bzip2:i386 bison:i386 flex:i386 oss4-dev:i386 checkinstall:i386 ocl-icd-libopencl1:i386 opencl-headers:i386 libasound2-dev:i386 build-essential
mkdir ~/tmp
cd ~/tmp
wget http://prdownloads.sourceforge.net/wine/wine-1.5.27.tar.bz2
cd wine-1.5.27/
./configure
time make -j2
sudo checkinstall --install=no
checkinstall 1.6.2, Copyright 2009 Felipe Eduardo Sanchez Diaz Duran This software is released under the GNU GPL. The package documentation directory ./doc-pak does not exist. Should I create a default set of package docs? [y]: Preparing package documentation...OK Please write a description for the package. End your description with an empty line or EOF. >> wine 1.5.27 >> ***************************************** **** Debian package creation selected *** ***************************************** This package will be built according to these values: 0 - Maintainer: [ root@beryllium ] 1 - Summary: [ wine 1.5.27 ] 2 - Name: [ wine ] 3 - Version: [ 1.5.27 ] 4 - Release: [ 1 ] 5 - License: [ GPL ] 6 - Group: [ checkinstall ] 7 - Architecture: [ i386 ] 8 - Source location: [ wine-1.5.27 ] 9 - Alternate source location: [ ] 10 - Requires: [ ] 11 - Provides: [ wine ] 12 - Conflicts: [ ] 13 - Replaces: [ ]

Compilation took ca 20 minutes. Checkinstall takes a little while (In particular this step: 'Copying files to the temporary directory...').

Installing Wine
Exit the chroot
sandbox@beryllium:~/tmp/wine-1.5.27$ exit
exit
root@beryllium:/# exit
exit
me@beryllium:~/tmp/architectures$ 

On your host system
Enable multiarch and install ia32-libs, since you've built a proper 32 bit binary:
sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install ia32-libs

Copy the .deb package and install it
sudo cp wine32/home/sandbox/tmp/wine-1.5.27/wine_1.5.27-1_i386.deb .
sudo chown $USER wine_1.5.27-1_i386.deb
sudo dpkg -i wine_1.5.27-1_i386.deb

The tricky bit
The question is what 32 bit libs you need on the host system, if any.
ldd /usr/local/bin/wine
linux-gate.so.1 => (0x55573000) libwine.so.1 => /usr/local/bin/../lib/libwine.so.1 (0x55577000) libpthread.so.0 => /lib/i386-linux-gnu/i686/cmov/libpthread.so.0 (0x556f5000) libc.so.6 => /lib/i386-linux-gnu/i686/cmov/libc.so.6 (0x5570e000) libdl.so.2 => /lib/i386-linux-gnu/i686/cmov/libdl.so.2 (0x55871000) /lib/ld-linux.so.2 (0x55555000)
So far everything is working fine with ia32-libs.

Links to this post:
http://forums.bistudio.com/showthread.php?149892-ARMA-III-on-Linux-servers-via-WINE%2Fpage4
http://forum.winehq.org/viewtopic.php?f=8&t=19010

04 April 2013

375. Bisecting the kernel -- looking for the commit behind the kworker/i915 issue.

This is an attempt to trace this issue: http://verahill.blogspot.com.au/2013/03/368-slow-mouse-and-keyboard-triggered.html.

See also
http://forums.gentoo.org/viewtopic-p-7278760.html
https://bbs.archlinux.org/viewtopic.php?pid=1248190

Update: I'm revising this post. Not quite done yet.

Another Update: Someone else has just bisected and ended up with a different commit:
https://bbs.archlinux.org/viewtopic.php?pid=1254285#p1254285
The patch they found makes a lot more sense than any of the ones I ended up with.

However, I do still see issues with the kernel I end up with, so I'm either starting from the wrong 'good', or there are several bad commits.

A third update: I just experienced slow-down with 1k interrupts per second on 3.7.10, which may indicate that I picked a bad starting point for bisecting. I'll leave the post up since it can serve as a guide for how to bisect in general.

A fourth update:
A fix is on the way to kernel 3.9 or 3.10.
https://patchwork.kernel.org/patch/2400621/
https://patchwork.kernel.org/patch/2402211/


Original post:

I didn't really want to do this, for several reasons:

* the bug only manifests itself on one of my computers, with intel graphics -- and that's a laptop (I'm not a huge fan of laptops, especially not with budding carpal tunnel syndrome, so testing the bisected version is a bit of a pain -- literally)

* the bug isn't consistently triggered, so you need to test the new kernel for an hour or more, and only by the absence of a specific behaviour can you see whether all is good i.e. testing is a bit of hit and miss.

* compiling the kernel takes a long time, and I don't currently have a suitable computer for it at home where my laptop (the one with the i915) is, so I'll build the kernel on a work computer, then install it at home, then repeat each day.

On the other hand, it's a learning experience, so here we go.

I'm looking at https://www.kernel.org/pub/software/scm/git/docs/git-bisect.html

I'm compiling the kernel debian style since it generates .deb files which are easy to install on other machines. Again, there are posts on how to compile the kernel using a more generic approach, or specifically for Arch, elsewhere on this blog.

NOTE: there's no sure-fire way of triggering this issue, but it seems to occur more frequently when the fan is on but approaching the thermal cut-off point where it will turn off. This makes troubleshooting somewhat more difficult.


1. Download/checkout
First get the sources -- it pulls everything from 2.6.12, so it's bigger than a normal kernel source download. I'll skip explicitly telling you what packages you need to have in order to compile the kernel or use git, since if you don't know, you're probably not really ready for this anyway. Besides, that information is available in other posts on this blog.

mkdir -p ~/tmp/kernel_bisect
cd ~/tmp/kernel_bisect/
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git linux-git
Cloning into 'linux-git'... remote: Counting objects: 2942099, done. remote: Compressing objects: 100% (443946/443946), done. remote: Total 2942099 (delta 2471580), reused 2940624 (delta 2470325) Receiving objects: 100% (2942099/2942099), 608.66 MiB | 4.58 MiB/s, done. Resolving deltas: 100% (2471580/2471580), done. Checking out files: 100% (42425/42425), done.
cd linux-git/


2A. Bisect
I know that 3.8.5 is bad and that 3.7.10 may be good (seem to remember having issues with the Arch version of 3.7.10, but no issues with my own build on debian. Not sure I can trust my memory).

I know for a fact that 3.2 is good. I don't want to bisect everything from 3.8.5 to 3.2 though, so I'll take a leap of faith and presume that 3.7.10 is good (which I'll regret if that's not true) and 3.8.0 is bad.

I know that the issue is present on amd64.

I think it's an issue to do with i915.

git bisect start -- arch/amd64 drivers/gpu/drm/i915
git bisect bad v3.8
git bisect good v3.7
Bisecting: 160 revisions left to test after this (roughly 7 steps) [28d491df4c6b00f9148a9885dba1f36a078535dc] drm/i915: Bad pixel formats can't reach the sprite code
2B. Compile
I'm being a bit lazy here as well, possibly at the cost of not being able to identify my issue: I'll use silentoldconfig to accept all the default settings.
make oldconfig
time fakeroot make-kpkg -j3 --initrd kernel_image kernel_headers

where 3 in -j3 is the number of cores (see here and here and here for more about the proper value for -j) -- I have a triple core AMD. It took around 43 minutes to build.

This will generate two .deb files in the parent folder -- one kernel image and one with headers.
Install these on your testing system.


sudo dpkg -i ../linux-image-3.7.0-rc2+_3.7.0-rc2+-10.00.Custom_amd64.deb ../linux-headers-3.7.0-rc2+_3.7.0-rc2+-10.00.Custom_amd64.deb

Since you installed .deb packages it's pretty easy to roll back any changes later.

3. Testing
What happens now depends on whether the kernel is plagued by the issue you're trying to bisect for:

3A. The problem persists
You've now narrowed it down to half as many commits.
git bisect bad
make-kpkg clean
make oldconfig
time fakeroot make-kpkg -j3 --initrd kernel_image kernel_headers
sudo dpkg -i ../linux-image-3.7.0-rc2+_3.7.0-rc2+-10.00.Custom_amd64.deb ../linux-headers-3.7.0-rc2+_3.7.0-rc2+-10.00.Custom_amd64.deb


3B. The problem is not present
You've now narrowed it down to half as many commits.
git bisect good
make-kpkg clean
make oldconfig
time fakeroot make-kpkg -j3 --initrd kernel_image kernel_headers
sudo dpkg -i ../linux-image-3.7.0-rc4+_3.7.0-rc4+-10.00.Custom_amd64.deb ../linux-headers-3.7.0-rc4+_3.7.0-rc4+-10.00.Custom_amd64.deb

Repeat step 3 until you've isolated the commit that caused the issue.

For me
* the first bisect didn't cause kworker slow downs (bisect good)
* the second bisect immediately led to slowdown (bisect bad)
* the third one was fine (bisect good)
* the fourth one had no issues (bisect good)
* the fifth one was good (bisect good)
* the sixth one was definitely bad (bisect bad)

powertop gives the following when it's slowing down
 0 mW    423.5 ms/s       4.5        kWork          i915_hotplug_work_func

and watch cat /proc/interrupts:
The number of interrupts increases very, very rapidly for '49:  PCI-MSI-edge i915' during mouse slowdown. Normal rate is around 70 per two seconds.
This is where it gets less clear -- from what I can see:
* the seventh one was a little bit bad (bisect bad).
* the eight one had minor issues when the fan went off (bisect bad).

But the first time I did the bisect I came to the exact opposite conclusion for the last two bisects. I'm suspecting that it's really down to two or more commits that together cause bad behavior, but on their own are merely annoying.

Anyway, I've ended up with this commit as the current culprit (again, the first time around I ended up with a different commit):
607a6f7a6621f65706ff536b2615ee65b5c2f575 is the first bad commit
commit 607a6f7a6621f65706ff536b2615ee65b5c2f575
Author: Daniel Vetter 
Date:   Wed Nov 14 17:47:39 2012 +0100

    drm/i915: drop buggy write to FDI_RX_CHICKEN register
    
    Jani Nikula noticed that the parentheses are wrong and we & the bit
    with the register address instead of the read-back value. He sent a
    patch to correct that.
    
    On second look, we write the same register in the previous line, and
    the w/a seems to be to set FDI_RX_PHASE_SYNC_POINTER_OVR to enable the
    logic, then keep always set FDI_RX_PHASE_SYNC_POINTER_OVR and toggle
    FDI_RX_PHASE_SYNC_POINTER_EN before/after enabling the pc transcoder.
    
    So the right things seems to be to simply kill the 2nd write.
    
    Cc: Jani Nikula 
    Reviewed-by: Chris Wilson 
    [danvet: Dropped a bogus ~ from the commit message that somehow crept
    in.]
    Signed-off-by: Daniel Vetter 

:040000 040000 f789c6c199c9db5c9d0d7961760574b5f0b1ede9 9e0cd2a09cab610b437164b1a74f620e43686ef1 M      drivers

It's just really difficult to reproduce the issue consistently with the last couple of kernels. I am in no way confident that the above commit is what's causing all this.

The last confirmed troublesome bisect (#6):

Here's the log:
git bisect start '--' 'arch/amd64' 'drivers/gpu/drm/i915'
# bad: [19f949f52599ba7c3f67a5897ac6be14bfcb1200] Linux 3.8
git bisect bad 19f949f52599ba7c3f67a5897ac6be14bfcb1200
# good: [29594404d7fe73cd80eaa4ee8c43dcc53970c60e] Linux 3.7
git bisect good 29594404d7fe73cd80eaa4ee8c43dcc53970c60e
# good: [28d491df4c6b00f9148a9885dba1f36a078535dc] drm/i915: Bad pixel formats can't reach the sprite code
git bisect good 28d491df4c6b00f9148a9885dba1f36a078535dc
# bad: [b4a98e57fc27854b5938fc8b08b68e5e68b91e1f] drm/i915: Flush outstanding unpin tasks before pageflipping
git bisect bad b4a98e57fc27854b5938fc8b08b68e5e68b91e1f
# good: [12f3382bc0262e981a2e58aca900cbbdbbe66825] drm/i915: implement WaDisablePSDDualDispatchEnable on IVB & VLV
git bisect good 12f3382bc0262e981a2e58aca900cbbdbbe66825
# good: [b9e0bda3cd325b55f336efb751736163f62abded] drm/i915: Always calculate 8xx WM values based on a 32-bpp framebuffer
git bisect good b9e0bda3cd325b55f336efb751736163f62abded
# good: [1c8b46fc8c865189f562c9ab163d63863759712f] drm/i915: Use LRI to update the semaphore registers
git bisect good 1c8b46fc8c865189f562c9ab163d63863759712f

and here are the remaining commits:


commit b4a98e57fc27854b5938fc8b08b68e5e68b91e1f
Author: Chris Wilson 
Date:   Thu Nov 1 09:26:26 2012 +0000

    drm/i915: Flush outstanding unpin tasks before pageflipping
    
    If we accumulate unpin tasks because we are pageflipping faster than the
    system can schedule its workers, we can effectively create a
    pin-leak. The solution taken here is to limit the number of unpin tasks
    we have per-crtc and to flush those outstanding tasks if we accumulate
    too many. This should prevent any jitter in the normal case, and also
    prevent the hang if we should run too fast.
    
    Note: It is important that we switch from the system workqueue to our
    own dev_priv->wq since all work items on that queue are guaranteed to
    only need the dev->struct_mutex and not any modeset resources. For
    otherwise if we have a work item ahead in the queue which needs the
    modeset lock (like the output detect work used by both polling or
    hpd), this work and so the unpin work will never execute since the
    pageflip code already holds that lock. Unfortunately there's no
    lockdep support for this scenario in the workqueue code.
    
    Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=46991
    Reported-and-tested-by: Tvrtko Ursulin 
    Signed-off-by: Chris Wilson 
    [danvet: Added note about workqueu deadlock.]
    Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=56337
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/intel_display.c |   22 ++++++++++++++++------
 drivers/gpu/drm/i915/intel_drv.h     |    4 +++-
 2 files changed, 19 insertions(+), 7 deletions(-)

commit a726915cef1daab57aad4c5b5e4773822f0a4bf8
Author: Daniel Vetter 
Date:   Tue Nov 20 14:50:08 2012 +0100

    drm/i915: resurrect panel lid handling
    
    But disabled by default. This essentially reverts
    
    commit bcd5023c961a44c7149936553b6929b2b233dd27
    Author: Dave Airlie 
    Date:   Mon Mar 14 14:17:55 2011 +1000
    
        drm/i915: disable opregion lid detection for now
    
    but leaves the autodetect mode disabled. There's also the explicit lid
    status option added in
    
    commit fca874092597ef946b8f07031d8c31c58b212144
    Author: Chris Wilson 
    Date:   Thu Feb 17 13:44:48 2011 +0000
    
        drm/i915: Add a module parameter to ignore lid status
    
    Which overloaded the meaning for the panel_ignore_lid parameter even
    more. To fix up this mess, give the non-negative numbers 0,1 the
    original meaning back and use negative numbers to force a given state.
    So now we have
    
    1  - disable autodetect, return unknown
    0  - enable autodetect
    -1 - force to disconnected/lid closed
    -2 - force to connected/lid open
    
    v2: My C programmer license has been revoked ...
    
    v3: Beautify the code a bit, as suggested by Chris Wilson.
    
    Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=27622
    Tested-by: Andreas Sturmlechner 
    Reviewed-by: Chris Wilson 
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/i915_drv.c    |    6 +++---
 drivers/gpu/drm/i915/intel_panel.c |   25 +++++++++++--------------
 2 files changed, 14 insertions(+), 17 deletions(-)

commit 8fed6193736bf22e0e44c03ee783761e9cc37238
Author: Takashi Iwai 
Date:   Mon Nov 19 18:06:51 2012 +0100

    drm/i915: Enable DP audio for Haswell
    
    This patch adds the missing code to send ELD for Haswell DisplayPort,
    based on Xingchao's original patch.
    
    A test was performed with HSW-D machine and NEC EA232Wmi DP monitor.
    
    Cc: Xingchao Wang 
    Signed-off-by: Takashi Iwai 
    Reviewed-by: Paulo Zanoni 
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/intel_ddi.c |    9 +++++++++
 1 file changed, 9 insertions(+)

commit c9839303d186d6270f570ff3c5f56c2327958086
Author: Chris Wilson 
Date:   Tue Nov 20 10:45:17 2012 +0000

    drm/i915: Pin the object whilst faulting it in
    
    In order to prevent reaping of the object whilst setting it up to
    handle the pagefault, we need to mark it as pinned. This has the nice
    side-effect of eliminating some special cases from the pagefault handler
    as well!
    
    Signed-off-by: Chris Wilson 
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/i915_gem.c |   29 +++++++++--------------------
 1 file changed, 9 insertions(+), 20 deletions(-)

commit fbdda6fb5ee5da401af42226878880069a6b8615
Author: Chris Wilson 
Date:   Tue Nov 20 10:45:16 2012 +0000

    drm/i915: Guard pages being reaped by OOM whilst binding-to-GTT
    
    In the circumstances that the shrinker is allowed to steal the mutex
    in order to reap pages, we need to be careful to prevent it operating on
    the current object and shooting ourselves in the foot.
    
    Signed-off-by: Chris Wilson 
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/i915_gem.c |    9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

commit be7cb6347e0c3aa1956748a860a2465a7ea128c4
Author: Chris Wilson 
Date:   Mon Nov 19 15:30:42 2012 +0000

    drm/i915: Remove bogus test for a present execbuffer
    
    The intention of checking obj->gtt_offset!=0 is to verify that the
    target object was listed in the execbuffer and had been bound into the
    GTT. This is guarranteed by the earlier rearrangement to split the
    execbuffer operation into reserve and relocation phases and then
    verified by the check that the target handle had been processed during
    the reservation phase.
    
    However, the actual checking of obj->gtt_offset==0 is bogus as we can
    indeed reference an object at offset 0. For instance, the framebuffer
    installed by the BIOS often resides at offset 0 - causing EINVAL as we
    legimately try to render using the stolen fb.
    
    Signed-off-by: Chris Wilson 
    Reviewed-by: Eric Anholt 
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/i915_gem_execbuffer.c |    9 ---------
 1 file changed, 9 deletions(-)

commit b92fa839015f27ba0f5c7ef9812eba9ecff538c2
Author: Chris Wilson 
Date:   Fri Nov 16 11:43:21 2012 +0000

    drm/i915: Remove save/restore of physical HWS_PGA register
    
    Now that we always restore the HWS registers (both physical and GTT
    virtual addresses) when re-initialising the rings, we can eliminate the
    superfluous save/restore of the register across suspend and resume.
    
    Signed-off-by: Chris Wilson 
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/i915_drv.h     |    1 -
 drivers/gpu/drm/i915/i915_suspend.c |    8 --------
 2 files changed, 9 deletions(-)

commit d09105c66eb813ab3f57ba5e738f477f6ff92dec
Author: Ben Widawsky 
Date:   Thu Nov 15 12:06:09 2012 -0800

    drm/i915: Fix warning in i915_gem_chipset_flush
    
    drivers/gpu/drm/i915/i915_drv.h:1545:2: warning: '______f' is static but
    declared in inline function 'i915_gem_chipset_flush' which is not static
    
    Reported-by: kbuild test robot 
    dri-devel-Reference: <50a4d41c data-blogger-escaped-.586vhmwghpukzbkb="" data-blogger-escaped-fengguang.wu="" data-blogger-escaped-intel.com="">
    Signed-off-by: Ben Widawsky 
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/i915_drv.h |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

commit 42d42e7e4220753bab3eb7b857721f203a4cd821
Author: Damien Lespiau 
Date:   Wed Oct 31 19:23:16 2012 +0000

    drm/i915: Only check for valid PP_{ON, OFF}_DELAYS on pre ILK hardware
    
    ILK+ have this register on the PCH. This check was triggering unclaimed
    writes.
    
    Signed-off-by: Damien Lespiau 
    Reviewed-by: Paulo Zanoni 
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/intel_bios.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

commit 607a6f7a6621f65706ff536b2615ee65b5c2f575
Author: Daniel Vetter 
Date:   Wed Nov 14 17:47:39 2012 +0100

    drm/i915: drop buggy write to FDI_RX_CHICKEN register
    
    Jani Nikula noticed that the parentheses are wrong and we & the bit
    with the register address instead of the read-back value. He sent a
    patch to correct that.
    
    On second look, we write the same register in the previous line, and
    the w/a seems to be to set FDI_RX_PHASE_SYNC_POINTER_OVR to enable the
    logic, then keep always set FDI_RX_PHASE_SYNC_POINTER_OVR and toggle
    FDI_RX_PHASE_SYNC_POINTER_EN before/after enabling the pc transcoder.
    
    So the right things seems to be to simply kill the 2nd write.
    
    Cc: Jani Nikula 
    Reviewed-by: Chris Wilson 
    [danvet: Dropped a bogus ~ from the commit message that somehow crept
    in.]
    Signed-off-by: Daniel Vetter 

 drivers/gpu/drm/i915/intel_display.c |    3 ---
 1 file changed, 3 deletions(-)

29 March 2013

374. Briefly: Evolution Mail on Debian Testing/Wheezy seems to have been improved

I switched to thunderbird about 6-12 months ago since I got fed up with certain issues, chief among which were:
* issues with syncing between google calendar and evolution calendar
* non-persistent calendar view -- back in the days you had the default calendar view, and that was that. You could change it temporarily, but it wouldn't persist between switching between mail and calendar.

Otherwise I've always found evolution 'prettier' than thunderbird, which gives me Eudora flashbacks, and works more seamlessly with PGP (thunderbird requires enigmail which works fine -- but it's still a plugin).

The only issue I've found so far was that I had to close and restart Evolution after adding my google calendar details before it would prompt me for the password and actually sync.

What I've noticed now:
* Calendar view (i.e. day/week/month) is persistent and remembers your preferences
* Sync between google and your local calendar works fine -- I had the appointment on my phone within a minute of entering it in Evolution
* The interface is cleaner, prettier (not sure whether anything has suffered from the deprecation syndrome most gnome-related software has been subject to recently)
* PGP required no set-up whatsoever -- an encrypted email was automatically decrypted without any set up at all. I'm not actually completely happy about that -- I'd prefer that to have been prompted for the password to unlock my key (I'd unlocked it for gajim a few hours earlier, and I guess it remained unlocked)

So while thunderbird has stood me well, I've historically been using Evolution since I switched to Linux (before that I used thunderbird on windows, and before that Pegasus Mail and Eudora) so I'm likely to switch back now that most of the things I were hung up on last time seem to have been fixed.

On the other hand, I'm noticing that I have to close and restart evolution any time I change account settings, which doesn't seem right, and which is fairly annoying.

28 March 2013

373. Cracking Windows passwords if you are a Linux user

Cracking windows passwords is no more difficult than any other set of passwords that need to be cracked via brute force/rainbow table lookup -- it's all about getting your hands on the right files to begin with.

Note that Vista and Windows 7 are the same in terms of how the passwords are stored. XP is different. You'll need to appropriate hash tables when cracking using Ophcrack-- you can't use XP hashes to crack Vista/7 and vice versa.

Why?
I actually have a fairly legit reason for this, other than pure curiosity: the professors and lecturers who have Win 7 at this Uni aren't given administrator access, so that they need to make an appointment with IT staff everytime they want to install or upgrade something. In my opinion, that's taking things way, way too far, especially in light of OSX user and Linux users don't face the same restrictions (I requested my computer to be delivered blank, so not even the IT staff can gain access without physically accessing my hdd).

I've also twice been in situations where a lab computer was set up by a PhD student who had since left, but without telling anyone the BIOS password in one case (remove CMOS battery, drain the power by shorting) and NT 4 password in another case (boot using linux, replace the screensaver file with cmd, boot, then wait and you've got admin access). Adding a few more techniques to your arsenal makes you a more well rounded human being. Plus you might gain a bit more respect from the students. Maybe.


Windows files that you need

You'll need to copy two files from your Windows harddrive (most likely o C:\). Since you can't access those files from within windows, the easiest way to do that is to make a bootable USB with linux on it, boot, then mount the windows partition and copy the files. It does require that you know how to mount devices from the command line in linux (hint: read up on mount) but it's really quite easy.

See here for how to make a linux usb install:
Arch: http://verahill.blogspot.com.au/2013/02/331-full-linux-install-on-usb-stick.html
Debian: http://verahill.blogspot.com.au/2013/03/361-installing-debian-on-usb-stick-from.html

On Windows Vista and Windows 7 you need to copy
/Windows/System32/config/SAM
/Windows/System32/config/SYSTEM.

On Windows XP you need to copy
/WINDOWS/system32/config/system
/WINDOWS/system32/config/SAM.


Generating input
First set up your linux system and install bkhive and samdump2. On Arch you can get these from the AUR (here and here), and on Debian you can install it from the repos by doing
sudo apt-get install bkhive samdump2

Once you have bkhive and samdump, do
bkhive system sys.out
samdump2 SAM sys.out > crack.list



Cracking
I prefer John the Ripper, but ophcrack is in the repos.

Ophcrack
Ophcrack is one alternative for which you can download rainbow tables, while John the Ripper can do it via password lists and brute force. You can get Ophcrack from the Debian repos as well as AUR in ARch.

To crack with Ophcrack you'll need tables -- you can get them from here: http://ophcrack.sourceforge.net/tables.php

Start with Xp Free Small or Vista Free, depending on whether you're going after XP or Vista/Windows 7 passwords. If your passwords still won't break, you can always get a bigger table. Put your tables in e.g. /tmp/crack/xp and /tmp/crack/vista

Do (for xp -- point towards /tmp/crack/vista for vista/windows 7)
ophcrack -n 6 -g -t /tmp/crack/xp -f crack.list

where -n 6 means to use 6 threads (use as many as you have cores). You can also launch ophcrack with a graphical user interface by simply typing ophcrack.

John the Ripper
On Arch you can get John the Ripper via the standard repos, and on Debian you can easily compile it.

You should get a good password list to guide your cracking -- rockyou is probably one of the best:
wget http://runamux.net/search/download/file/95wr2RdM/rockyoutxt.bz2
bunzip2 rockyoutxt.bz2

To crack XP passwords, run
john --wordlist=rockyoutxt --format=lm crack.list

To crack Vista or Windows 7 passwords, run
 john --wordlist=rockyoutxt --format=nt crack.list

If you have compiled john the ripper with mpi support, just launch it along the lines of
mpirun -n 6 john --wordlist=rockyoutxt --format=nt crack.list

There's no guarantee that you'll successfully crack all passwords -- it simply depends on 1) how common the passwords are or 2) whether they are short enough to brute force. I've used the methods above to crack passwords on a couple of machines (even virtual ones).

If you're really lazy and want a fully automated cracking experience, just use the ophcrack live cd. The downside is that you'll be occupying the machine which holds the windows installation until you're done cracking. With the method here you can crack at your leisure on your own windows machine.

372. Leadtek DTV1000S on Arch linux

While this DVB-T card seems to work fine out of the box on Arch linux (using kaffeine, gnome-dvb-setup, me-tv, w_scan etc.) it is quite unstable --  you often get 'failed to lock channel' errors, and the picture gets corrupted after watching the same channel too long (1 hour). . In contrast, I've been using it for over a year on Debian without issues, so I decided to have a quick look at what I did back in the days to get it working, and realised I'd been a bit lazy setting things up (i.e. I had treated it like PnP).


To improve the stabilit of Leadtek DTV 1000S on Arch linux, do the following:

1. Edit /etc/modprobe.d/modprobe.conf
options saa7134 tuner=48 card=175
2. Install the proper firmware driver
If you've noticed errors in dmesg/during boot-up relating to missing tda10048 firmware, install the dvb-fe-tda10048-1.0.fw firmware from AUR

wget https://aur.archlinux.org/packages/dv/dvb-fe-tda10048-1.0.fw/dvb-fe-tda10048-1.0.fw.tar.gz
tar xvf dvb-fe-tda10048-1.0.fw.tar.gz
cd dvb-fe-tda10048-1.0.fw
makepkg -s
sudo pacman -U dvb-fe-tda10048-1.0.fw-1-2-any.pkg.tar.xz