27 February 2012

81. nvidia 295.20 bug causing gnome-shell to crash on Debian Testing


UPDATE: Here's how to downgrade your drivers:
http://verahill.blogspot.com.au/2012/03/debian-testing-downgrading-nvidia.html

Update: 
A bigger issue is what this bug does to evolution:
http://verahill.blogspot.com.au/2012/02/debian-testing-wheezy-64-no-real.html


The symptoms:
I've tried to use the correct gnome-shell terminology.

1. Go to the top left corner (Hot Corner) of the desktop to get the Overview and Search Entry field
2. Start typing in the name of an application
3. The window will flicker as if gnome-shell is being restarted (similar to alt+f2++r)
4. Do it again and you get a full-on crash with an unhappy looking computer

dmesg based on repeatedly crashing gnome-shell says:

[ 7011.967820] gnome-shell[32742]: segfault at 10 ip 00007fa1b6d98c0f sp 00007fa1914a1638 error 6 in libnvidia-tls.so.295.20[7fa1b6d98000+3000]
[ 7111.276979] gnome-shell[748]: segfault at 10 ip 00007ff7eb598c0f sp 00007ff7beffc638 error 6 in libnvidia-tls.so.295.20[7ff7eb598000+3000]
[ 7620.952276] gnome-shell[2933]: segfault at 10 ip 00007f0a9fdd9c0f sp 00007f0a710fe638 error 6 in libnvidia-tls.so.295.20[7f0a9fdd9000+3000]
[ 7628.106656] gnome-shell[2986]: segfault at 10 ip 00007f26423f3c0f sp 00007f2612ffd638 error 6 in libnvidia-tls.so.295.20[7f26423f3000+3000]
[ 7658.755466] gnome-shell[3818]: segfault at 10 ip 00007f76bbf2cc0f sp 00007f7691a77638 error 6 in libnvidia-tls.so.295.20[7f76bbf2c000+3000]
[ 7666.310714] gnome-shell[3905]: segfault at 10 ip 00007f3279e64c0f sp 00007f325469d638 error 6 in libnvidia-tls.so.295.20[7f3279e64000+3000]
[ 7717.061483] gnome-shell[4829]: segfault at 10 ip 00007f245ad26c0f sp 00007f243469c638 error 6 in libnvidia-tls.so.295.20[7f245ad26000+3000]


The libnvidia-tls files are found here:
/usr/lib/x86_64-linux-gnu/libnvidia-tls.so.295.20
/usr/lib/x86_64-linux-gnu/tls/libnvidia-tls.so.295.20

and
 dpkg --search libnvidia-tls.so.295.20 
gives
libgl1-nvidia-glx: /usr/lib/x86_64-linux-gnu/tls/libnvidia-tls.so.295.20
libgl1-nvidia-glx: /usr/lib/x86_64-linux-gnu/libnvidia-tls.so.295.20
Ergo, that's where the bug is.


Cause:
Bad nvidia drivers -- in package libgl1-nvidia-glx

This is not unique to debian.
"Confirmed, I'm seeing the same on Gentoo ~amd64. gnome-shell 3.2.2.1 crashes while doing a search with nvidia-drivers 295.20 installed (backtrace is in libnvidia-tls.so). Downgrading to nvidia-drivers 290.10 resolves the issue, so it is a problem with the drivers."

http://www.nvnews.net/vbulletin/showthread.php?t=174049 (14 Feb 2012)

UPDATE: Here's how to downgrade your drivers:
http://verahill.blogspot.com.au/2012/03/debian-testing-downgrading-nvidia.html


There are no bugs listed for libgl1-nvidia-glx
http://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=libgl1-nvidia-glx

But nvidia-glx has it's fair share of bugs filed against it:
http://bugs.debian.org/cgi-bin/pkgreport.cgi?package=nvidia-glx

From what I can tell this is the relevant bug report (17 February 2012):
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=660189
which points to
http://www.nvnews.net/vbulletin/showthread.php?t=174049&page=3

Solution:

1. The 'proper way':
Downgrade your drivers.

UPDATE: Here's how to downgrade your drivers:
http://verahill.blogspot.com.au/2012/03/debian-testing-downgrading-nvidia.html


2. The easy, interesting way:
"For me deleting recently-used.xbel and recreating it with no content solved the problem just temporary. But instead of creating a equally named directory one could also sudo chattr +i recently-used.xbel to keep the file empty.  Keeping the file empty also significantly speeds up the application launcher for me. so it would be nice to have a way to configure this instead of fixing it that rude way, for folks that dont want or need recently used files."


In practical terms, this means:

echo ""> ~/.local/share/recently-used.xbel
sudo chattr +i ~/.local/share/recently-used.xbel

And you are done!

Once the bug has been fixed, you can do
 chattr -i ~/.local/share/recently-used.xbel 


to restore normal functionality

This solution worked for me on an up-to-date debian testing.

Oh well. At least the folks at nvidia are aware of the bug:


Thoughts:
The nvidia binaries only entered the debian testing repos around the 25-26 of February from what I can tell. The bug was known for ten days by then, so why did the binaries get promoted to testing?

Here's what I've got installed:

i A glx-alternative-nvidia          - allows the selection of NVIDIA as GLX prov
i A libgl1-nvidia-alternatives      - transition libGL.so* diversions to glx-alt
i A libgl1-nvidia-glx               - NVIDIA binary OpenGL libraries          
i A libglx-nvidia-alternatives      - transition libgl.so diversions to glx-alte
pi  libnvidia-compiler-ia32         - NVIDIA runtime compiler library (32-bit)
i A libnvidia-ml1                   - NVIDIA management library (NVML) runtime l
i A nvidia-alternative              - allows the selection of NVIDIA as GLX prov
i A nvidia-compute-profiler         - NVIDIA Compute Visual Profiler          
i   nvidia-cuda-dev                 - NVIDIA CUDA development files          
i A nvidia-cuda-doc                 - NVIDIA CUDA and OpenCL documentation    
i A nvidia-cuda-gdb                 - NVIDIA CUDA GDB                        
i A nvidia-cuda-toolkit             - NVIDIA CUDA toolkit                    
i   nvidia-glx                      - NVIDIA metapackage                      
i A nvidia-installer-cleanup        - Cleanup after driver installation with the
i   nvidia-kernel-3.1.0-1-amd64     - NVIDIA binary kernel module for Linux 3.1.
i   nvidia-kernel-common            - NVIDIA binary kernel module support files
i A nvidia-kernel-dkms              - NVIDIA binary kernel module DKMS source
i   nvidia-kernel-source            - NVIDIA binary kernel module source      
i A nvidia-libopencl1               - NVIDIA OpenCL library                  
i   nvidia-libopencl1-ia32          - NVIDIA OpenCL 32-bit library            
pi  nvidia-opencl-common            - NVIDIA OpenCL driver                    
i   nvidia-opencl-dev               - NVIDIA OpenCL development files        
pi  nvidia-opencl-icd-ia32          - NVIDIA OpenCL ICD (32-bit)              
i   nvidia-settings                 - Tool for configuring the NVIDIA graphics d
i A nvidia-smi                      - NVIDIA System Management Interface      
i A nvidia-support                  - NVIDIA binary graphics driver support file
i A nvidia-vdpau-driver             - NVIDIA vdpau driver                    
pi  nvidia-xconfig                  - X configuration tool for non-free NVIDIA d
i A xserver-xorg-video-nvidia       - NVIDIA binary Xorg driver






26 February 2012

80. Compiling octave 3.6.1 on Debian Testing

I originally was hesitant towards building my own packages, not so much because it is difficult (well-maintained packages are easy to build using configure/make), but because you can end up with a system where dependencies break and you constantly have to manually track new releases of packages you want.

Having had to struggle with outdated or bug-ridden debian packages during the past year has made me change my mind. It obviously depends on what you use you computer for -- the debian packages are supposed to be of high quality and they do unarguably go through a lot of stability testing before being released down the chain from SID to Stable. But I also get the impression that some debian maintainers aren't...well...maintaining their packages sufficiently.

Take the alt+f2 gnome bug for example -- in an ideal world a fix should've been rolled out within 24 hours, rather than just waiting for future version increments to take care of it. The gnuplot bug is another example. There's no solution in sight, even though it has been solved in upstream. Maybe there are reasons to which I'm oblivious for this, but the end result is that I'm a lot more cautious with trusting the debian vs upstreams maintainers now.

So, as part of the  series of science packages that you should compile yourself, here's Octave 3.6.1
At the moment of writing the versions in the deb repos are 3.4.2-8, 3.4.2-12 and 3.4.2-12+b1.

As an aside, if you know what you'll be doing with each software package, create a couple of test scripts so you can track regressions. That way you can be just a little bit more confident in the results you obtain -- if you publish incorrect data you will have to go through the humiliating experience of a retraction or correction.

In addition to everything listed below you need to have a working build environment  with a good fortran compiler.
sudo apt-get install gfortran build-essential 
at a minimum thus.

-- START HERE ---
sudo apt-get install libqhull-dev libpcre++-dev libblas-dev liblapack-dev libreadline-dev
sudo apt-get install libcurl4-openssl-dev libfltk1.3-dev libgraphicsmagick++-dev libhdf5-serial-dev libqrupdate-dev libsuitesparse-metis-dev glpk gperf flex bison libfontconfig1-dev
 wget ftp://ftp.gnu.org/gnu/octave/octave-3.6.1.tar.gz
tar -xvf octave-3.6.1.tar.gz
cd octave-3.6.1/
./configure
make -j4

where 4 is the number of cores +1 (for me 3 cores)

To validate your  compiled version, run
make check

Summary:
  PASS  10008
  FAIL      0
There were 2 expected failures (see fntests.log for details).
Expected failures are known bugs.  Please help improve Octave
by contributing fixes for them.



To install
either
sudo checkinstall
or
sudo make install
Done.


Error:
checking for qhull.h... no
configure: WARNING: Qhull library not found -- this will result in loss of functionality of some geometry functions.
Solution:
sudo apt-get libqhull-dev


Error:
checking pcre.h usability... no
checking pcre.h presence... no
checking for pcre.h... no
checking pcre/pcre.h usability... no
checking pcre/pcre.h presence... no
checking for pcre/pcre.h... no
checking whether pcre.h defines the macros we need... no
configure: error: to build Octave, you must have the PCRE library and header files installed

Solution:
sudo apt-get install libpcre++-dev


Error:
checking if sgemm_ is being linked in already... no
checking for ATL_xerbla in -latlas... no
checking for sgemm_ in -lblas... no
checking for sgemm_ in -lmkl... no
checking for sgemm_ in -framework vecLib... no
checking for sgemm_ in -lcxml... no
checking for sgemm_ in -ldxml... no
checking for sgemm_ in -lscs... no
checking for sgemm_ in -lcomplib.sgimath... no
checking for sgemm_ in -lblas... (cached) no
checking for sgemm_ in -lblas... (cached) no
configure: error: You are required to have BLAS and LAPACK libraries
Solution:
sudo apt-get install libblas-dev

(libopenblas-dev is still only in the unstable repo)


Error:

checking for cheev_... no
checking for cheev_ in -llapack... no
checking for cheev_ in -llapack_rs6k... no
configure: error: You are required to have BLAS and LAPACK libraries
Solution:
sudo apt-get install liblapack-dev


Error:
configure: WARNING: I need GNU Readline 4.2 or later
configure: error: this is fatal unless you specify --disable-readline
Solution:
sudo apt-get install libreadline-dev


Error:

configure: WARNING: I didn't find gperf, but it's only a problem if you need to reconstruct oct-gperf.h
configure: WARNING: UMFPACK not found.  This will result in some lack of functionality for sparse matrices.
configure: WARNING: qrupdate not found.  The QR & Cholesky updating functions will be slow.
configure: WARNING: AMD library not found.  This will result in some lack of functionality for sparse matrices.
configure: WARNING: COLAMD library not found.  This will result in some lack of functionality for sparse matrices.
configure: WARNING: CCOLAMD library not found.  This will result in some lack of functionality for sparse matrices.
configure: WARNING: CHOLMOD library not found.  This will result in some lack of functionality for sparse matrices.
configure: WARNING: CXSparse library not found.  This will result in some lack of functionality for sparse matrices.
configure: WARNING: cURL library not found.  The ftp objects, urlread and urlwrite functions will be disabled.
configure: WARNING: GLPK library not found.  The glpk function for solving linear programs will be disabled.
configure: WARNING: GraphicsMagick++ library not found.  The imread function for reading image files will not be fully functional.
configure: WARNING: HDF5 library not found.  Octave will not be able to save or load HDF5 data files.
configure: WARNING: FLTK config script not found.  Native graphics will be disabled.
configure: WARNING: 
configure: WARNING: I didn't find the necessary libraries to compile native
configure: WARNING: graphics.  It isn't necessary to have native graphics,
configure: WARNING: but you will need to have gnuplot installed or you won't
configure: WARNING: be able to use any of Octave's plotting commands
configure: WARNING: 
configure: 
configure: NOTE: libraries may be skipped if a library is not found OR
configure: NOTE: if the library on your system is missing required features.

Solution:
First, make a few decisions:
Package libhdf5-dev is a virtual package provided by:
  libhdf5-serial-dev 1.8.4-patch1-3
  libhdf5-openmpi-dev 1.8.4-patch1-3
  libhdf5-mpich-dev 1.8.4-patch1-3
  libhdf5-lam-dev 1.8.4-patch1-3
You should explicitly select one to install.
Package libcurl-dev is a virtual package provided by:
  libcurl4-openssl-dev 7.24.0-1
  libcurl4-nss-dev 7.24.0-1
  libcurl4-gnutls-dev 7.24.0-1
You should explicitly select one to install.
Package libfltk-dev is a virtual package provided by:
  libfltk1.3-dev 1.3.0-5
  libfltk1.1-dev 1.1.10-10
You should explicitly select one to install.
Package libumfpack4-dev is a virtual package provided by:
  libsuitesparse-metis-dev 3.1.0-2
  libsuitesparse-dev 1:3.4.0-2
You should explicitly select one to install.
I chose thusly:

sudo apt-get install libcurl4-openssl-dev libfltk1.3-dev libgraphicsmagick++-dev libhdf5-serial-dev libqrupdate-dev libsuitesparse-metis-dev

The following errors/mesages remain:

  Do internal array bounds checking:  false
  Use octave_allocator:               false
  Build static libraries:             false
  Build shared libraries:             true
  Dynamic Linking:                    true (dlopen)
  Include support for GNU readline:   true
  64-bit array dims and indexing:     false

configure: WARNING: I didn't find gperf, but it's only a problem if you need to reconstruct oct-gperf.h
configure: WARNING: GLPK library not found.  The glpk function for solving linear programs will be disabled.
configure: 
configure: NOTE: libraries may be skipped if a library is not found OR
configure: NOTE: if the library on your system is missing required features.


The 64 bit issue can probably  be solved using ./configure --enable-64 BUT requires BLAS that is compatible with 64 bit integers.


Links to this page:
http://tata-box-blog.blogspot.nl/2012/04/install-octave-361-in-debian-squeeze.html
http://askubuntu.com/questions/124731/when-will-octave-3-6-1-be-available-to-be-installed

79. Bankid/nexus personal and iceweasel on Debian Testing

Update 19 Feb. 2013:
Here's an updated post: http://verahill.blogspot.com.au/2013/02/341-upgradinginstalling-bankid-on-64.html -- see that one instead, in particularly if you're upgrading.

Note that you may have to compile your own nspluginwrapper:
http://verahill.blogspot.com.au/2013/03/366-nspluginwrapper-on-debian.html

and you will need to enable multiarch to install ia32-libs:
sudo dpkg --add-architecture i386
sudo apt-get update

Original post:
Swedish banks and government institutions use bankid/nexus personal for electronic id verification. Sadly, it's a horrible solution -- it seems to be closed source, the bankid website is a POS which prevents me from downloading the 64 bit version claiming that it's not supported (I've used it for a few years now, so it's clearly bunkum).

Note that there's a FOSS alternative in Fribid (http://verahill.blogspot.se/2012/02/debian-testing-wheezy-64-fribid-as.html) which seems to be working perfectly -- and if you can use it, use it. The main limitation is that in practice you'll have to collect your certificate/ID with it, since newer versions of BankID saves the ID in an incompatible format. Like many foreigners, I don't have the opportunity to visit Sweden for the sole sake of picking up a new ID, so I'm stuck with BankID. But you may not be.

* A Swedish how-to is available here: http://ubuntu-se.org/wiki/NexusPersonal#Installation_p.C3.A5_64-bitarssystem

* Another, more recent how-to is here: http://popqvarnstrom.blogspot.com.au/2011/06/bankid-nexus-personal-on-ubuntu-1104-64.html

Note: I have never 'exported' my ID, but have always copied the ~/.personal directory between computers. The problem with exporting is that you are only allowed to do it once. The problem with Nexus allowing your to copy the file itself is that anyone with physical access to your computer can copy the key.

--START HERE --
 Here's a summary of how to get it working on debian testing wheezy:

In theory you should install nexus personal from here:
https://install.bankid.com/

I've got v 4.17.0.11 installed on a 64 bit system. The message on this page is a load of bollocks:


Whatever -- the good guys over at Arch supply a link:
wget http://install.bankid.com/Repository/BISP-4.19.0.11351.tar.gz

EDIT: you can use this generic url instead  https://install.bankid.com/Download?defaultFileId=Linux

tar -xvf  BISP-4.19.0.11351.tar.gz
cd  BISP-4.19.0.11351
 sudo sh install.4.19.0.11351.sh i
 Installing BankID Security Application
ln: failed to create symbolic link `/usr/lib/firefox-addons/plugins': No such file or directory
WARNING: Failed installing plugin for Firefox 3. Manually add symlink to libplugins.so in your Firefox 3 plugin directory if this browser is to be used.
Installation complete.

Since the plugin is 32 bit, you need to link it with nspluginwrapper, and you need 32 bit libs, so

sudo apt-get install nspluginwrapper ia32-libs




Also, as far as I can tell, you need iceweasel/firefox. Chrome/ium won't work.

sudo nspluginwrapper -i /usr/local/lib/personal/libplugins.so  

Check to see if it's installed:
nspluginwrapper -l
 /usr/lib/mozilla/plugins/npwrapper.libplugins.so 
 Original plugin: /usr/local/lib/personal/libplugins.so 
Plugin viewer: /usr/lib/nspluginwrapper/i386/linux/npviewer
Wrapper version string: 1.3.0  
And visit  the following page to make sure
 https://test.bankid.com/
Or your bank.


Your  key -- on a computer where you've used bankid before -- will be in ~/.personal -- don't bother trying to import or export it using the bankid/nexus personal programme (http://popqvarnstrom.blogspot.com.au/2011/06/bankid-nexus-personal-on-ubuntu-1104-64.html) since you're apparently only allowed to do that a certain number of times.

If you just plain copy the files, however, you can do it as many times as you want. I told you the programme is a POS. Anyway,

tree .personal
.personal
|-- backup
|   |-- config
|   |   `-- Personal.cfg
|   `-- store
|-- config
|   `-- Personal.cfg
`-- store
    |-- 1.ngp
    `-- 2.ngp


Nexus Personal/BankID is installed in /usr/local/lib/personal/

Links to this page:
http://popqvarnstrom.blogspot.se/2011/06/bankid-nexus-personal-on-ubuntu-1104-64.html