Showing posts with label compile. Show all posts
Showing posts with label compile. Show all posts

21 March 2017

634. Compiling EQ3/6 on Debian

EQ3/6, a program for calculation of geochemical equilibrium in aqueous systems and which can be downloaded here, has a spectacularly unhelpful Install.txt file.

Sadly, we do not provide specific instructions on how to compile and link
the software to build the executables. You have to figure that out yourself
for your specific platform. It will require some number of steps. Also, we 
do not provide makefiles. It is not necessary to use make files, although
some people like them. If all this seems to exceed your knowledge and capacity,
seek the assistance of a qualified Linux or Unix maven who knows how to do
these things on your platform. 


Here I describe how I went about to compile it.


1. Download EQ36_80a_linux.zip from https://www-gs.llnl.gov/about/energy-security/geochemistry


2. Extract it. Move the EQ3_6v8.0a folder to ~
You can put it wherever you want, but then need to edit eq36cfg, and you'll have to deal with csh not handling long paths that well. It's just easier to give up and put EQ3_6v8.0a under $HOME


3. Time to extract further, compile and set up
cd ~/EQ3_6v8.0a
ls *.tar |xargs -I {} tar xvf {}
find -name "*.gz"|xargs -I {} gunzip {}

Time to look at what we have. There are .f files in
xcon6/src
eq3nr/src
eqlib/src
eqlibu/src
eq6/src
xcon3/src
eqpt/src
eqlibg/src

A lot of these are subroutines. We'll compile all files as modules and make static libraries
cd eqlibg/src
gfortran -c *.f
ar cr eqlibg.a *.o
cd ../../eqlibu/src
gfortran -c *.f
ar cr eqlibu.a *.o
cd ../../eqlib/src
gfortran -c *.f
ar cr eqlib.a *.o

cd ../../eqpt/src
sh makelinks
gfortran *.f *.a -o eqpt
cp eqpt ../../bin/
cd ../../xcon3/src
sh makelinks
gfortran *.f *.a -o xcon3
cp xcon3 ../../bin/
cd ../../xcon6/src
sh makelinks
gfortran *.f *.a -o xcon6
cp xcon6 ../../bin/
cd ../../eq3nr/src
sh makelinks
gfortran *.f *.a -o eq3nr
cp eq3nr ../../bin/
cd ../../eq6/src
sh makelinks
gfortran *.f *.a -o eq6

You'll get
eq6.f:16.9: use mod6pt 1 Fatal Error: Can't open module file 'mod6pt.mod' for reading at (1): No such file or directory eqcalc.f:77.9: use mod6pt 1 Fatal Error: Can't open module file 'mod6pt.mod' for reading at (1): No such file or directory eqphas.f:129.9: use mod6pt 1 Fatal Error: Can't open module file 'mod6pt.mod' for reading at (1): No such file or directory eqshel.f:146.9: use mod6pt 1 Fatal Error: Can't open module file 'mod6pt.mod' for reading at (1): No such file or directory exivar.f:121.9: use mod6pt 1 Fatal Error: Can't open module file 'mod6pt.mod' for reading at (1): No such file or directory
Don't despair, just run again:

gfortran *.f *.a -o eq6
cp eq6 ../../bin/

You should now have a bin/ that looks like
eq3nr eq6 eqpt readme.txt runeq3 runeq6 runeqpt xcif3 xcif6 xcon3 xcon6
cd ~/EQ3_6v8.0a
cat eq36cfg >> ~/.cshrc
cd scripts/
ln -s runeq36 runeq3
ln -s runeq36 runeq6
sudo mkdir /usr/tmp
sudo chmod ugo+rw /usr/tmp
cd ../db/
csh ../scripts/runeqpt all
------------------------------------ All done

4. Test
cd ~/EQ3_6v8.0a/3tlib_fmt
csh ../scripts/runeq36 fmt ../db/data0 c4pgwbN2.3i
[..] Done. Optimization ended outside requested limits. Starting hybrid Newton-Raphson iteration. Done. Hybrid Newton-Raphson iteration converged in 56 iterations. * Warning - (EQ3NR/eq3nr) The calculated TDS of 3.44395E+05 mg/L differs from the input file value of 3.54016E+05 mg/L by more than 1%. The calculated value will be used in subsequent calculations. The pickup file has been written. No further input found. Start time = 13:57:15 21Mar2017 End time = 13:57:15 21Mar2017 Run time = 0.586E-01 seconds Normal exit Note: The following floating-point exceptions are signalling: IEEE_UNDERFLOW_FLAG ------------------------------------ The following output files were written: c4pgwbN2.3o c4pgwbN2.3p ------------------------------------ The following input files were run without generating any EQ3NR error messages: c4pgwbN2.3i ------------------------------------ All done
The question here is whether the error is indicative of a serious issue with the compiled software.

I reran as
csh ../scripts/runeq3 fmt c4pgwbN2.3i |tee c4pgwbN2.3o

and compared
diff c4pgwbN2.3o Outputs/c4pgwbN2.3o
18c18 < Run 14:12:44 21Mar2017 --- > Run 13:18:06 07Sep2011 870,871c870,871 < -0.0000 per cent of the total charge < -0.0000 per cent of the mean charge --- > 0.0000 per cent of the total charge > 0.0000 per cent of the mean charge 1235,1236c1235,1236 < Start time = 14:12:44 21Mar2017 < End time = 14:12:44 21Mar2017 --- > Start time = 13:18:06 07Sep2011 > End time = 13:18:06 07Sep2011 1238c1238 < Run time = 0.586E-01 seconds --- > Run time = 0.781E-01 seconds

The results are the same.We're good.

Everything put together as a script:
mkdir ~/tmp cd ~/tmp wget http://www-gs.llnl.gov/content/assets/docs/energy/EQ36_80a_Linux.zip unzip EQ36_80a_Linux.zip cd Linux mv EQ3_6v8.0a ~/ rm -rf Linux cd ~/EQ3_6v8.0a ls *.tar |xargs -I {} tar xvf {} find -name "*.gz"|xargs -I {} gunzip {} cd eqlibg/src gfortran -c *.f ar cr eqlibg.a *.o cd ../../eqlibu/src gfortran -c *.f ar cr eqlibu.a *.o cd ../../eqlib/src gfortran -c *.f ar cr eqlib.a *.o cd ../../eqpt/src sh makelinks gfortran *.f *.a -o eqpt cp eqpt ../../bin/ cd ../../xcon3/src sh makelinks gfortran *.f *.a -o xcon3 cp xcon3 ../../bin/ cd ../../xcon6/src sh makelinks gfortran *.f *.a -o xcon6 cp xcon6 ../../bin/ cd ../../eq3nr/src sh makelinks gfortran *.f *.a -o eq3nr cp eq3nr ../../bin/ cd ../../eq6/src sh makelinks gfortran *.f *.a -o eq6 gfortran *.f *.a -o eq6 cp eq6 ../../bin/ cd ~/EQ3_6v8.0a cat eq36cfg >> ~/.cshrc cd scripts/ ln -s runeq36 runeq3 ln -s runeq36 runeq6 sudo mkdir /usr/tmp sudo chmod ugo+rw /usr/tmp cd ../db/ csh ../scripts/runeqpt all

05 June 2015

611. Building ecce on debian jessie

NOTE:
* I've confirmed that ECCE built like this installs and works perfectly on a Thinkpad SL410 with intel graphics

* It also compiles and runs perfectly on a home built desktop with external nvidia card (GF119) using the binary non-free nvidia drivers

* It also compiles and runs perfectly on a home built desktop with onboard nvidia (GeForce 7025/nForce 630a) using the nouveau drivers. I had issues on this desktop before, but reinstalled debian jessie from scratch. Before that I used nvidia-legacy drivers, which may or may not  (probably not) have had something to do with it not working.

* UPDATE: instead of putting
#include <freetype.h>
the recommended method is to use
#include FT_FREETYPE_H
I've updated the patch_script.sh below accordingly. Same goes for ftoutln.h vs FT_OUTLINE_H, but the latter didn't work (error saying #include must use "" or <>)

* UPDATE: If you're having issues with undetected -lGL and -lGLU in the wxwidgets step, it's because
667       endif
668       cat configure.orig | sed -e 's%^SEARCH_INCLUDE="\\%SEARCH_INCLUDE="$ECCE_HOME/${ECCE_SYSDIR}3rdparty/mesa/include \\%' >! configure
669       chmod a+x configure
670 

needs to be changed to
667       endif
668       cat configure.orig | sed -e 's%^SEARCH_INCLUDE="\\%SEARCH_INCLUDE="$ECCE_HOME/${ECCE_SYSDIR}3rdparty/mesa/include $ECCE_HOME/${ECCE_SYSDIR}3rdparty/mesa/lib \\%' >! configure
669       chmod a+x configure
670 
I had this issue on a debian wheezy system with the vendor nvidia libraries.
I wouldn't have spotted this bug otherwise.

* Another error from my debian wheezy nvidia system: if you get
checking how to run the C preprocessor... x86_64-linux-gnu-gcc -E ./configure: line 2880: syntax error near unexpected token `Using' ./configure: line 2880: `  AC_MSG_NOTICE(Using external PCRE library from $PCRE_CONFIG)'
then make sure you're not using autoconf2.13, which is an obsolete version. I think I have it due to my system originally being installed back in 2010.

* Finally, I'm currently  working on fixing minor things that have been nagging me in ecce. One is the basis set quicklist (in src/dsm/edsiimpl/ICalcUtils.C), but obviously that's a personal preference. A more serious one is the 256 character limit for csh commands:
Exceeds maximum C shell command length of 256 characters
Note that this isn't the C shell complaining -- this is a built-in limit in ecce (in src/comm/rcommand/RCommand.C). I've changed that limit to 16384 characters (the real limit is much, much higher)
I've also added two basis sets to ECCE, and have tinkered with the exchange-correlation functionals.

I'll try to push the fixes back upstreams when I'm ready if they'll accept them; otherwise I'll create my own github/sourceforge repo.

THE POST:
Building ECCE on debian wheezy was a breeze. Building ECCE on debian jessie was painful.

In the end it boiled down to two things:
*freetype headers are no longer in freetype2/freetype/ but in freetype2/

*-Wformat-security is turned on by default

mkdir ~/tmp/ecce_compile -p
cd ~/tmp/ecce_compile
sudo apt-get install bzip2 build-essential autoconf libtool ant pkg-config
sudo apt-get install gtk+-2.0-dev libxt-dev csh gfortran openjdk-7-jdk python-dev
sudo apt-get install libjpeg-dev imagemagick xterm libfreetype6-dev libfl-dev libtool-bin

As usual I'm not 100% sure when it comes to the necessary packages. libfl-dev might not be needed.

Download the ECCE source code and put the ecce-v7.0-src.tar.bz2 file in ~/tmp/ecce_compile. Put the patch_script.sh file (see below in this post for the code) in ~/tmp/ecce_compile. Then do
 
tar xvf ecce-v7.0-src.tar.bz2 
cd ecce-v7.0/
export ECCE_HOME=`pwd`
cd build/
./build_ecce

You'll now step through a list over programs and libraries that are needed and what ECCE can find. If you're having issues with e.g. javac and java being different versions, use sudo update-alternative --config javac.

At the end you'll be asked whether to skip these checks next time -- answer y(es).

Next do
./build_ecce|tee xerxes.log && ./build_ecce |tee mesa.log && ./build_ecce |tee wxwidgets.log
sh ../../patch_script.sh && ./build_ecce|tee wxpython.log 
./build_ecce|tee httpd.log && ./build_ecce|tee ecce.log

If all went well you've managed to build the ecce binaries. If not, go through wxpython.log and check for errors relating to format-security statements. Then go through ecce.log and look for issues with freetype.

What to do with the binaries? Follow one of the many ECCE installation posts on this blog, e.g. http://verahill.blogspot.com.au/2013/08/487-version-70-of-ecce-out-now.html

NOTE that if you get ''Invalid null command." trying to execute install_ecce.v7.0.csh, install tcsh and do
tcsh install_ecce.v7.0.csh

The patch_script.sh file -- when copying, make sure to check that the lines end where they are supposed to and not broken up.
#!/bin/bash cp ${ECCE_HOME}/build/3rdparty-dists/wxPython-src-2.8.12.1.tar.bz2 ${ECCE_HOME}/3rdparty/build/ cd ${ECCE_HOME}/3rdparty/build/ tar xvf wxPython-src-2.8.12.1.tar.bz2 rm wxPython-src-2.8.12.1.tar.bz2 cd ${ECCE_HOME}/3rdparty/build/wxPython-src-2.8.12.1/wxPython grep -rsl "PyErr_Format(PyExc_RuntimeError, mesg)" *|xargs -I {} sed -i 's/PyErr_Format(PyExc_RuntimeError, mesg)/PyErr_Format(PyExc_RuntimeError, "%s", mesg)/g' {} cd ${ECCE_HOME}/3rdparty/build/wxPython-src-2.8.12.1/wxPython/src/gtk/ sed -i 's/wxLogFatalError(m)/wxLogFatalError("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogError(m)/wxLogError("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogWarning(m)/wxLogWarning("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogMessage(m)/wxLogMessage("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogInfo(m)/wxLogInfo("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogDebug(m)/wxLogDebug("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogVerbose(m)/wxLogVerbose("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogStatus(pFrame, m)/wxLogStatus(pFrame, "%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogStatus(m)/wxLogStatus("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogSysError(m)/wxLogSysError("%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogGeneric(level, m)/wxLogGeneric(level, "%s", m.c_str())/g' _misc_wrap.cpp sed -i 's/wxLogTrace(mask, m)/wxLogTrace(mask, "%s", m.c_str())/g' _misc_wrap.cpp cd ${ECCE_HOME}/src grep -srl "<freetype/freetype.h>" |xargs -I {} sed -i 's,<freetype/freetype.h>,FT_FREETYPE_H,g' {} grep -srl "freetype/" |xargs -I {} sed -i 's,freetype/,,g' {} cd ${ECCE_HOME}/build


What I found during troubleshooting:

In files such as:
./3rdparty/build/wxPython-src-2.8.12.1/wxPython/src/gtk/_core_wrap.cpp
./3rdparty/build/wxPython-src-2.8.12.1/wxPython/src/gtk/_gdi_wrap.cpp
./3rdparty/build/wxPython-src-2.8.12.1/wxPython/src/gtk/_windows_wrap.cpp
./3rdparty/build/wxPython-src-2.8.12.1/wxPython/src/gtk/_controls_wrap.cpp
there are sections which look like this:
863 } else { 864 PyErr_Format(PyExc_RuntimeError, mesg); 865 }
Compiling with -Wformat-security means that you'll have to patch all those expression to
863 } else { 864 PyErr_Format(PyExc_RuntimeError, "%s", mesg); 865 }
There were similar issue with wxLog*(m) statements in other files, e.g.
3rdparty/build/wxPython-src-2.8.12.1/wxPython/src/gtk/_misc_wrap.cpp -> ("%s", m.c_str()) 3093 m.Replace(wxT("%"), wxT("%%")); 3094 wxLogFatalError(m); 3095 } .. 3177 m.Replace(wxT("%"), wxT("%%")); 3178 wxLogTrace(mask, m); 3179 }

30 May 2013

433. Wine 1.5.31 on Debian

Here's a generic way of building wine which works for 1.5.31 (and 1.5.28 and everything in between except 1.5.31).




See here for information about 3D acceleration using libGL/U with Wine: http://verahill.blogspot.com.au/2013/05/429-briefly-wine-libglliubglu-blender.html

Getting started:
If you set up a e.g. chroot to build 1.5.28 before, you don't need to set up a new chroot to build 1.5.31. In that case, skip the set-up step below and instead re-enter your existing chroot like this:
sudo mount -o bind /proc wine32/proc
sudo cp /etc/resolv.conf wine32/etc/resolv.conf
sudo chroot wine32
su sandbox
cd ~/tmp

Setting up the Chroot
sudo apt-get install debootstrap
mkdir $HOME/tmp/architectures/wine32 -p
cd $HOME/tmp/architectures
sudo debootstrap --arch i386 wheezy $HOME/tmp/architectures/wine32 http://ftp.au.debian.org/debian/
sudo mount -o bind /proc wine32/proc
sudo cp /etc/resolv.conf wine32/etc/resolv.conf
sudo chroot wine32

You're now in the chroot:
apt-get update
apt-get install locales sudo vim
echo 'export LC_ALL="C"'>>/etc/bash.bashrc
echo 'export LANG="C"'>>/etc/bash.bashrc
echo '127.0.0.1 localhost beryllium' >> /etc/hosts
source /etc/bash.bashrc
adduser sandbox
usermod -g sudo sandbox
echo 'Defaults !tty_tickets' >> /etc/sudoers
su sandbox
cd ~/

Replace 'beryllium' with the name your host system (it's just to suppress error messages)

Building Wine
While still in the chroot, continue (the i386 is ok; don't worry about it -- you don't actually need it):

sudo apt-get install libx11-dev:i386 libfreetype6-dev:i386 libxcursor-dev:i386 libxi-dev:i386 libxxf86vm-dev:i386 libxrandr-dev:i386 libxinerama-dev:i386 libxcomposite-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libdbus-1-dev:i386 libgnutls-dev:i386 libncurses-dev:i386 libsane-dev:i386 libv4l-dev:i386 libgphoto2-2-dev:i386 liblcms-dev:i386 libgstreamer-plugins-base0.10-dev:i386 libcapi20-dev:i386 libcups2-dev:i386 libfontconfig-dev:i386 libgsm1-dev:i386 libtiff-dev:i386 libpng-dev:i386 libjpeg-dev:i386 libmpg123-dev:i386 libopenal-dev:i386 libldap-dev:i386 libxrender-dev:i386 libxml2-dev:i386 libxslt-dev:i386 libhal-dev:i386 gettext:i386 prelink:i386 bzip2:i386 bison:i386 flex:i386 oss4-dev:i386 checkinstall:i386 ocl-icd-libopencl1:i386 opencl-headers:i386 libasound2-dev:i386 build-essential
mkdir ~/tmp
cd ~/tmp
wget http://prdownloads.sourceforge.net/wine/wine-1.5.31.tar.bz2
tar xvf wine-1.5.31.tar.bz2
cd wine-1.5.31/./configure
time make -j3
sudo checkinstall --install=no
checkinstall 1.6.2, Copyright 2009 Felipe Eduardo Sanchez Diaz Duran This software is released under the GNU GPL. The package documentation directory ./doc-pak does not exist. Should I create a default set of package docs? [y]: Preparing package documentation...OK Please write a description for the package. End your description with an empty line or EOF. >> wine 1.5.31 >> ***************************************** **** Debian package creation selected *** ***************************************** This package will be built according to these values: 0 - Maintainer: [ root@beryllium ] 1 - Summary: [ wine 1.5.31] 2 - Name: [ wine ] 3 - Version: [ 1.5.31] 4 - Release: [ 1 ] 5 - License: [ GPL ] 6 - Group: [ checkinstall ] 7 - Architecture: [ i386 ] 8 - Source location: [ wine-1.5.31 ] 9 - Alternate source location: [ ] 10 - Requires: [ ] 11 - Provides: [ wine ] 12 - Conflicts: [ ] 13 - Replaces: [ ]
Checkinstall takes a little while (In particular this step: 'Copying files to the temporary directory...').

Installing Wine

Exit the chroot
sandbox@beryllium:~/tmp/wine-1.5.31$ exit
exit
root@beryllium:/# exit
exit
me@beryllium:~/tmp/architectures$ 

On your host system
 Enable multiarch* and install ia32-libs, since you've built a proper 32 bit binary:

sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install ia32-libs

*At some point I think ia32-libs may be replaced by proper multiarch packages, but maybe not. So we're kind of doing both here.

 Copy the .deb package and install it
sudo cp wine32/home/sandbox/tmp/wine-1.5.31/wine_1.5.31-1_i386.deb .
sudo chown $USER wine_1.5.31-1_i386.deb
sudo dpkg -i wine_1.5.31-1_i386.deb

Links to this post:
http://appdb.winehq.org/objectManager.php?sClass=version&iId=19141&iTestingId=79415&bShowAll=true

19 May 2013

423. Openblas on Debian Wheezy

Update 23 May 2013: the url for the openblas tarball has been updated and a tab has been removed from the lapack url.

Note that the links to the openblas file tends to die after a while, so you might have to download it manually.

Original post:
I need more modular posts. I've covered how to build openblas several times before, but I need a simple post I can use as a reference. So here it is:

openblas
sudo mkdir /opt/openblas
sudo chown ${USER} /opt/openblas
cd ~/tmp
wget http://github.com/xianyi/OpenBLAS/tarball/v0.2.6

tar xvf v0.2.6
cd xianyi-OpenBLAS-87b4d0c
wget http://www.netlib.org/lapack/lapack-3.4.1.tgz
make all BINARY=64 CC=/usr/bin/gcc FC=/usr/bin/gfortran USE_THREAD=0 INTERFACE64=1 1> make.log 2>make.err
make PREFIX=/opt/openblas install
nm: '/lib/libpthread.so*': No such file make -j 2 -f Makefile.install install make[1]: Entering directory `/home/me/tmp/xianyi-OpenBLAS-87b4d0c' Generating openblas_config.h in /opt/openblas/include Generating f77blas.h in /opt/openblas/include Generating cblas.h in /opt/openblas/include Copying LAPACKE header files to /opt/openblas/lib Copy the static library to /opt/openblas/lib cp: cannot stat `libopenblas_opteronp-r0.2.6.a': No such file or directory make[1]: *** [install] Error 1 make[1]: Leaving directory `/home/me/tmp/xianyi-OpenBLAS-87b4d0c' make: *** [install] Error 2
Don't worry about the errors -- as long as you have libopenblas_PROCESSOR_rX.X.X.so, you're fine.

cp lib*.* /opt/openblas/lib

add
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/openblas/lib
to your ~/.bashrc, and/or add

/opt/openblas/lib

to your /etc/ld.so.conf

16 May 2013

417. Briefly: Patching kernel 3.9 with the CK patchset: 3.9-ck-1

Nothing strange here -- basically the same as http://verahill.blogspot.com.au/2013/04/395-ck-kernel-on-debian-and-patching.html but with updated links.

I haven't found a good and succinct description of what the -ck patch set does and that I could link to here, but here's what it says on the Arch -ck page:
"..many Archers elect to use this package for the BFS' excellent desktop interactivity and responsiveness under any load situation. Additionally, the bfs imparts performance gains beyond interactivity"

I don't know if there are objective benchmarks that one can use to demonstrate an improvement in 'responsiveness and interactivity'. Subjectively, however, I feel that there's a slight improvement. You decide for yourself.

Begin here

sudo apt-get install kernel-package fakeroot build-essential ncurses-dev
mkdir ~/tmp
cd ~/tmp
wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.9.2.tar.xz
tar xvf linux-3.9.2.tar.xz
cd linux-3.9.2/
wget http://ck.kolivas.org/patches/3.0/3.9/3.9-ck1/patch-3.9-ck1.bz2
bunzip2 patch-3.9-ck1.bz2
patch -p1 < patch-3.9-ck1
patching file arch/powerpc/platforms/cell/spufs/sched.c patching file Documentation/scheduler/sched-BFS.txt patching file Documentation/sysctl/kernel.txt patching file fs/proc/base.c patching file include/linux/init_task.h patching file include/linux/ioprio.h patching file include/linux/sched.h Hunk #6 succeeded at 2738 (offset -10 lines). patching file init/Kconfig patching file init/main.c patching file kernel/delayacct.c patching file kernel/exit.c patching file kernel/posix-cpu-timers.c patching file kernel/sysctl.c patching file lib/Kconfig.debug patching file include/linux/jiffies.h patching file drivers/cpufreq/cpufreq.c patching file drivers/cpufreq/cpufreq_ondemand.c patching file drivers/cpufreq/cpufreq_conservative.c patching file kernel/sched/bfs.c patching file kernel/sched/Makefile patching file include/uapi/linux/sched.h patching file include/linux/sched/rt.h patching file kernel/stop_machine.c patching file include/linux/swap.h patching file mm/memory.c patching file mm/swapfile.c patching file mm/vmscan.c patching file arch/x86/Kconfig patching file kernel/Kconfig.hz patching file kernel/Kconfig.preempt patching file Makefile
make-kpkg clean cat /boot/config-`uname -r`>.config make oldconfig

You might now be asked a long series of questions about how the kernel should be configured (or you might not be -- depending on what kernel version you're currently running). In MOST cases you can select the default option (i.e. hit enter) but you should still read each question and consider it. Making a mistake won't break your computer, so don't be scared.

Next, start the compilation (will take a while):

time fakeroot make-kpkg -j4 --initrd kernel_image kernel_headers
sudo dpkg -i ../linux*3.9-ck*.deb

where 4 is the number of cores on your machine (note: it only has to do with compiling -- you can use the compiled binaries on any number of cores).

Anyway, that's all -- you've now patched, compiled and installed a new kernel. And it didn't even hurt.

416. Compiling Wine 1.5.30 in a chroot (fixed)

Update 2, 22 May 2013: Thanks to the Anonymous poster who pointed out that wine 1.5.30 was broken! Anyway, I've updated this post with instructions how to patch the wine 1.5.30 sources so that it includes libwine in the final .deb package.

(I normally attached a screenshot of the winecfg about tab, but not this time -- had I done that I would've realised something was wrong. )

It's fixed now. Wine 1.5.30 is OK again.






Update 22 May 2013: libwine.so.1 doesn't get included in the deb package, which causes severely reduced functionality. I've confirmed that Wine 1.5.28 built as shown in http://verahill.blogspot.com.au/2013/04/387-compiling-wine-1528-in-i386-chroot.html works fine though.

I'll update here when I've figured out why the compiled libraries don't get included.

It's similar to what is mentioned in these bug reports:
https://bugs.archlinux.org/task/35189
https://bugs.archlinux.org/task/35190
https://bugs.archlinux.org/task/35191

There's a fix here: http://bugs.winehq.org/attachment.cgi?id=44422


Original post:
While it'd be absolutely fair to accuse me of recycling posts, I have a reasonably good reason for doing so: posting build instructions for the latest version -- even if identical to instructions for earlier versions -- confirms that it 'works'. Also, it shows that the instructions are current.

I'm too much of a hoarder to go back and update old posts.

Anyway, here's a generic way of building wine which works for 1.5.30 (and 1.5.28 and everything in between). And yes, I've copy/pasted from my old 1.5.28 post...

See here for information about 3D acceleration using libGL/U: http://verahill.blogspot.com.au/2013/05/429-briefly-wine-libglliubglu-blender.html

Getting started:
If you set up a chroot to build 1.5.28 before, you don't need to set up a new chroot to build 1.5.30. In that case, skip the set-up step below and instead re-enter your existing chroot like this:
sudo mount -o bind /proc wine32/proc
sudo cp /etc/resolv.conf wine32/etc/resolv.conf
sudo chroot wine32
su sandbox
cd ~/tmp

Setting up the Chroot
sudo apt-get install debootstrap
mkdir $HOME/tmp/architectures/wine32 -p
cd $HOME/tmp/architectures
sudo debootstrap --arch i386 wheezy $HOME/tmp/architectures/wine32 http://ftp.au.debian.org/debian/
sudo mount -o bind /proc wine32/proc
sudo cp /etc/resolv.conf wine32/etc/resolv.conf
sudo chroot wine32

You're now in the chroot:
apt-get update
apt-get install locales sudo vim
echo 'export LC_ALL="C"'>>/etc/bash.bashrc
echo 'export LANG="C"'>>/etc/bash.bashrc
echo '127.0.0.1 localhost beryllium' >> /etc/hosts
source /etc/bash.bashrc
adduser sandbox
usermod -g sudo sandbox
echo 'Defaults !tty_tickets' >> /etc/sudoers
su sandbox
cd ~/

Replace 'beryllium' with the name your host system (it's just to suppress error messages)

Building Wine
While still in the chroot, continue (the i386 is ok; don't worry about it -- you don't actually need it):

sudo apt-get install libx11-dev:i386 libfreetype6-dev:i386 libxcursor-dev:i386 libxi-dev:i386 libxxf86vm-dev:i386 libxrandr-dev:i386 libxinerama-dev:i386 libxcomposite-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libdbus-1-dev:i386 libgnutls-dev:i386 libncurses-dev:i386 libsane-dev:i386 libv4l-dev:i386 libgphoto2-2-dev:i386 liblcms-dev:i386 libgstreamer-plugins-base0.10-dev:i386 libcapi20-dev:i386 libcups2-dev:i386 libfontconfig-dev:i386 libgsm1-dev:i386 libtiff-dev:i386 libpng-dev:i386 libjpeg-dev:i386 libmpg123-dev:i386 libopenal-dev:i386 libldap-dev:i386 libxrender-dev:i386 libxml2-dev:i386 libxslt-dev:i386 libhal-dev:i386 gettext:i386 prelink:i386 bzip2:i386 bison:i386 flex:i386 oss4-dev:i386 checkinstall:i386 ocl-icd-libopencl1:i386 opencl-headers:i386 libasound2-dev:i386 build-essential
mkdir ~/tmp
cd ~/tmp
wget http://prdownloads.sourceforge.net/wine/wine-1.5.30.tar.bz2
tar xvf wine-1.5.30.tar.bz2
cd wine-1.5.30/
wget http://bugs.winehq.org/attachment.cgi?id=44422 -O diff.patch
patch -p1 < diff .patch
patching file configure patching file configure.ac patching file libs/wine/Makefile.in
./configure time make -j3 sudo checkinstall --install=no
checkinstall 1.6.2, Copyright 2009 Felipe Eduardo Sanchez Diaz Duran This software is released under the GNU GPL. The package documentation directory ./doc-pak does not exist. Should I create a default set of package docs? [y]: Preparing package documentation...OK Please write a description for the package. End your description with an empty line or EOF. >> wine 1.5.30-2 >> ***************************************** **** Debian package creation selected *** ***************************************** This package will be built according to these values: 0 - Maintainer: [ root@beryllium ] 1 - Summary: [ wine 1.5.30-2 ] 2 - Name: [ wine ] 3 - Version: [ 1.5.30-2 ] 4 - Release: [ 1 ] 5 - License: [ GPL ] 6 - Group: [ checkinstall ] 7 - Architecture: [ i386 ] 8 - Source location: [ wine-1.5.30 ] 9 - Alternate source location: [ ] 10 - Requires: [ ] 11 - Provides: [ wine ] 12 - Conflicts: [ ] 13 - Replaces: [ ]
Compilation took ca 13 minutes with three threads. Checkinstall takes a little while (In particular this step: 'Copying files to the temporary directory...').

Installing Wine

Exit the chroot
sandbox@beryllium:~/tmp/wine-1.5.30$ exit
exit
root@beryllium:/# exit
exit
me@beryllium:~/tmp/architectures$ 

On your host system
 Enable multiarch* and install ia32-libs, since you've built a proper 32 bit binary:

sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install ia32-libs

*At some point I think ia32-libs may be replaced by proper multiarch packages, but maybe not. So we're kind of doing both here.

 Copy the .deb package and install it
sudo cp wine32/home/sandbox/tmp/wine-1.5.30/wine_1.5.30-1_i386.deb .
sudo chown $USER wine_1.5.30-1_i386.deb
sudo dpkg -i wine_1.5.30-1_i386.deb

09 May 2013

409.A.GAMESS US with GPU support on debian wheezy. This works (probably).


Update 27/6/2013:
Please note that Kirill Berezovsky has published a series of posts on GAMESS US, including how to compile it for both CPU and GPU use. See
http://biochemicalmatters.blogspot.com.au/2013/06/gamess-us-frequently-asked-questions_26.html
http://biochemicalmatters.blogspot.ru/2013/06/gamess-us-frequently-asked-questions_1687.html
http://biochemicalmatters.blogspot.ru/2013/06/gamess-us-frequently-asked-questions_1447.html
http://biochemicalmatters.blogspot.com.au/2013/06/gamess-us-frequently-asked-questions.html


Update 21 May 2013: See the comments below this post. This approach most likely works -- what has been confusing me is the lack of reports of GPU timings in the output, but this doesn't necessarily mean that the GPU isn't being used. The poster below this post, using nvidia-smi, observed GPU usage, although the speed-up was not major.


Update 10/05/2013: fixed libcchem compile.

Everything compiles fine and computations run fine and fast. To date there's only one other detailed step-by-step example of successful compilation of GAMESS with GPU support out there. At least based on google.

For various reasons I'm beginning to suspect that ATLAS isn't working out for me -- I've had issues getting things to converge with ATLAS, but which work fine with ACML (see post B).

I was in part following http://combichem.blogspot.com.au/2011/02/compiling-gamess-with-cuda-gpu-support.html and ./libcchem/aaa.readme.1st

This took a while to hammer out, so the write-up is a bit messy.


Set up
sudo apt-get install libboost-all-dev build-essential g++ gfortran automake nvidia-cuda-toolkit python-cheetah openmpi-bin libopenmpi-dev zlib1g-dev checkinstall
mkdir ~/tmp

Get gamess (see e.g. http://verahill.blogspot.com.au/2012/09/compiling-and-testing-gamess-us-on.html).

Put gamess-current.tar.gz in  ~/tmp

cd ~/tmp
tar xvf gamess-current.tar.gz
sudo mv gamess /opt/gamess_cuda
sudo chown $USER:$USER /opt/gamess_cuda -R


Preparing Boost
Edit /usr/include/boost/mpl/aux_/integral_wrapper.hpp
47 // other compilers (e.g. MSVC) are not particulary happy about it 48 #if BOOST_WORKAROUND(__EDG_VERSION__, <= 238) || defined(__CUDACC__) 49 typedef struct AUX_WRAPPER_NAME type;
Edit /usr/include/boost/mpl/size_t_fwd.hpp
20 21 BOOST_MPL_AUX_ADL_BARRIER_NAMESPACE_OPEN 22 #if defined(__CUDACC__) 23 typedef std::size_t std_size_t; 24 template< std_size_t N > struct size_t; 25 #else 26 template< std::size_t N > struct size_t; 27 #endif 28 29 BOOST_MPL_AUX_ADL_BARRIER_NAMESPACE_CLOSE
Edit /usr/include/boost/mpl/size_t.hpp
19 #if defined(__CUDACC__) 20 #define AUX_WRAPPER_VALUE_TYPE std_size_t 21 #define AUX_WRAPPER_NAME size_t 22 #define AUX_WRAPPER_PARAMS(N) std_size_t N 23 #else 24 #define AUX_WRAPPER_VALUE_TYPE std::size_t 25 #define AUX_WRAPPER_NAME size_t 26 #define AUX_WRAPPER_PARAMS(N) std::size_t N 27 #endif 28

HDF5
You'll have to compile that yourself for now since H5Cpp.h missing in the debian packages.(i.e. cxx support)

mkdir ~/tmp
cd ~/tmp
wget http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.10-patch1.tar.gz
tar xvf hdf5-1.8.10-patch1.tar.gz
cd hdf5-1.8.10-patch1/
export CC=/usr/bin/gcc-4.6 && export CXX=/usr/bin/g++-4.6
./configure --prefix=/opt/gamess_cuda/hdf5 --with-pthread --enable-cxx --enable-threadsafe --enable-unsupported
make
mkdir /opt/gamess_cuda/hdf5/lib -p
mkdir /opt/gamess_cuda/hdf5/include -p
sudo checkinstall
This package will be built according to these values: 0 - Maintainer: [ root@neon ] 1 - Summary: [ hdf5-cxx] 2 - Name: [ hdf5-1.8.10 ] 3 - Version: [ 1.8.10-1 ] 4 - Release: [ 1 ] 5 - License: [ GPL ] 6 - Group: [ checkinstall ] 7 - Architecture: [ amd64 ] 8 - Source location: [ hdf5-1.8.10-patch1 ] 9 - Alternate source location: [ ] 10 - Requires: [ ] 11 - Provides: [ hdf5-1.8.10 ] 12 - Conflicts: [ ] 13 - Replaces: [ ]
Make sure to edit the Version field since Patch-1 leads to an error (must start with digit).
Openmpi 1.6 Can't remember why I ended up compiling it myself instead of using the stock debian version. From here.

sudo apt-get install build-essential gfortran
wget http://www.open-mpi.org/software/ompi/v1.6/downloads/openmpi-1.6.tar.bz2
tar xvf openmpi-1.6.tar.bz2
cd openmpi-1.6/

sudo mkdir /opt/openmpi/
sudo chown ${USER} /opt/openmpi/
./configure --prefix=/opt/openmpi/1.6/ --with-sge

make
make install

compiling libcchem
cd /opt/gamess_cuda/libcchem
edit /opt/gamess_cuda/libcchem/rysq/src/externals/boost/cuda/device_ptr.hpp
  4 #include <cstdlib>
  5 #include <iterator>
  6 #include <stddef.h>
  7 
  8 namespace boost {
Edit /opt/gamess_cuda/libcchem/src/externals/boost/cuda/device_ptr.hpp
  4 #include <cstdlib>
  5 #include <iterator>
  6 #include <stddef.h>
  7 
  8 namespace boost {
  9 namespace cuda {
./configure --with-gamess --with-hdf5=/opt/gamess_cuda/hdf5 CPPFLAGS="-I/opt/gamess_cuda/hdf5/include" --with-cuda=/usr --disable-openmp --prefix=/opt/gamess_cuda/libcchem --with-gpu=fermi --with-integer8 --with-cublas
make
make install

Configure Gamess US Mainly follow this: http://verahill.blogspot.com.au/2012/09/compiling-and-testing-gamess-us-on.html
cd /opt/gamess_cuda
./config
please enter your target machine name: linux64 GAMESS directory? [/opt/gamess_cuda] /opt/gamess_cuda Setting up GAMESS compile and link for GMS_TARGET=linux64 GAMESS software is located at GMS_PATH=/opt/gamess_cuda Please provide the name of the build locaation. This may be the same location as the GAMESS directory. GAMESS build directory? [/home/me/tmp/gamess] Please provide a version number for the GAMESS executable. This will be used as the middle part of the binary's name, for example: gamess.00.x Version? [00] 12r2 Please enter your choice of FORTRAN: gfortran gfortran is very robust, so this is a wise choice. Please type 'gfortran -dumpversion' or else 'gfortran -v' to detect the version number of your gfortran. This reply should be a string with at least two decimal points, such as 4.1.2 or 4.6.1, or maybe even 4.4.2-12. The reply may be labeled as a 'gcc' version, but it is really your gfortran version. Please enter only the first decimal place, such as 4.1 or 4.6: 4.6
Enter your choice of 'mkl' or 'atlas' or 'acml' or 'none': atlas Please enter the Atlas subdirectory on your system: /opt/ATLAS/lib Math library 'atlas' will be taken from /opt/ATLAS If you have an expensive but fast network like Infiniband (IB), and if you have an MPI library correctly installed, choose 'mpi'. communication library ('sockets' or 'mpi')? mpi Enter MPI library (impi, mvapich2, mpt, sockets): openmpi
Please enter your openmpi's location: /opt/openmpi/1.6

Build Gamess US
cd /opt/gamess_cuda/ddi/
./compddi
cd ../

Edit comp
872 # see ~/gamess/libcchem/aaa.readme.1st for more information 873 set GPUCODE=true 874 if ($GPUCODE == true) then
and
1663 # -fno-whole-file suppresses argument's data type checking 1664 set OPT='-O0' 1665 if (".$GMS_DEBUG_FLAGS" != .) set OPT="$GMS_DEBUG_FLAGS"
./compall

Edit lked
69 # 70 set GPUCODE=true 71 # 72 # 5. optional MPQC interface
and
958 case openmpi: 959 set MPILIBS="-L$GMS_MPI_PATH/lib" 960 set MPILIBS="$MPILIBS -lmpi -lpthread" 961 breaksw
and
1214 if ($GPUCODE == true) then 1215 echo " Using 'libcchem' add-in C++ codes for Nvidia/CUDA GPUs." 1216 set GPU_LIBS="-L/opt/gamess_cuda/libcchem/lib -lcchem_gamess -lcchem -lrysq" 1217 set GPU_LIBS="$GPU_LIBS -lcudart -lcublas" 1218 ### GPU_LIBS="$GPU_LIBS -lcudart -lcublas" 1219 set GPU_LIBS="$GPU_LIBS /usr/lib/libboost_thread.a" 1220 set GPU_LIBS="$GPU_LIBS /opt/gamess_cuda/hdf5/lib/libhdf5.a" 1221 set GPU_LIBS="$GPU_LIBS /opt/gamess_cuda/hdf5/lib/libhdf5_cpp.a" 1222 set GPU_LIBS="$GPU_LIBS /opt/gamess_cuda/hdf5/lib/libhdf5_hl.a" 1223 set GPU_LIBS="$GPU_LIBS /opt/gamess_cuda/hdf5/lib/libhdf5.a" 1224 set GPU_LIBS="$GPU_LIBS /opt/ATLAS/lib/libcblas.a" 1225 set GPU_LIBS="$GPU_LIBS -lz" 1226 set GPU_LIBS="$GPU_LIBS -lstdc++" 1227 ### GPU_LIBS="$GPU_LIBS -lgomp" 1228 set GPU_LIBS="$GPU_LIBS -lpthread" 1229 echo " libcchem GPU code's libraries are" 1230 echo "$GPU_LIBS" 1231 else

./lked gamess gpu.12

Create gpurun
#!/bin/csh set TARGET=mpi set SCR=$HOME/scratch set USERSCR=/scratch set GMSPATH=/opt/gamess_cuda set JOB=$1 set VERNO=$2 set NCPUS=$3 @ NUMGPU=1 if ($NUMGPU > 0) then @ NUMCPU = $NCPUS - 1 echo libcchem kernels will use $NUMCPU cores and $NUMGPU GPUs per node... set echo setenv CCHEM_PROFILE 1 setenv NUM_THREADS $NCPUS #--if ($NUMGPU == 0) setenv GPU_DEVICES -1 #--if ($NUMGPU == 2) setenv GPU_DEVICES 0,1 #--if ($NUMGPU == 4) setenv GPU_DEVICES 0,1,2,3 #setenv LD_LIBRARY_PATH /share/apps/cuda/lib64:$LD_LIBRARY_PATH ###### LD_LIBRARY_PATH /usr/local/cuda/lib64:$LD_LIBRARY_PATH unset echo else setenv GPU_DEVICES -1 endif if ( $JOB:r.inp == $JOB ) set JOB=$JOB:r echo "Copying input file $JOB.inp to your run's scratch directory..." cp $JOB.inp $SCR/$JOB.F05 setenv TRAJECT $USERSCR/$JOB.trj setenv RESTART $USERSCR/$JOB.rst setenv INPUT $SCR/$JOB.F05 setenv PUNCH $USERSCR/$JOB.dat if ( -e $TRAJECT ) rm $TRAJECT if ( -e $PUNCH ) rm $PUNCH if ( -e $RESTART ) rm $RESTART source $GMSPATH/gms-files.csh setenv LD_LIBRARY_PATH /opt/openmpi/lib:$LD_LIBRARY_PATH set path= ( /opt/openmpi/bin $path ) mpiexec -n $NCPUS $GMSPATH/gamess.gpu.$VERNO.x|tee $JOB.out cp $PUNCH .

echo 'export PATH=$PATH:/opt/gamess_cuda' >> ~/.bashrc
source ~/.bashrc
chmod +x gpurun
cd test/standard/
 gpurun exam44 12 2


The only evidence of GPU usage in the output is e.g. in exam44.out:
388           -----------------------
389           MP2 CONTROL INFORMATION
390           -----------------------
391           NACORE =        6  NBCORE =        6
392           LMOMP2 =        F  AOINTS = DUP
393           METHOD =        2  NWORD  =               0
394           MP2PRP =        F  OSPT   = NONE
395           CUTOFF = 1.00E-09  CPHFBS = BASISAO
396           CODE   = GPU
397 
398           NUMBER OF CORE -A-  ORBITALS =     6
399           NUMBER OF CORE -B-  ORBITALS =     6

but in the summary only CPU utilisation is mentioned.



I modified rungms:

me@neon:/opt/gamess_cuda/tests/standard$ diff /opt/gamess_cuda/gpurungms /opt/gamess/rungms 
59,62c59,62
< set TARGET=mpi
< set SCR=$HOME/scratch
< set USERSCR=/scratch
< set GMSPATH=/opt/gamess_cuda
---
> set TARGET=sockets
> set SCR=/scr/$USER
> set USERSCR=~$USER/scr
> set GMSPATH=/u1/mike/gamess
67d66
< set NNODES=1
513c512
< set PPN=$3
---
>    set PPN=$4
601c600
<          @ PPN2 = $PPN
---
>          @ PPN2 = $PPN + $PPN
742c741
<    @ NUMGPU=1
---
>    @ NUMGPU=0
752c751
< #      setenv LD_LIBRARY_PATH /share/apps/cuda/lib64:$LD_LIBRARY_PATH
---
>       setenv LD_LIBRARY_PATH /share/apps/cuda/lib64:$LD_LIBRARY_PATH
793c792,793
<       /opt/openmpi/1.6/bin/mpiexec -n $NPROCS $GMSPATH/gamess.$VERNO.x < /dev/null
---
>       mpiexec.hydra -f $PROCFILE -n $NPROCS \
>             /home/mike/gamess/gamess.$VERNO.x < /dev/null

29 April 2013

401. AMD FX 8150:issues building kernel -- random failures.

Update 4:  I found the receipts for one pair of sticks and took it to MSY in Melbourne -- they were replaced on the spot without any questions asked. Very happy.

Update 3: The errors were all due to 3 bad ram sticks. Using the only good stick everything works fine. That's 24 Gb of bad ram...this won't be cheap if I can't find the receipts...

Update 2:
Running memtest86 I caught lots of errors (51 in 50 minutes) before I killed the test. I'm currently testing each stick one by one. I'm hoping that what is seemingly RAM errors can be caused by inapproriate BIOS settings, because 32 Gb bios is not cheap to replace...

While I'm swapping RAM sticks I'm also testing a separate set of stick on a different box. If they are error free it will be interesting to see if they trigger errors on the troublesome node. I'm still hoping for BIOS as being the culprit...

So far three out of four tested sticks have shown errors -- they all happen during test #6. The fourth stick has passed all tests seven times.

Update 1: dmesg also shows the same message as the OP here sees: https://bugzilla.redhat.com/show_bug.cgi?id=909702

The OP puts it down to a misconfigured bios, so the quest continues.
Searching for 990FX and FX8150 I get a number of hits:

Here's a newegg review for 990FX:

 I purchased This MB to run with the AMD FX 8150. I have built computers from high end to low end and know the ones in the middle last the longest and are the most stable.
[..]
At this point the fun of the build is gone, and I have too many hours dealing with problems. 
And that's not the only negative FX8?50 + 990FX review.

The worst part of it is that I've been thinking about building another, identical node (good value for money) as well as recommending my build to a student whom is about to do calcs.

Mind you, I've only ever had issues when it comes to compiling the kernel -- it's been solid when it comes to running calculations.

Original post:


NOTE: this is NOT a solution. Just observations.

My AMD FX 8150 is a great CPU -- it makes up the heart of the fastest of my computational nodes, and is eminently affordable. It does, however, cause me grief in one respect -- I can't compile the linux kernel.

The system
The box that's causing me trouble has
* AMD FX 8150 cpu
* gigabyte 990FXA-D3 motherboard
* nvidia GeForce 210 video card
* Corsair GS 800 PSU
* 4x8 Gb patriot viper PV316G186C0K RAM
While not top of the range, the components should be of reasonable quality.

In terms of software and OS, it's an up-to-date wheezy install (gcc 4.7), running kernel 3.7.2 (compiled on a different machine).

Compiling the kernel
I'm compiling the kernel as shown here: http://verahill.blogspot.com.au/2013/02/342-compiling-kernel-38-on-debian.html

The errors are shown at the end of the post

The fact that the errors keep changing might also be pointing towards there being a hardware fault with my CPU, rather than with FX 8150 in general.

3.8 built fine twice, and crashed the third time. 3.8.10 crashed twice, then built fine the third time.

It all sounds like I'm having hardware issues...but they only seem to be triggered during kernel builds. During 'normal use (i.e. using 100% cpu for weeks at a time) it is perfectly stable. Compiling e.g. nwchem (another pretty heavy compile) also goes absolutely fine.

Troubleshooting something like this also wouldn't be easy. See the end of the post for a list over various errors that I was getting during compilation of different kernel versions.

Anyway, I hit google...



BIOS
That Windows has issues with 8150 might seem unrelated, but it appeared that my errors could be solved by a bios update to my 990 fxa-d3 mobo:
 http://scalibq.wordpress.com/2011/10/19/amd-bulldozer-can-it-get-even-worse/
"The actual reported error is quite random, it just depends on where the CPU fails first. So you generally get a different error code with every BSOD."
and
"AMD’s KB article focuses solely on some boards with the 990FX chipset."
Well, I do have a 990FXA-D3 gigabyte motherboard.

My bios is shown by lshw as
*-firmware
          description: BIOS
          vendor: Award Software International, Inc.
          physical id: 0
          version: F7
          date: 05/30/2012
          size: 128KiB
          capacity: 4032KiB
So the obvious solution was to flash the bios.

Turns out, flashing the BIOS is a headache on Gigabyte motherboards (not buying anything from them again). What happened with simply burning a CD and booting with it in the drive?

Flashing the bios

I downloaded the bios (version F8): http://download.gigabyte.eu/FileList/BIOS/mb_bios_ga-990fxa-d3_f8.exe.

I unzipped it with 7z, giving me 990FXAD3.F8 -- I then put that file in the root of a USB stick..

I've tried with a number of USB sticks, including a blank stick formatted with W95 Fat32 and keeping the stick plugged in before rebooting.

In Q-flash, I always ended up with a prompt saying Floppy A <Drive>, and when I hit enter it says '..    <dir>'. 0 Files found. Yet it also said Total size 7.48G, Free Size: 7.44 G, which matched the size of the USB stick.

Finally I managed to get it to work:
*  in fdisk I only created a 1 gb partition on the USB stick, set type (t) to 6 (Fat16), made it bootable, and wrote changes to disk.
* I then ran mkdosfs -F 16 /dev/sdb1 (my usb stick was /dev/sdb).
*  I then copied the 990FXD3.F8 file to the usb stick root (after mounting it of course) and THAT worked.

Memtest86
Because RAM has traditionally been a major culprit behind hardware errors (especially the random, difficult-to-diagnose type) it's always a good idea to run a memtest. To do that, install memtest86+ (sudo apt-get install memtest86+) and reboot. There should be a new menu item (scroll down) in grub. Memtest takes quite a while, especially if you have a lot of RAM (32 Gb...).

Lo and behold, there are errors:
Tst  Pass  Failing Address              Good        Bad        Err-Bits  Count Chan
------------------------------------------------------------------------------------
6     0     0007383b4f4  -  1848.2MB   fffffbff     ffffffff   00000400    1
6     0     00039c1f294  -   924.1MB   fffffbff     ffffffff   00000004    2
6     0     00120203034  -  4610.0MB   00000004     00000000   00000004    3
6     0     001ca16c464  -  7329.4MB   00020004     00000000   00020000    4
[..]

I counted 51 errors before killing the test (time to identify the bad stick). Many of these occurred in a more limited address space than those shown above. Sigh...the RAM was the most expensive part of this build...

According to this there's a slight chance that the RAM might be ok, but it's still not a good sign.

I've tested each stick by itself -- so far 3 out of 4 sticks have yielded errors during test 6. I did seven passes on the fourth stick and no errors.

The outcome
However, even with the new bios the kernel compiles still fail -- it takes longer for it to fail, but it fails.

I do see the odd thing in dmesg though:
[ 4260.342268] as[29370]: segfault at 4541b5e ip 0000000000410306 sp 00007fff40ec4420 error 4 in as[400000+51000]

So either FX 8150 is still not properly supported by the BIOS, or I've bought a lemon.

The question remains: why do I only see failure during kernel compiles and no other conditions?

After bios flash:

Kernel 3.9-rc8
  CC      drivers/base/dd.o
In file included from /home/me/tmp/linux-3.9-rc8/arch/x86/include/asm/processor.h:23:0,
                 from /home/me/tmp/linux-3.9-rc8/arch/x86/include/asm/atomic.h:6,
                 from include/linux/atomic.h:4,
                 from include/linux/sysfs.h:20,
                 from include/linux/kobject.h:21,
                 from include/linux/device.h:17,
                 from drivers/base/dd.c:20:
/home/me/tmp/linux-3.9-rc8/arch/x86/include/asm/special_insns.h: In function 'native_read_cr0':
/home/me/tmp/linux-3.9-rc8/arch/x86/include/asm/special_insns.h:24:2: internal compiler error: in build_int_cst_wide, at tree.c:1238
Please submit a full bug report,
with preprocessed source if appropriate.
See  for instructions.
The bug is not reproducible, so it is likely a hardware or OS problem.
make[3]: *** [drivers/base/dd.o] Error 1
make[2]: *** [drivers/base] Error 2
make[1]: *** [drivers] Error 2
make[1]: Leaving directory `/home/me/tmp/linux-3.9-rc8'
make: *** [debian/stamp/build/kernel] Error 2
Kernel 3.8.10
  UPD     include/generated/compile.h
  CC      init/version.o
  LD      init/built-in.o
ipc/built-in.o:(.debug_info+0x1ed81): undefined reference to `.LASF108'
make[1]: *** [vmlinux] Error 1
make[1]: Leaving directory `/home/me/tmp/linux-3.8.10'
make: *** [debian/stamp/build/kernel] Error 2
Kernel 3.7.6
  CC [M]  fs/gfs2/super.o
  CC [M]  fs/gfs2/sys.o
In file included from /home/me/tmp/linux-3.7.6/arch/x86/include/asm/smp.h:13:0,
                 from include/linux/smp.h:38,
                 from include/linux/sched.h:30,
                 from fs/gfs2/sys.c:10:
/home/me/tmp/linux-3.7.6/arch/x86/include/asm/apic.h:394:1: internal compiler error: Segmentation fault
Please submit a full bug report,
with preprocessed source if appropriate.
See  for instructions.
The bug is not reproducible, so it is likely a hardware or OS problem.
make[3]: *** [fs/gfs2/sys.o] Error 1
make[2]: *** [fs/gfs2] Error 2
make[1]: *** [fs] Error 2
make[1]: Leaving directory `/home/me/tmp/linux-3.7.6'
make: *** [debian/stamp/build/kernel] Error 2
Kernel 3.5
CC [M] drivers/scsi/lpfc/lpfc_els.o CC [M] drivers/scsi/lpfc/lpfc_hbadisc.o CC [M] drivers/scsi/lpfc/lpfc_init.o In file included from /home/me/tmp/linux-3.5/arch/x86/include/asm/msr.h:139:0, from /home/me/tmp/linux-3.5/arch/x86/include/asm/processor.h:20, from /home/me/tmp/linux-3.5/arch/x86/include/asm/thread_info.h:22, from include/linux/thread_info.h:54, from include/linux/preempt.h:9, from include/linux/spinlock.h:50, from include/linux/seqlock.h:29, from include/linux/time.h:8, from include/linux/timex.h:56, from include/linux/sched.h:57, from include/linux/blkdev.h:4, from drivers/scsi/lpfc/lpfc_init.c:22: /home/me/tmp/linux-3.5/arch/x86/include/asm/paravirt.h: In function 'store_gdt': /home/me/tmp/linux-3.5/arch/x86/include/asm/paravirt.h:304:2: internal compiler error: Segmentation fault Please submit a full bug report, with preprocessed source if appropriate. See for instructions. The bug is not reproducible, so it is likely a hardware or OS problem. make[4]: *** [drivers/scsi/lpfc/lpfc_init.o] Error 1 make[3]: *** [drivers/scsi/lpfc] Error 2 make[2]: *** [drivers/scsi] Error 2 make[1]: *** [drivers] Error 2 make[1]: Leaving directory `/home/me/tmp/linux-3.5' make: *** [debian/stamp/build/kernel] Error 2

Kernel 3.4.42
Second crash:
  CC [M]  fs/coda/psdev.o
  CC [M]  fs/coda/cache.o
In file included from include/linux/mm.h:256:0,
                 from fs/coda/coda_linux.h:17,
                 from fs/coda/cache.c:24:
include/linux/page-flags.h:232:1: internal compiler error: Segmentation fault
Please submit a full bug report,
with preprocessed source if appropriate.
See  for instructions.
The bug is not reproducible, so it is likely a hardware or OS problem.
make[3]: *** [fs/coda/cache.o] Error 1
make[2]: *** [fs/coda] Error 2
make[1]: *** [fs] Error 2
make[1]: Leaving directory `/home/me/tmp/linux-3.4.42'
make: *** [debian/stamp/build/kernel] Error 2
First crash:
  CC      kernel/signal.o
gcc: internal compiler error: Segmentation fault (program as)
Please submit a full bug report,
with preprocessed source if appropriate.
See  for instructions.
make[2]: *** [kernel/signal.o] Error 4
make[1]: *** [kernel] Error 2
make[1]: Leaving directory `/home/me/tmp/linux-3.4.42'
make: *** [debian/stamp/build/kernel] Error 2


Kernels that won't build and the errors -- before bios flash:
3.9-rc8
CC [M] fs/nfs/nfs4client.o CC [M] fs/nfs/nfs4sysctl.o CC [M] fs/nfs/nfs4session.o CC [M] fs/nfs/pnfs.o fs/nfs/pnfs.c: In function 'read_seqcount_retry': fs/nfs/pnfs.c:1951:1: internal compiler error: Segmentation fault Please submit a full bug report, with preprocessed source if appropriate. See for instructions. The bug is not reproducible, so it is likely a hardware or OS problem. make[3]: *** [fs/nfs/pnfs.o] Error 1 make[2]: *** [fs/nfs] Error 2 make[1]: *** [fs] Error 2 make[1]: Leaving directory `/home/me/tmp/linux-3.9-rc8' make: *** [debian/stamp/build/kernel] Error 2
3.8.10
  CC [M]  fs/nfs/inode.o
In file included from include/net/scm.h:6:0,
                 from include/linux/netlink.h:8,
                 from /home/me/tmp/linux-3.8.10/include/uapi/linux/neighbour.h:5,
                 from include/linux/netdevice.h:51,
                 from include/linux/icmpv6.h:12,
                 from include/linux/ipv6.h:59,
                 from include/net/ipv6.h:16,
                 from include/linux/sunrpc/clnt.h:26,
                 from fs/nfs/inode.c:26:
include/linux/security.h:2581:1: internal compiler error: Segmentation fault
Please submit a full bug report,
3.8.6
CC [M] drivers/hid/hid-lg.o CC [M] drivers/hid/hid-lgff.o CC [M] drivers/hid/hid-lg2ff.o CC [M] drivers/hid/hid-lg3ff.o CC [M] drivers/hid/hid-lg4ff.o CC [M] drivers/hid/hid-picolcd_core.o CC [M] drivers/hid/hid-picolcd_fb.o CC [M] drivers/hid/hid-picolcd_backlight.o drivers/hid/hid-picolcd_backlight.c:120:1: internal compiler error: Segmentation fault Please submit a full bug report, with preprocessed source if appropriate. See for instructions. The bug is not reproducible, so it is likely a hardware or OS problem. make[3]: *** [drivers/hid/hid-picolcd_backlight.o] Error 1 make[2]: *** [drivers/hid] Error 2 make[1]: *** [drivers] Error 2 make[1]: Leaving directory `/home/me/tmp/linux-3.8.6' make: *** [debian/stamp/build/kernel] Error 2
3.8
CC mm/dmapool.o CC mm/hugetlb.o /bin/sh: line 1: 25153 Done(2) gcc -E -D__GENKSYMS__ -Wp,-MD,mm/.hugetlb.o.d -nostdinc -isystem /usr/lib/gcc/x86_64-linux-gnu/4.7/include -I/home/me/tmp/linux-3.8/arch/x86/include -Iarch/x86/include/generated -Iinclude -I/home/me/tmp/linux-3.8/arch/x86/include/uapi -Iarch/x86/include/generated/uapi -I/home/me/tmp/linux-3.8/include/uapi -Iinclude/generated/uapi -include /home/me/tmp/linux-3.8/include/linux/kconfig.h -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -Os -m64 -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -maccumulate-outgoing-args -fstack-protector -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1 -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -Wframe-larger-than=2048 -Wno-unused-but-set-variable -fomit-frame-pointer -g -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -DCC_HAVE_ASM_GOTO -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(hugetlb)" -D"KBUILD_MODNAME=KBUILD_STR(hugetlb)" mm/hugetlb.c 25154 Segmentation fault | scripts/genksyms/genksyms -a x86_64 -r /dev/null > mm/.tmp_hugetlb.ver make[2]: *** [mm/hugetlb.o] Error 139 make[1]: *** [mm] Error 2 make[1]: Leaving directory `/home/me/tmp/linux-3.8' make: *** [debian/stamp/build/kernel] Error 2
3.7.6

The errors differ each time:

Second run:
  CC [M]  fs/ext2/namei.o
  CC [M]  fs/ext2/super.o
fs/ext2/super.c: In function 'ext2_fill_super':
fs/ext2/super.c:762:12: internal compiler error: Segmentation fault
Please submit a full bug report,
with preprocessed source if appropriate.
See  for instructions.
The bug is not reproducible, so it is likely a hardware or OS problem.
make[3]: *** [fs/ext2/super.o] Error 1
make[2]: *** [fs/ext2] Error 2
make[1]: *** [fs] Error 2
make[1]: Leaving directory `/home/me/tmp/linux-3.7.6'
make: *** [debian/stamp/build/kernel] Error 2
First run:
  CC      drivers/base/power/main.o
/bin/sh: line 1: 12317 Done                    gcc -E -D__GENKSYMS__ -Wp,-MD,drivers/base/power/.main.o.d -nostdinc -isystem /usr/lib/gcc/x86_64-linux-gnu/4.7/include -I/home/me/tmp/linux-3.7.6/arch/x86/include -Iarch/x86/include/generated -Iinclude -I/home/me/tmp/linux-3.7.6/arch/x86/include/uapi -Iarch/x86/include/generated/uapi -I/home/me/tmp/linux-3.7.6/include/uapi -Iinclude/generated/uapi -include /home/me/tmp/linux-3.7.6/include/linux/kconfig.h -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -Os -m64 -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -maccumulate-outgoing-args -fstack-protector -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_AVX=1 -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -Wframe-larger-than=2048 -Wno-unused-but-set-variable -fomit-frame-pointer -g -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -DCC_HAVE_ASM_GOTO -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(main)" -D"KBUILD_MODNAME=KBUILD_STR(main)" drivers/base/power/main.c
     12318 Segmentation fault      | scripts/genksyms/genksyms -a x86_64 -r /dev/null > drivers/base/power/.tmp_main.ver
make[4]: *** [drivers/base/power/main.o] Error 139
make[3]: *** [drivers/base/power] Error 2
make[2]: *** [drivers/base] Error 2
make[1]: *** [drivers] Error 2
3.7.2
The errors differ every time:

Second run:
CC [M] drivers/hwmon/tmp102.o ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x76d76)[0x2b2bb07f6d76] /lib/x86_64-linux-gnu/libc.so.6(+0x7a658)[0x2b2bb07fa658] /lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x70)[0x2b2bb07fbb90] scripts/genksyms/genksyms[0x4075fa] scripts/genksyms/genksyms[0x4037c0] scripts/genksyms/genksyms[0x402de6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd)[0x2b2bb079eead] scripts/genksyms/genksyms[0x400f59] ======= Memory map: ======== 00400000-0040e000 r-xp 00000000 08:05 36440720 /home/me/tmp/linux-3.7.2/scripts/genksyms/genksyms 0060d000-0060e000 rw-p 0000d000 08:05 36440720 /home/me/tmp/linux-3.7.2/scripts/genksyms/genksyms 0060e000-00616000 rw-p 00000000 00:00 0 0089b000-00c58000 rw-p 00000000 00:00 0 [heap] 2b2bb0330000-2b2bb0350000 r-xp 00000000 08:01 11802457 /lib/x86_64-linux-gnu/ld-2.13.so 2b2bb0350000-2b2bb0352000 rw-p 00000000 00:00 0 2b2bb054f000-2b2bb0550000 r--p 0001f000 08:01 11802457 /lib/x86_64-linux-gnu/ld-2.13.so 2b2bb0550000-2b2bb0551000 rw-p 00020000 08:01 11802457 /lib/x86_64-linux-gnu/ld-2.13.so 2b2bb0551000-2b2bb0552000 rw-p 00000000 00:00 0 2b2bb0558000-2b2bb0561000 r-xp 00000000 08:01 2233201 /usr/lib/x86_64-linux-gnu/libfakeroot/libfakeroot-sysv.so 2b2bb0561000-2b2bb0761000 ---p 00009000 08:01 2233201 /usr/lib/x86_64-linux-gnu/libfakeroot/libfakeroot-sysv.so 2b2bb0761000-2b2bb0762000 rw-p 00009000 08:01 2233201 /usr/lib/x86_64-linux-gnu/libfakeroot/libfakeroot-sysv.so 2b2bb0762000-2b2bb0763000 rw-p 00000000 00:00 0 2b2bb077c000-2b2bb077d000 rw-p 00000000 00:00 0 2b2bb0780000-2b2bb0900000 r-xp 00000000 08:01 11802454 /lib/x86_64-linux-gnu/libc-2.13.so 2b2bb0900000-2b2bb0b00000 ---p 00180000 08:01 11802454 /lib/x86_64-linux-gnu/libc-2.13.so 2b2bb0b00000-2b2bb0b04000 r--p 00180000 08:01 11802454 /lib/x86_64-linux-gnu/libc-2.13.so 2b2bb0b04000-2b2bb0b05000 rw-p 00184000 08:01 11802454 /lib/x86_64-linux-gnu/libc-2.13.so 2b2bb0b05000-2b2bb0b0a000 rw-p 00000000 00:00 0 2b2bb0b10000-2b2bb0b12000 r-xp 00000000 08:01 11802447 /lib/x86_64-linux-gnu/libdl-2.13.so 2b2bb0b12000-2b2bb0d12000 ---p 00002000 08:01 11802447 /lib/x86_64-linux-gnu/libdl-2.13.so 2b2bb0d12000-2b2bb0d13000 r--p 00002000 08:01 11802447 /lib/x86_64-linux-gnu/libdl-2.13.so 2b2bb0d13000-2b2bb0d14000 rw-p 00003000 08:01 11802447 /lib/x86_64-linux-gnu/libdl-2.13.so 2b2bb0d14000-2b2bb0d16000 rw-p 00000000 00:00 0 2b2bb0d30000-2b2bb0d45000 r-xp 00000000 08:01 11796731 /lib/x86_64-linux-gnu/libgcc_s.so.1 2b2bb0d45000-2b2bb0f45000 ---p 00015000 08:01 11796731 /lib/x86_64-linux-gnu/libgcc_s.so.1 2b2bb0f45000-2b2bb0f46000 rw-p 00015000 08:01 11796731 /lib/x86_64-linux-gnu/libgcc_s.so.1 2b2bb4000000-2b2bb4021000 rw-p 00000000 00:00 0 2b2bb4021000-2b2bb8000000 ---p 00000000 00:00 0 7fff5cd00000-7fff5cd23000 rw-p 00000000 00:00 0 [stack] 7fff5cdd8000-7fff5cdd9000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] /bin/sh: line 1: 18863 Done(2) gcc -E -D__GENKSYMS__ -Wp,-MD,drivers/acpi/.video.o.d -nostdinc -isystem /usr/lib/gcc/x86_64-linux-gnu/4.7/include -I/home/me/tmp/linux-3.7.2/arch/x86/include -Iarch/x86/include/generated -Iinclude -I/home/me/tmp/linux-3.7.2/arch/x86/include/uapi -Iarch/x86/include/generated/uapi -I/home/me/tmp/linux-3.7.2/include/uapi -Iinclude/generated/uapi -include /home/me/tmp/linux-3.7.2/include/linux/kconfig.h -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -Os -m64 -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -maccumulate-outgoing-args -fstack-protector -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_AVX=1 -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -Wframe-larger-than=2048 -Wno-unused-but-set-variable -fomit-frame-pointer -g -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -DCC_HAVE_ASM_GOTO -Os -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(video)" -D"KBUILD_MODNAME=KBUILD_STR(video)" drivers/acpi/video.c 18864 Aborted | scripts/genksyms/genksyms -a x86_64 -r /dev/null > drivers/acpi/.tmp_video.ver make[3]: *** [drivers/acpi/video.o] Error 134 make[2]: *** [drivers/acpi] Error 2 make[1]: *** [drivers] Error 2 make[1]: Leaving directory `/home/me/tmp/linux-3.7.2' make: *** [debian/stamp/build/kernel] Error 2

First run:
  CHK     include/generated/uapi/linux/version.h
  CHK     include/generated/utsrelease.h
  CALL    scripts/checksyscalls.sh
  Building modules, stage 2.
  MODPOST 2369 modules
ERROR: "ieee80211_get_hdrlen" [drivers/staging/rtl8192u/r8192u_usb.ko] undefined!
ERROR: "ieee80211_is_empty_essid" [drivers/staging/rtl8192u/r8192u_usb.ko] undefined!
make[2]: *** [__modpost] Error 1
make[1]: *** [modules] Error 2
make[1]: Leaving directory `/home/me/tmp/linux-3.7.2'
make: *** [debian/stamp/build/kernel] Error 2
3.6.3
LD [M] drivers/input/misc/pcf50633-input.ko CC drivers/input/misc/pcspkr.mod.o In file included from drivers/input/misc/pcspkr.mod.c:1:0: include/linux/module.h:299:9: internal compiler error: Segmentation fault Please submit a full bug report, with preprocessed source if appropriate. See for instructions. The bug is not reproducible, so it is likely a hardware or OS problem. make[2]: *** [drivers/input/misc/pcspkr.mod.o] Error 1 make[1]: *** [modules] Error 2 make[1]: Leaving directory `/home/me/tmp/linux-3.6.3' make: *** [debian/stamp/build/kernel] Error 2
3.5.0
The errors keep changing.

Second run:
  CC [M]  drivers/gpu/drm/via/via_map.o
  CC [M]  drivers/gpu/drm/via/via_mm.o
  CC [M]  drivers/gpu/drm/via/via_dma.o
drivers/gpu/drm/via/via_dma.c:741:21: internal compiler error: Segmentation fault
Please submit a full bug report,
with preprocessed source if appropriate.
See  for instructions.
The bug is not reproducible, so it is likely a hardware or OS problem.
make[5]: *** [drivers/gpu/drm/via/via_dma.o] Error 1
make[4]: *** [drivers/gpu/drm/via] Error 2
make[3]: *** [drivers/gpu/drm] Error 2
make[2]: *** [drivers/gpu] Error 2
make[1]: *** [drivers] Error 2
make[1]: Leaving directory `/home/me/tmp/linux-3.5'
make: *** [debian/stamp/build/kernel] Error 2
First run:
  CC      drivers/hid/hid-sony.mod.o
drivers/hid/hid-sony.mod.c:46:1: internal compiler error: Segmentation fault
Please submit a full bug report,
with preprocessed source if appropriate.
See  for instructions.
The bug is not reproducible, so it is likely a hardware or OS problem.
make[2]: *** [drivers/hid/hid-sony.mod.o] Error 1
make[1]: *** [modules] Error 2
make[1]: Leaving directory `/home/me/tmp/linux-3.5'
make: *** [debian/stamp/build/kernel] Error 2
3.4.42
CC [M] fs/quota/quota_tree.o CC [M] fs/reiserfs/bitmap.o fs/reiserfs/bitmap.c: In function 'scan_bitmap_block.constprop.9': fs/reiserfs/bitmap.c:236:9: warning: 'next' may be used uninitialized in this function [-Wmaybe-uninitialized] CC [M] fs/reiserfs/do_balan.o CC [M] fs/reiserfs/namei.o gcc: internal compiler error: Segmentation fault (program as) Please submit a full bug report, with preprocessed source if appropriate. See for instructions. make[3]: *** [fs/reiserfs/namei.o] Error 4 make[2]: *** [fs/reiserfs] Error 2 make[1]: *** [fs] Error 2 make[1]: Leaving directory `/home/me/tmp/linux-3.4.42' make: *** [debian/stamp/build/kernel] Error 2


Another dmesg error:

24 March 2013

369. Compiling ECCE 6.4 on Fedora 18

UPDATE 16 April 2013:
The getops.pl issues has now been fixed in the latest release of ECCE
(http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id756/Problem_with_httpd.html)

Original post
(there were initially some issues with getops.pl being deprecated. Fixed now)

This post on the NWChem/ECCE forum got me interested in trying to build ECCE on Fedora 18.

I'm presuming that you just clicked your way through the fedora 18 installation so that e.g.  java-1.7.0-openjdk is already installed.  Note that the default partitioning scheme in Fedora may lead to a very small (<1 Gb) /tmp partition, which may prevent you from building properly, so be aware.

Building form source will solve at the very least :
* the issues of there being no 32 bit binary (build your own)
* issues with ubuntu libraries (libXfixes) if you try to install an older 32 bit binary

You'll also have access to the source code which means that you could -- at least in theory -- contribute to the project by doing bug fixes.

Building is easy once you've got all dependencies figured out. The initial run of the build script misses the detection of two required sets of packages: openGL libs, and perl-Digest-MD5

Start here

1. Download
Go to http://ecce.emsl.pnl.gov/using/download.shtml, fill out the form, and download the source code on the next page (ecce-v6.4-src.tar.gz). Create the folder ~/tmp (mkdir ~/tmp) and put ecce-v6.4-src.tar.bz2 in ~/tmp

2. Prepare
tar xvf ecce-v6.4-src.tar.bz2
cd ecce-v6.4/
export ECCE_HOME=`pwd`
cd build/
sudo yum install vim csh gcc gcc-c++ gcc-gfortran java-1.7.0-openjdk-devel python-devel ant gtk2-devel libjpeg-turbo-devel libtool ImageMagick libXt-devel xterm mesa-libGLU-devel kernel-devel perl-Digest-Perl-MD5 perl-Digest-MD5

(if you're doing this in virtualbox, the way to get openGL libs is via the virtualbox guest additions, and in order to build those you'll need the kernel headers (kernel-devel package) for the currently running kernel)

3. Build
pwd
/home/verahill/tmp/ecce-v6.4/build
./build_ecce

The script will see if all the requirements are met by stating what version is needed and echoing the currently installed version of different dependencies. Presumably all of the requirements will be met after installing the dependencies in step 2. At the very end you'll be asked

Do you want to skip these checks for future build_ecce invocations (y/n)? Y

Answer Y. The run build_ecce again and again...

./build_ecce
Xerces built
./build_ecce
Mesa OpenGL built
./build_ecce
wxWidgets built
This step will fail if you don't have openGL libs installed.

./build_ecce
running build_ext wxPython built
./build_ecce
Apache HTTP server built
./build_ecce
Making combined tar file ecce.v6.4.tar Copying NWChem binary distribution nwchem-6.1.1-binary-rhel5-gcc4.1.2-m64.tar.bz2 Copying NWChem common distribution nwchem-6.1.1-binary-common.tar.bz2 Concatenating install script and combined tar file ecce.v6.4.tar create_ecce_bin finished ECCE built and distribution created in /home/verahill/tmp/ecce-v6.4
This step will fail with
make[2]: Leaving directory `/home/verahill/tmp/ecce-v6.4/src/apps/vizthumbnail'

make[1]: Leaving directory `/home/verahill/tmp/ecce-v6.4/src/apps'

Can't locate Digest/MD5.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /home/verahill/tmp/ecce-v6.4/build/create_ecce_bin line 37.

BEGIN failed--compilation aborted at /home/verahill/tmp/ecce-v6.4/build/create_ecce_bin line 37.

if you don't have perl-Digest-MD5 installed.


4. Installation
You can now install ecce:
cd ..
./install_ecce.v6.4.csh

and follow the instructions. This post has an overview of the installation process, as do several of the posts here: http://verahill.blogspot.com.au/p/compilinginstalling-software-for.html

5. Patching
getops.pl has been deprecated in perl 5 -- see under incompatible changes on this page: http://fedoraproject.org/wiki/Features/perl5.16
Luckily, the fix is simple:

Assuming that you install ECCE in ~/ecce-v6.4
cd ~/ecce-v6.4/apps/scripts
sed -i 's/require "getopts.pl";/use Getopt::Std;/g' *
sed: couldn't edit codereg: not a regular file
set -i 's/Getopts/getops/g' * cd parsers/ sed -i 's/require "getopts.pl";/use Getopt::Std;/g' * set -i 's/Getopts/getops/g' *

While getopts.pl has been deprecated, Getopt::Std supplies both a getop() and a getopts() function. I've done a little bit of testing (vib calcs, energy calcs) and so far everything looks great.

If you're still having issues, make sure that Getopt is in /usr/share/perl5

6. Run
You can now have fun with ECCE

24 January 2013

326. Compiling Thunderbird 17.0.2 on Debian Testing

I tested this compile in a bare chroot, so the dependencies should be pretty much hammered out. The build is easy but fairly slow.

sudo apt-get install bzip2 build-essential python zip libgtk2.0-dev libdbus-glib-1-dev libasound2-dev libogg-dev libxt-dev yasm libcurl4-openssl-dev mesa-common-dev
mkdir ~/tmp
cd ~/tmp
wget ftp://ftp.mozilla.org/pub/mozilla.org/thunderbird/releases/17.0.2/source/thunderbird-17.0.2.source.tar.bz2
rm -rf comm-release/
tar xvf thunderbird-17.0.2.source.tar.bz2
mkdir bldthunder17
cd bldthunder17/
../comm-release/./configure --disable-necko-wifi
make
sudo make install

That took 122 minutes on a single core.
Checkinstall doesn't work and ends with segfault.


23 January 2013

325. Compiling ECCE 6.4 on Debian Testing

!NOTE! If you provide ECCE with 'localhost' as the hostname, be aware that this will block outside access: http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id858/#post_3178

There's a new release of ECCE 6.4 out which fixes the following bug: http://verahill.blogspot.com.au/2012/11/minor-bug-in-ecce.html


A note on Java: I've only had luck with the openjdk packages -- not with the Oracle/sun java ones (see here: http://verahill.blogspot.com.au/2012/06/building-ecce-on-debian-testingwheezy.html).

This build was tested in a chrooted Testing/Wheezy i.e. I should've caught most of the necessary packages.


Compiling:
Download the source to ecce from http://ecce.emsl.pnl.gov/using/download.shtml, and put it in e.g. ~/tmp

sudo apt-get install bzip2 build-essential autoconf libtool ant pkg-config gtk+-2.0-dev libxt-dev csh gfortran openjdk-7-jdk python-dev libjpeg-dev imagemagick xterm
cd ~/tmp
tar xvf ecce-v6.4-src.tar.bz2
cd ecce-v6.4/
export ECCE_HOME=`pwd`
cd build/
./build_ecce
Hit return if xterm was found... The /home/sandbox/tmp/ecce-v6.4/scripts/sysdir script identifies the build platform directory as: empty Because this value is empty no platform-specific parent directory will be created for ECCE executables, libraries, etc. This works fine unless your site needs support for multiple platforms. Finished checking prerequisites for building ECCE. Do you want to skip these checks for future build_ecce invocations (y/n)? Y
./build_ecce
Xerces built
./build_ecce
Mesa OpenGL built
./build_ecce
wxWidgets built
./build_ecce
running build_ext wxPython built
./build_ecce
Apache HTTP server built
./build_ecce
Making combined tar file ecce.v6.4.tar Copying NWChem binary distribution nwchem-6.1.1-binary-rhel5-gcc4.1.2-m64.tar.bz2 Copying NWChem common distribution nwchem-6.1.1-binary-common.tar.bz2 Concatenating install script and combined tar file ecce.v6.4.tar create_ecce_bin finished ECCE built and distribution created in /home/sandbox/tmp/ecce-v6.4
cd ../ ./install_ecce.v6.4.csh

And see the next section for the installation steps.
The compilation steps take progressively longer and longer, so be patient during the build.

Installing:
Digital ink is cheap, so I'll show the whole process:
./install_ecce.v6.4.csh
Main ECCE installation menu =========================== 1) Help on main menu options 2) Prerequisite software check 3) Full install 4) Full upgrade 5) Application software install 6) Application software upgrade 7) Server install 8) Server upgrade IMPORTANT: If you are uncertain about any aspect of installing or running ECCE at your site, please refer to the detailed ECCE Installation and Administration Guide at http://ecce.pnl.gov/docs/installation/2864B-Installation.pdf Hit at prompts to accept the default value in brackets. Selection: [1] 3 Host name: [beryllium] localhost Application installation directory: [/home/sandbox/tmp/ecce-v6.4/ecce-v6.4/apps] /home/sandbox/.ecce/apps Server installation directory: [/home/sandbox/.ecce/server] ECCE v6.4 will be installed using the settings: Installation type: [full install] Host name: [localhost] Application installation directory: [/home/sandbox/.ecce/apps] Server installation directory: [/home/sandbox/.ecce/server] Are these choices correct (yes/no/quit)? [yes] Installing ECCE application software in /home/sandbox/.ecce/apps... Extracting application distribution... Extracting NWChem binary distribution... Extracting NWChem common distribution... Extracting client WebHelp distribution... Configuring application software... Configuring NWChem... Installing ECCE server in /home/sandbox/.ecce/server... Extracting data server in /home/sandbox/.ecce/server/httpd... Extracting data libraries in /home/sandbox/.ecce/server/data... Extracting Java Messaging Server in /home/sandbox/.ecce/server/activemq... Configuring ECCE server... ECCE installation succeeded. *************************************************************** !! You MUST perform the following steps in order to use ECCE !! -- Unless only the user 'sandbox' will be running ECCE, start the ECCE server as 'sandbox' with: /home/sandbox/.ecce/server/ecce-admin/start_ecce_server -- To register machines to run computational codes, please see the installation and compute resource registration manuals at http://ecce.pnl.gov/using/installguide.shtml -- Before running ECCE each user must source an environment setup script. For csh/tcsh users add this to ~/.cshrc: if ( -e /home/sandbox/.ecce/apps/scripts/runtime_setup ) then source /home/sandbox/.ecce/apps/scripts/runtime_setup endif For sh/bash users, add this to ~/.profile or ~/.bashrc: if [ -e /home/sandbox/.ecce/apps/scripts/runtime_setup.sh ]; then . /home/sandbox/.ecce/apps/scripts/runtime_setup.sh fi ***************************************************************

Instead of following the instructions above I normally do:
echo 'export ECCE_HOME=/home/sandbox/.ecce/apps' >> ~/.bashrc
echo 'PATH=$PATH:/home/sandbox/.ecce/server/ecce-admin/:/home/sandbox/.ecce/apps/scripts/' >> ~/.bashrc
echo 
source ~/.bashrc

You can now start ecce by either doing
ecce

or if that complains, do
start_ecce_server

then waiting a little while (10 s), followed by
ecce


Apppendix:

Selecting Java version
sudo update-alternatives --config java
There are 7 choices for the alternative java (providing /usr/bin/java). Selection Path Priority Status ------------------------------------------------------------ 0 /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java 1061 auto mode 1 /usr/bin/gij-4.4 1044 manual mode 2 /usr/bin/gij-4.6 1046 manual mode 3 /usr/bin/gij-4.7 1047 manual mode 4 /usr/lib/jvm/j2re1.6-oracle/bin/java 314 manual mode 5 /usr/lib/jvm/j2sdk1.6-oracle/jre/bin/java 315 manual mode 6 /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java 1061 manual mode * 7 /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java 1051 manual mode

I've got ECCE running fine with openjdk 7 as well as openjdk 6.

20 January 2013

323. Wine 1.5.22 on Debian Testing/Wheezy. Compiling using multiarch

UPDATE 16 May 2013: See here for Wine 1.5.30: http://verahill.blogspot.com.au/2013/05/416-wine-1530-in-chroot.html

Another update: It seems like your mileage in terms of how well Wine will work for you will vary using this method. It works fine for me and I suspect it's because I've built earlier Wine versions of my systems and have ia32-libs-dev installed. This package is no longer available in Testing. ia32-libs installs the libs you need, but does not provide symlinks between .so and .so.X files -- you can probably do that by hand if necessary though. It's not for beginners.

An additional thing to remember is that the 1.5 series of Wine is a development series -- 1.4 is the stable series which is meant for end users. Progress made in 1.5 will be included in 1.6.

NOTE: there are a couple of issues, and they will depend on how your system is set up.
* libosmesa6-dev:i386 will require mesa-common-dev:i386 which will remove libosmesa6-dev:amd64 and more, which is bad.
* libglu1-mesa-dev:i386 depends on libgl1-mesa-dev:i386 which will remove libgl1-mesa-dev:amd64 and more, which is bad.
*  libgstreamer-plugins-base0.10-dev:i386 depends on libgstreamer0.10-dev:i386 which depends on gir1.2-gstreamer-0.10:i386 which depends on gir1.2-freedesktop:i386 which depends on gir1.2-glib-2.0:i386 which depends on libgirepository-1.0-1:i386 which will remove a whole lot of packages (132 on one of my systems, including gnome, gdm3 etc.)

I'm working on figuring out what's triggering this on some systems but not others.
In the mean time see http://verahill.blogspot.com.au/2013/01/308-compiling-wine-1521-on-debian.html to see how to build wine in a chroot, which is safe. You can then install that .deb package on your normal system and HOPEFULLY there won't be any broken dependencies. You won't need the -dev:i386 packages on the install target.

ldd `which wine` 
linux-gate.so.1 => (0x55573000) libwine.so.1 => /usr/local/lib/libwine.so.1 (0x55576000) libpthread.so.0 => /lib32/libpthread.so.0 (0x556ec000) libc.so.6 => /lib32/libc.so.6 (0x55705000) libdl.so.2 => /lib32/libdl.so.2 (0x55867000)
so you really don't seem to need much in the way of shared libs installed.

A new incremental version of wine is out.

There's little difference between building 1.5.21 and 1.5.22 but here it is anyway. The build is very easy on an up-to-date installation of Testing/Wheezy. If you are on stable, the list over needed packages can be found in this post.

See here for recent changes between different Wine versions: http://linux.softpedia.com/progChangelog/Wine-Changelog-148.html

Compiling
sudo dpkg --add-architecture i386
sudo apt-get update

sudo apt-get install libx11-dev:i386 libfreetype6-dev:i386 libxcursor-dev:i386 libxi-dev:i386 libxxf86vm-dev:i386 libxrandr-dev:i386 libxinerama-dev:i386 libxcomposite-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libdbus-1-dev:i386 libgnutls-dev:i386 libncurses-dev:i386 libsane-dev:i386 libv4l-dev:i386 libgphoto2-2-dev:i386 liblcms-dev:i386 libgstreamer-plugins-base0.10-dev:i386 libcapi20-dev:i386 libcups2-dev:i386 libfontconfig-dev:i386 libgsm1-dev:i386 libtiff-dev:i386 libpng-dev:i386 libjpeg-dev:i386 libmpg123-dev:i386 libopenal-dev:i386 libldap-dev:i386 libxrender-dev:i386 libxml2-dev:i386 libxslt-dev:i386 libhal-dev:i386 libcurl4-openssl-dev:i386 gettext prelink bzip2 bison flex oss4-dev checkinstall ocl-icd-libopencl1:i386 opencl-headers gcc-multilib

mkdir ~/tmp
cd ~/tmp
wget http://prdownloads.sourceforge.net/wine/wine-1.5.22.tar.bz2
tar xvf wine-1.5.22.tar.bz2
cd wine-1.5.22/
./configure
config.status: executing include/wine commands config.status: executing Makefile commands configure: Finished. Do 'make' to compile Wine.
time make -j4 sudo checkinstall --install=no
***************************************** **** Debian package creation selected *** ***************************************** This package will be built according to these values: 0 - Maintainer: [ root@lithium ] 1 - Summary: [ wine 1.5.22 ] 2 - Name: [ wine ] 3 - Version: [ 1.5.22 ] 4 - Release: [ 1 ] 5 - License: [ GPL ] 6 - Group: [ checkinstall ] 7 - Architecture: [ amd64 ] 8 - Source location: [ wine-1.5.22 ] 9 - Alternate source location: [ ] 10 - Requires: [ ] 11 - Provides: [ wine ] 12 - Conflicts: [ ] 13 - Replaces: [ ] [..] Copying files to the temporary directory...
sudo dpkg -i wine_1.5.22-1_amd64.deb

where 4 is the number of cores on your build machine (see here and here for -jN). The build took 13 minutes on a three-core AMD Athlon II X3 445 (3.1 GHz) with -j4. The 'Copying files to...' can take quite a while, so let it run to completion.

Anyway, that's it. An easy build.