370. Compiling ECCE 6.4 on Scientific Linux 6.3 (Carbon; Red Hat; CentOS)

!NOTE! If you provide ECCE with 'localhost' as the hostname, be aware that this will block outside access: http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id858/#post_3178

This build is almost identical to that on Fedora 18. I'm posting it as a separate post since I suspect a fair number of nwchem/ecce users are using Scientific Linux/CentOS/Red Hat. The main difference between fedora 18 and scientific linux 6.3 is that you don't have to worry about getops.pl errors on scientific linux. Yes, no manual patching, which probably reflects the more conservative approach in SL/RHEL than in fedora.


1. Get the files
Go to http://ecce.emsl.pnl.gov/using/download.shtml, fill out the form and download ecce-v6.4-src.tar.gz and put it in ~/tmp.

2. Install dependencies and build

sudo yum install vim csh gcc gcc-c++ gcc-gfortran java-1.7.0-openjdk-devel python-devel ant gtk2-devel libjpeg-turbo-devel libtool ImageMagick libXt-devel xterm mesa-libGLU-devel
cd ~/tmp
tar xvf ecce-v6.4-src.tar.bz2
cd ecce-v6.4/
export ECCE_HOME=`pwd`
cd build/
./build_ecce

Do you want to skip these checks for future build_ecce invocations (y/n)? 

Answer yes

./build_ecce

Xerces built
./build_ecce

Mesa OpenGL built
./build_ecce

wxWidgets built
./build_ecce

wxPython built
./build_ecce

Apache HTTP server built
./build_ecce

Copying JMS server distribution jms_server.tar.bz2
Making server tar file ecce_server.tar
Copying WebHelp tar file eccewebhelp.tar.bz2
Making combined tar file ecce.v6.4.tar
Copying NWChem binary distribution nwchem-6.1.1-binary-rhel5-gcc4.1.2-m32.tar.bz2
Copying NWChem common distribution nwchem-6.1.1-binary-common.tar.bz2
Concatenating install script and combined tar file ecce.v6.4.tar
create_ecce_bin finished


ECCE built and distribution created in /home/me/tmp/ecce-v6.4

3. Install

cp ~/tmp/ecce-v6.4/install_ecce.v6.4.csh ~/
cd ~
./install_ecce.v6.4.csh

Extracting ECCE distribution from ./install_ecce.v6.4.csh...
hostname: Unknown host

Main ECCE installation menu
===========================
1) Help on main menu options
2) Prerequisite software check
3) Full install
4) Full upgrade
5) Application software install
6) Application software upgrade
7) Server install
8) Server upgrade

IMPORTANT: If you are uncertain about any aspect of installing
or running ECCE at your site, please refer to the detailed
ECCE Installation and Administration Guide at 
http://ecce.pnl.gov/docs/installation/2864B-Installation.pdf

Hit  at prompts to accept the default value in brackets.

Selection: [1] 3

Host name: [localhost (could not successfully ping science!)] localhost
Application installation directory: [/home/me/ecce-v6.4/apps] 
Server installation directory: [/home/me/ecce-v6.4/server]

ECCE v6.4 will be installed using the settings:

  Installation type: [full install]
  Host name: [localhost]
  Application installation directory: [/home/me/ecce-v6.4/apps]
  Server installation directory: [/home/me/ecce-v6.4/server]

Are these choices correct (yes/no/quit)? [yes] yes

Put the following in your ~/.bashrc:

export ECCE_HOME=/home/me/ecce-v6.4/apps
export PATH=${ECCE_HOME}/scripts:${ECCE_HOME}/scripts/parsers:$PATH
alias stopecce='/home/me/ecce-v6.4/server/ecce-admin/stop_ecce_server'
alias startecce='/home/me/ecce-v6.4/server/ecce-admin/start_ecce_server'

and do

source ~/.bashrc

Make sure that you hostname is in your hosts file, e.g.

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 science!
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

where science! is my hostname (I regret using ! in the hostname, but oddly I haven't actually had issues with it yet. Still, don't do it.). If it's not, you'll get a ton of errors.

4. Run.
You can now start ecce by doing

startecce
ecce

I've tested this on a headless box (small intel Atom with 512 Mb RAM) and it worked fine. I tried running calcs locally as well, and everything worked.

369. Compiling ECCE 6.4 on Fedora 18

UPDATE 16 April 2013:
The getops.pl issues has now been fixed in the latest release of ECCE
(http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id756/Problem_with_httpd.html)

Original post
(there were initially some issues with getops.pl being deprecated. Fixed now)

This post on the NWChem/ECCE forum got me interested in trying to build ECCE on Fedora 18.

I'm presuming that you just clicked your way through the fedora 18 installation so that e.g.  java-1.7.0-openjdk is already installed.  Note that the default partitioning scheme in Fedora may lead to a very small (<1 Gb) /tmp partition, which may prevent you from building properly, so be aware.

Building form source will solve at the very least :
* the issues of there being no 32 bit binary (build your own)
* issues with ubuntu libraries (libXfixes) if you try to install an older 32 bit binary

You'll also have access to the source code which means that you could -- at least in theory -- contribute to the project by doing bug fixes.

Building is easy once you've got all dependencies figured out. The initial run of the build script misses the detection of two required sets of packages: openGL libs, and perl-Digest-MD5

Start here

1. Download
Go to http://ecce.emsl.pnl.gov/using/download.shtml, fill out the form, and download the source code on the next page (ecce-v6.4-src.tar.gz). Create the folder ~/tmp (mkdir ~/tmp) and put ecce-v6.4-src.tar.bz2 in ~/tmp

2. Prepare

tar xvf ecce-v6.4-src.tar.bz2
cd ecce-v6.4/
export ECCE_HOME=`pwd`
cd build/
sudo yum install vim csh gcc gcc-c++ gcc-gfortran java-1.7.0-openjdk-devel python-devel ant gtk2-devel libjpeg-turbo-devel libtool ImageMagick libXt-devel xterm mesa-libGLU-devel kernel-devel perl-Digest-Perl-MD5 perl-Digest-MD5

(if you're doing this in virtualbox, the way to get openGL libs is via the virtualbox guest additions, and in order to build those you'll need the kernel headers (kernel-devel package) for the currently running kernel)

3. Build

pwd

/home/verahill/tmp/ecce-v6.4/build

./build_ecce

The script will see if all the requirements are met by stating what version is needed and echoing the currently installed version of different dependencies. Presumably all of the requirements will be met after installing the dependencies in step 2. At the very end you'll be asked

Do you want to skip these checks for future build_ecce invocations (y/n)? Y

Answer Y. The run build_ecce again and again...

./build_ecce

Xerces built
./build_ecce

Mesa OpenGL built
./build_ecce

wxWidgets built

This step will fail if you don't have openGL libs installed.

./build_ecce

running build_ext
wxPython built
./build_ecce

Apache HTTP server built
./build_ecce

Making combined tar file ecce.v6.4.tar
Copying NWChem binary distribution nwchem-6.1.1-binary-rhel5-gcc4.1.2-m64.tar.bz2
Copying NWChem common distribution nwchem-6.1.1-binary-common.tar.bz2
Concatenating install script and combined tar file ecce.v6.4.tar
create_ecce_bin finished

ECCE built and distribution created in /home/verahill/tmp/ecce-v6.4

This step will fail with

make[2]: Leaving directory `/home/verahill/tmp/ecce-v6.4/src/apps/vizthumbnail'

make[1]: Leaving directory `/home/verahill/tmp/ecce-v6.4/src/apps'

Can't locate Digest/MD5.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /home/verahill/tmp/ecce-v6.4/build/create_ecce_bin line 37.

BEGIN failed--compilation aborted at /home/verahill/tmp/ecce-v6.4/build/create_ecce_bin line 37.

if you don't have perl-Digest-MD5 installed.


4. Installation
You can now install ecce:

cd ..
./install_ecce.v6.4.csh

and follow the instructions. This post has an overview of the installation process, as do several of the posts here: http://verahill.blogspot.com.au/p/compilinginstalling-software-for.html

5. Patching
getops.pl has been deprecated in perl 5 -- see under incompatible changes on this page: http://fedoraproject.org/wiki/Features/perl5.16
Luckily, the fix is simple:

Assuming that you install ECCE in ~/ecce-v6.4

cd ~/ecce-v6.4/apps/scripts
sed -i 's/require "getopts.pl";/use Getopt::Std;/g' *

sed: couldn't edit codereg: not a regular file


set -i 's/Getopts/getops/g' *
cd parsers/
sed -i 's/require "getopts.pl";/use Getopt::Std;/g' *
set -i 's/Getopts/getops/g' *

While getopts.pl has been deprecated, Getopt::Std supplies both a getop() and a getopts() function. I've done a little bit of testing (vib calcs, energy calcs) and so far everything looks great.

If you're still having issues, make sure that Getopt is in /usr/share/perl5

6. Run
You can now have fun with ECCE

368. Slow mouse and keyboard triggered by kworker running custom 3.8.0 kernel on debian wheezy on laptop SL410

Another update (8 Apr. 2013):
A fix is on the way to kernel 3.9 or 3.10.
https://patchwork.kernel.org/patch/2400621/
https://patchwork.kernel.org/patch/2402211/

Update: More links:
http://verahill.blogspot.com.au/2013/04/375-bisecting-kernel-looking-for-commit.html
https://bbs.archlinux.org/viewtopic.php?pid=1248190
https://bbs.archlinux.org/viewtopic.php?pid=1254285#p1254285

Update: it might be an issue in kernel 3.8 -- http://forums.gentoo.org/viewtopic-p-7278760.html#7278760. Seems related to intel graphics which would explain why I only see it on my laptop. Finally, disabling polling probably had no effect -- it was going back to stock kernel 3.2 that solved. The issue is still present in kernel 3.8.5. Awaiting fix in 3.9 or 3.10?

I compiled 3.8.5 and it has the same issue. Powertop shows:

Usage       Events/s    Category       Description
21.9 ms/s     1921.2       Interrupt      [49] i915@pci:0000:00:02.0
 1.8 ms/s     111.9        kWork          od_dbs_timer

I've since also compiled 3.7.10 and haven't yet experienced any issues. Still testing though.

Original post

I've rediscovered an annoying problem that I must have solved at some point with my old debian installation on my laptop (lenovo SL410):  whenever kworker/u starts my mouse and keyboard become unresponsive. They work, but they lag horribly. The total cpu usage of kworker is only around 2-3%, and there's no excessive I/O, and it's consistent with the appearance of kworker in top.

When does kworker start? Whenever anything, anywhere happens e.g. if the fan goes on or if the fan goes off.

Unplugging mains power makes my USB keyboard and mouse behave strangely as well, although it's a bit more subtle. The fan off/mouse slowdown happens everytime though

I disabled kworker polling by editing /etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="quiet drm_kms_helper.poll=N"

(don't forget to run sudo update-grub after you've made your changes) but while it makes things run smoothly as long as there's no system state change, fan on/fan off still makes things slow down occasionally. Still, the slowdowns are a lot less frequent and random than before.

I also pulled in laptop-mode-tools, powertop and cpufrequtils. Not sure whether laptop-mode-tools could have a (positive) effect here, but from my reading it shouldn't kick in unless the laptop is running on battery. [update: DO NOT INSTALL laptop-mode-tools -- you'll have issues shutting down the laptop when it's on battery -- it'll reboot itself instead: http://verahill.blogspot.com.au/2013/04/379-thinkpad-sl410-reboot-instead-of.html]

The problem exists on both debian testing/wheezy (my own kernel 3.8) and arch linux (kernel 3.7.10-3.8).

After another reboot I have issues reproducing any sort of slowdown, and I don't seem to be able to reproduce it on the stock wheezy kernel (3.2). Maybe it's a kernel regression?

To anyone else with a similar issue: 
Most likely your best bet is to downgrade the kernel to pre 3.8. For me, kernel 3.2 works fine, but I haven't tested e.g. 3.7.

If you don't think the kernel is the issues you can edit /etc/default/grub as shown above and see how that goes.