Many, many years ago I learned basic programming using BASIC (the version that came with PC DOS 5, I think). I even wrote the odd game, but it was all pretty awful. A few years later I learned Turbo Pascal, which was a fantastic experience compared to Basic. It felt all sciency and grown up, and it was my first experience with a real IDE. I even ended up buying a TP book, and became somewhat proficient. This must've been when I was around 18-19.I then stopped programming completely.
At around 30 years of age I decided it was time to get serious about programming again -- I was doing mass spectrometry and needed a simple program that could generate a series of solutions to the identity of a mass/charge ratio given a range of elements. I probably had a quick look at C and C++, but ended up getting a Python book and have been happy Python programmer ever since.
The problem is that I've never been a /good/ python programmer -- and in all these years I've never fully understood the use for (or, in all fairness, use OF) OOP. And at the moment it seems to be holding me back -- all the examples that I find of the use the threading module as well as writing GUIs (using e.g. wxPython) involve using classes. And I just don't understand them well enough to sort out what I need done.
Anyway, long story short: I've written a basic program for communicating with a Metrohm Titrino 736 GP via RS 232. It's found here: https://sourceforge.net/projects/pytitrino/
Currently:
* the code is a mess (see above)
* it works fine for doing monotonic and dynamic end point titrations (MET and DET)
* it saves data to a file, but does so silently (i.e. when you run you won't get any feedback that things are working properly...)
* it uses the thread (not threading) module
* I've managed to pass parameters back and forth between the thread and the main loop using Queue
There are probably much better solutions. One day I hope to be able to stick a GUI on top of it, but the more I look at it I get the impression that one writes the GUI first, then the engine...not that I'd know.
Anyway. That's what I've been up to. Anyone with a bit of programming experience, whom is in possession of an old-school Titrino (i.e. using RS 232) and wants to save $1.5k in software licenses may be interested in taking the sources and turning them into something useful.
Lindqvist - a blog about Linux and Science. Mostly.
03 July 2013
472. Briefly: Iranian PhD students in Australia
I'm not going to leave much in the way of a comment, but this doesn't seem to have been publicised enough. Searching the web quickly didn't bring this up at all, and it's a shame since it's important, in particular if you are an Iranian national thinking about doing a PhD in Australia.
About a week ago the faculty in the chemistry department at my university were informed that heavy restrictions in terms of access to instrumentation has been put in place for students from North Korea, Syria and Iran via Federal legislation.
While I don't think there are any students from North Korea or Syria around, there are several Iranian students at different stages of their PhD. In fact, I would say around 50% of our applicants are from India, 25% are from Pakistan and 20 % are from Iran (in terms of accepted students the ratio is very different)
In practical terms, this means that Iranian students in the department are not allowed to use:
FT-IR
UV-Vis
NMR
Mass spectrometers
Raman
dosimeters
OES/AAS
etc.
All of which are standard instruments which most chemists would find necessary to do research. In addition, they can hardly be considered as being cutting edge, trade secrets or anything like that -- commercial NMR instruments have been around since the 1950s, infrared an UV/Visible spectroscopy go much further back. Mass spectrometry is a standard tool which, although many of the current designs only go back to the 1980s (e.g. ESI), is so conceptually simple and innocuous, that (to me) restrictions on it doesn't make sense. And so on.
In addition, supervisors of Iranian students have been asked to draw up a risk management plan to prevent student access to the above instruments, which is a particular problem given that they are used in teaching as well, and are available on a walk-in basis to undergraduate students doing projects in research labs.
Currently, any supervisor who has an Iranian student needing to use any of the instruments above will need to assign another student to do these measurements for the Iranian national.
While this doesn't formally preclude Iranians from coming to Australia to do a PhD, we have been advised that we should reject any applicants at this point. This may change once the university has figured out exactly where they must draw the line in terms of restricting access to Iranians to different facilities, but for now it's a blanket ban.
My personal opinion is that while you'd be led by the media to think of anyone from North Korea, Syria and Iran as potential spies, these are real people too. Many Iranians would either be completely disinterested in politics, or actively antipathetic to their regime. And the best thing about democracies -- we shouldn't have any issues with them supporting their government either. So I don't really agree with this as a security measure to prevent nuclear proliferation, which must surely be the stated goal.
And if the idea is to put in place sanctions to promote regime change, then why limit the type of instrumentation that students can access? Or are we trying to punish the children of the leadership in Iran? Then why not limit the sanctions to those specifically? Top students tend to come from all socioeconomic classes.
The timing is also very odd, given the recent election of a moderate.
And why Iran and not Belarus, China, Zimbabwe etc.?
Again, I don't like putting opinion pieces on this blog (other than as minor parts/rants of posts with actual content) but I think this should be publicized more.
About a week ago the faculty in the chemistry department at my university were informed that heavy restrictions in terms of access to instrumentation has been put in place for students from North Korea, Syria and Iran via Federal legislation.
While I don't think there are any students from North Korea or Syria around, there are several Iranian students at different stages of their PhD. In fact, I would say around 50% of our applicants are from India, 25% are from Pakistan and 20 % are from Iran (in terms of accepted students the ratio is very different)
In practical terms, this means that Iranian students in the department are not allowed to use:
FT-IR
UV-Vis
NMR
Mass spectrometers
Raman
dosimeters
OES/AAS
etc.
All of which are standard instruments which most chemists would find necessary to do research. In addition, they can hardly be considered as being cutting edge, trade secrets or anything like that -- commercial NMR instruments have been around since the 1950s, infrared an UV/Visible spectroscopy go much further back. Mass spectrometry is a standard tool which, although many of the current designs only go back to the 1980s (e.g. ESI), is so conceptually simple and innocuous, that (to me) restrictions on it doesn't make sense. And so on.
In addition, supervisors of Iranian students have been asked to draw up a risk management plan to prevent student access to the above instruments, which is a particular problem given that they are used in teaching as well, and are available on a walk-in basis to undergraduate students doing projects in research labs.
Currently, any supervisor who has an Iranian student needing to use any of the instruments above will need to assign another student to do these measurements for the Iranian national.
While this doesn't formally preclude Iranians from coming to Australia to do a PhD, we have been advised that we should reject any applicants at this point. This may change once the university has figured out exactly where they must draw the line in terms of restricting access to Iranians to different facilities, but for now it's a blanket ban.
My personal opinion is that while you'd be led by the media to think of anyone from North Korea, Syria and Iran as potential spies, these are real people too. Many Iranians would either be completely disinterested in politics, or actively antipathetic to their regime. And the best thing about democracies -- we shouldn't have any issues with them supporting their government either. So I don't really agree with this as a security measure to prevent nuclear proliferation, which must surely be the stated goal.
And if the idea is to put in place sanctions to promote regime change, then why limit the type of instrumentation that students can access? Or are we trying to punish the children of the leadership in Iran? Then why not limit the sanctions to those specifically? Top students tend to come from all socioeconomic classes.
The timing is also very odd, given the recent election of a moderate.
And why Iran and not Belarus, China, Zimbabwe etc.?
Again, I don't like putting opinion pieces on this blog (other than as minor parts/rants of posts with actual content) but I think this should be publicized more.
02 July 2013
471. Debian Jessie -- gnome-shell bug
Update 3/7/2013:
there are now *gnome-bluetooth packages (3.8.1-2) in the jessie repos now. While I haven't looked closer at them, I presume that they fix this issue.
(on a different note: dist-upgrade currently removes gnome...)
Original post:
I've used debian testing since early 2011, and I've only had a few minor issues during that time.
However, sometimes things happen that reminds you that the Testing release is not meant for mission critical work (and makes me happy that I only use Jessie on my laptop, which I mainly use at home).
So...
Last night I did upgrade and dist-upgrade, which installed the following packages according to /var/log/apt/history:
Now what happens when I log in to gnome via gdm3 I get an empty desktop with no menus, no hot-spots or anything else indicating that things worked out. Alt+F2 doesn't work either, and conky doesn't start.
The only thing that does work is
* my keyboard shortcuts (I've mapped ctrl+shift+Down arrow to chromium)
* guake (which starts with gnome)
Looking at the debian forums the closest post seems to be (although erroneously labelled -- gdm3 DOES start): http://forums.debian.net/viewtopic.php?f=6&t=105393&p=504077&hilit=gnome+shell#p504077
That in turn led to this bug report: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=712861
My gnome-shell version is 3.4.2-8,
I don't understand how gnome-bluetooth causes this, especially given that I've disabled bluetooth in rcconf, but whatever it takes...
I tried applying the patch but it failed:
Instead, I ended up making the changes to /usr/share/gnome-shell/js/ui/status/bluetooth.js by hand (remember that you can always use the ttys using ctrl+Fx):
and then delete the Applet part in GnomeBluetoothApplet so that it reads
Then do it again:
At this point I rebooted and everything was back to normal (you can try simply doing 'sudo service gdm3 restart' instead of rebooting).
Anyway, done.
there are now *gnome-bluetooth packages (3.8.1-2) in the jessie repos now. While I haven't looked closer at them, I presume that they fix this issue.
(on a different note: dist-upgrade currently removes gnome...)
Original post:
I've used debian testing since early 2011, and I've only had a few minor issues during that time.
However, sometimes things happen that reminds you that the Testing release is not meant for mission critical work (and makes me happy that I only use Jessie on my laptop, which I mainly use at home).
So...
Last night I did upgrade and dist-upgrade, which installed the following packages according to /var/log/apt/history:
Start-Date: 2013-07-01 22:03:17 Commandline: apt-get dist-upgrade Install: p11-kit:amd64 (0.18.3-2, automatic), libgnome-bluetooth11:amd64 (3.8.1-1, automatic), libgcr-base-3-1:amd64 (3.8.2-3, automatic), libtasn1-6:amd64 (3.3-1, automatic), libgcr-ui-3-1:amd64 (3.8.2-3, automatic) Upgrade: libnm-gtk0:amd64 (0.9.8.2-1, 0.9.8.2-1+b1), libgcr-3-1:amd64 (3.4.1-3, 3.8.2-3), gir1.2-gcr-3:amd64 (3.4.1-3, 3.8.2-3), network-manager-gnome:amd64 (0.9.8.2-1, 0.9.8.2-1+b1), gnome-keyring:amd64 (3.4.1-5, 3.8.2-2), gcr:amd64 (3.4.1-3, 3.8.2-3), gnome-bluetooth:amd64 (3.4.2-1, 3.8.1-1), gir1.2-gnomebluetooth-1.0:amd64 (3.4.2-1, 3.8.1-1), gir1.2-gck-1:amd64 (3.4.1-3, 3.8.2-3) End-Date: 2013-07-01 22:03:29
Now what happens when I log in to gnome via gdm3 I get an empty desktop with no menus, no hot-spots or anything else indicating that things worked out. Alt+F2 doesn't work either, and conky doesn't start.
The only thing that does work is
* my keyboard shortcuts (I've mapped ctrl+shift+Down arrow to chromium)
* guake (which starts with gnome)
ps aux|grep gnome-shellreturns nothing, which might be a clue.
Looking at the debian forums the closest post seems to be (although erroneously labelled -- gdm3 DOES start): http://forums.debian.net/viewtopic.php?f=6&t=105393&p=504077&hilit=gnome+shell#p504077
That in turn led to this bug report: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=712861
My gnome-shell version is 3.4.2-8,
I don't understand how gnome-bluetooth causes this, especially given that I've disabled bluetooth in rcconf, but whatever it takes...
I tried applying the patch but it failed:
mkdir ~/tmp cd ~/tmp wget "http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=66;filename=GnomeBluetooth.patch;att=1;bug=712861" -O blue.patch sed -i 's_js/ui/status/bluetooth.js_/usr/share/gnome-shell/js/ui/status/bluetooth.js_g' blue.patch sudo patch -p0 < blue.patch
Instead, I ended up making the changes to /usr/share/gnome-shell/js/ui/status/bluetooth.js by hand (remember that you can always use the ttys using ctrl+Fx):
6 const Gio = imports.gi.Gio; 7 const GnomeBluetoothApplet = imports.gi.GnomeBluetoothApplet; 8 const GnomeBluetooth = imports.gi.GnomeBluetooth; 9 const Gtk = imports.gi.Gtk;
and then delete the Applet part in GnomeBluetoothApplet so that it reads
38 this._killswitch.connect('toggled', Lang.bind(this, function() {
39 let current_state = this._applet.killswitch_state;
40 if (current_state != GnomeBluetooth.KillswitchState.HARD_BLOCKED &&
41 current_state != GnomeBluetooth.KillswitchState.NO_ADAPTER) {
42 this._applet.killswitch_state = this._killswitch.state ?
43 GnomeBluetooth.KillswitchState.UNBLOCKED:
44 GnomeBluetooth.KillswitchState.SOFT_BLOCKED;
45 } else
46 this._killswitch.setToggleState(false);
Then do it again:
96 _updateKillswitch: function() {
97 let current_state = this._applet.killswitch_state;
98 let on = current_state == GnomeBluetooth.KillswitchState.UNBLOCKED;
99 let has_adapter = current_state != GnomeBluetooth.KillswitchState.NO_ADAPTER;
100 let can_toggle = current_state != GnomeBluetooth.KillswitchState.NO_ADAPTER &&
101 current_state != GnomeBluetooth.KillswitchState.HARD_BLOCKED;
102
At this point I rebooted and everything was back to normal (you can try simply doing 'sudo service gdm3 restart' instead of rebooting).
Anyway, done.
470. Very briefly: compiling nwchem 6.3 with ifort and mkl
This used to be part of http://verahill.blogspot.com.au/2013/07/469-intel-compiler-on-debian.html, but I think it makes more sense making it a separate post.
I did this on debian wheezy.
1. Installing mkl and the compiler
MKL: http://verahill.blogspot.com.au/2013/06/465-intel-mkl-math-kernel-library-on.html
Intel compiler collection: http://verahill.blogspot.com.au/2013/07/469-intel-compiler-on-debian.html
I will henceforth presume that you have put the files in the same location as shown in those posts, and that you have created /etc/ld.so.conf.d/intel.conf as shown in the second post.
2 Compiling nwchem 6.3
And it works quite fine. See e.g. here if you want to patch to allow to compile with python, and to support gabedit.
3. Performance
This is always a bit contentious, and I want to be upfront with the fact that I haven't spent much time considering whether my test example is a good one. I simply did a geo-opt + vibrational analysis as shown in this post: http://verahill.blogspot.com.au/2013/05/430-briefly-crude-comparison-of.html
The jobs were run using all cores available on that node.
gnu= gfortran + acml 5.3.1 for the Phenom and FX8150, and openblas for the i5-2400 and the Athlon 3800+..
ifort= ifort + mkl for all architectures.
The times are in seconds and are CPU times, not wall times.
In the last case I also compiled using gfortran but with mkl and got 1550s.
It's a fairly small sample set, but it does seem that there's a little bit of an advantage with mkl+ifort over gfortran+acml on the newest AMD core. One would need much more data though.
A clear downside of using mkl and ifort is the fact that they are not freely available though -- i.e. you can register and download them for free for non-commercial use, but there's no guarantee that your colleague, next-door-neighbour or distant-cousin will be able to use it.
I did this on debian wheezy.
1. Installing mkl and the compiler
MKL: http://verahill.blogspot.com.au/2013/06/465-intel-mkl-math-kernel-library-on.html
Intel compiler collection: http://verahill.blogspot.com.au/2013/07/469-intel-compiler-on-debian.html
I will henceforth presume that you have put the files in the same location as shown in those posts, and that you have created /etc/ld.so.conf.d/intel.conf as shown in the second post.
2 Compiling nwchem 6.3
sudo apt-get install build-essential libopenmpi-dev openmpi-bin sudo mkdir /opt/nwchem -p sudo chown $USER:$USER /opt/nwchem cd /opt/nwchem wget http://www.nwchem-sw.org/download.php?f=Nwchem-6.3.revision1-src.2013-05-28.tar.gz -O Nwchem-6.3.revision1-src.2013-05-28.tar.gz tar xvf Nwchem-6.3.revision1-src.2013-05-28.tar.gz mv nwchem-6.3-src.2013-05-28 nwchem-6.3-src.2013-05-28.ifort export NWCHEM_TOP=`pwd` export LARGE_FILES=TRUE export TCGRSH=/usr/bin/ssh export NWCHEM_TOP=`pwd` export NWCHEM_TARGET=LINUX64 export NWCHEM_MODULES="all" export PYTHONVERSION=2.7 export PYTHONHOME=/usr export BLASOPT="-L/opt/intel/composer_xe_2013.4.183/mkl/lib/intel64/ -lmkl_core -lmkl_sequential -lmkl_intel_ilp64" export LIBRARY_PATH="$LIBRARY_PATH:/usr/lib/openmpi/lib:/opt/intel/composer_xe_2013.4.183/mkl/lib/intel64/" export USE_MPI=y export USE_MPIF=y export USE_MPIF4=y export MPI_LOC=/usr/lib/openmpi/lib export MPI_INCLUDE=/usr/lib/openmpi/include export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread" export ARMCI_NETWORK=SOCKETS cd $NWCHEM_TOP/src make clean make nwchem_config make FC=ifort 1> make.log 2>make.err cd $NWCHEM_TOP/contrib export FC=ifort ./getmem.nwchem
And it works quite fine. See e.g. here if you want to patch to allow to compile with python, and to support gabedit.
3. Performance
This is always a bit contentious, and I want to be upfront with the fact that I haven't spent much time considering whether my test example is a good one. I simply did a geo-opt + vibrational analysis as shown in this post: http://verahill.blogspot.com.au/2013/05/430-briefly-crude-comparison-of.html
The jobs were run using all cores available on that node.
gnu= gfortran + acml 5.3.1 for the Phenom and FX8150, and openblas for the i5-2400 and the Athlon 3800+..
ifort= ifort + mkl for all architectures.
The times are in seconds and are CPU times, not wall times.
Arch| cores gnu ifort Instruction sets ------------------------------------------------------------------------- AMD Athlon 64 X2 3800+: 2 10828* 12516 sse, sse2, sse3 AMD Phenom II X6 1055T: 6 2044 2048 sse, sse2, sse3 AMD FX8150 : 8 1611 1507 sse, sse2, sse3, AVX, FMA4 Intel i5-12400 : 4 1652* 1498 sse, sse2, sse3, sse4,AVX
In the last case I also compiled using gfortran but with mkl and got 1550s.
It's a fairly small sample set, but it does seem that there's a little bit of an advantage with mkl+ifort over gfortran+acml on the newest AMD core. One would need much more data though.
A clear downside of using mkl and ifort is the fact that they are not freely available though -- i.e. you can register and download them for free for non-commercial use, but there's no guarantee that your colleague, next-door-neighbour or distant-cousin will be able to use it.
01 July 2013
469. Intel compiler (icc, icpc, ifort) on Debian
I've heard it said that MKL is faster than ACML even on AMD cpus. I've also heard it said that Intel compile + mkl beats everything else, even on AMD cpus.
So let's test the veracity of that statement. I'm in particular looking forward to seeing how this affects my amd fx 8150.
Note that the ACML libs are available as separate packages for different compilers -- download the right libs when linking (i.e. the gnu ones for gcc, and the intel ones for intel composer)
But first we need to install and set-up the intel compiler suite, and that's what we'll do in this post.
In the example below I've installed in on an AMD Athlon II X3, hence the message about non-Intel architecture.
Installation:
Register for the Intel Parallel Studio XE as touched upon in this post: http://verahill.blogspot.com.au/2013/06/465-intel-mkl-math-kernel-library-on.html
Download. It's about 2 Gb. Then extract, and run install.sh
The Files and Setup:
The compiler binaries are now found in /opt/intel/composer_xe_2013.3.163/bin/intel64/ . Of particular interest are ifort, icc and icpc (fortran, c and c++).
In addition, you'll need the lib and include files, which are found in /opt/intel/composer_xe_2013.3.163/compiler/lib/intel64/ and /opt/intel/composer_xe_2013.3.163/compiler/include/intel64.
You can either simply add the libs using LD_LIBRARY_PATH, but a perhaps easier and better method is to create a file: /etc/ld.so.conf.d/intel.conf
Then do
Testing:
See this post for an example of how to compile nwchem using ifort: http://verahill.blogspot.com.au/2013/07/470-very-briefly-compiling-nwchem-63.html
So let's test the veracity of that statement. I'm in particular looking forward to seeing how this affects my amd fx 8150.
Note that the ACML libs are available as separate packages for different compilers -- download the right libs when linking (i.e. the gnu ones for gcc, and the intel ones for intel composer)
But first we need to install and set-up the intel compiler suite, and that's what we'll do in this post.
In the example below I've installed in on an AMD Athlon II X3, hence the message about non-Intel architecture.
Installation:
Register for the Intel Parallel Studio XE as touched upon in this post: http://verahill.blogspot.com.au/2013/06/465-intel-mkl-math-kernel-library-on.html
Download. It's about 2 Gb. Then extract, and run install.sh
sudo apt-get install build-essential sudo sh install.shStep no: 1 of 7 | Welcome -------------------------------------------------------------------------------- Welcome to the Intel(R) Parallel Studio XE 2013 Update 3 for Linux* installation program. -------------------------------------------------------------------------------- You will complete the steps below during this installation: Step 1 : Welcome Step 2 : License Step 3 : Activation Step 4 : Intel(R) Software Improvement Program Step 5 : Options Step 6 : Installation Step 7 : Complete -------------------------------------------------------------------------------- Press "Enter" key to continue or "q" to quit: -------------------------------------------------------------------------------- Checking the prerequisites. It can take several minutes. Please wait... -------------------------------------------------------------------------------- Step no: 1 of 7 | Options > Missing Optional Pre-requisite(s) -------------------------------------------------------------------------------- There are one or more optional unresolved issues. It is highly recommended to resolve them all before you continue the installation. You can fix them without exiting from the installation and re-check. Or you can quit from the installation, fix them and run the installation again. -------------------------------------------------------------------------------- Missing optional pre-requisites -- Intel(R) VTune(TM) Amplifier XE 2013 Update 5: unsupported OS -- Intel(R) Inspector XE 2013 Update 5: unsupported OS -- Intel(R) Advisor XE 2013 Update 2: unsupported OS -- Intel(R) Composer XE 2013 Update 3 for Linux*: unsupported OS -------------------------------------------------------------------------------- 1. Skip missing optional pre-requisites [default] 2. Show the detailed info about issue(s) 3. Re-check the pre-requisites h. Help b. Back to the previous menu q. Quit -------------------------------------------------------------------------------- Please type a selection or press "Enter" to accept default choice [1]: Step no: 2 of 7 | License -------------------------------------------------------------------------------- As noted in the Intel(R) Software Development Product End User License Agreement, the Intel(R) Software Development Product you install will send Intel [..] -------------------------------------------------------------------------------- Do you agree to be bound by the terms and conditions of this license agreement? Type "accept" to continue or "decline" to back to the previous menu: accept Step no: 3 of 7 | Activation -------------------------------------------------------------------------------- If you have purchased this product and have the serial number and a connection to the internet you can choose to activate the product at this time. Activation is a secure and anonymous one-time process that verifies your software licensing rights to use the product. Alternatively, you can choose to evaluate the product or defer activation by choosing the evaluate option. Evaluation software will time out in about one month. Also you can use license file, license manager, or remote activation if the system you are installing on does not have internet access activation options. -------------------------------------------------------------------------------- 1. I want to activate my product using a serial number [default] 2. I want to evaluate my product or activate later 3. I want to activate either remotely, or by using a license file, or by using a license manager h. Help b. Back to the previous menu q. Quit -------------------------------------------------------------------------------- Please type a selection or press "Enter" to accept default choice [1]: Note: Press "Enter" key to back to the previous menu. Please type your serial number (the format is XXXX-XXXXXXXX): -------------------------------------------------------------------------------- Activation completed successfully. -------------------------------------------------------------------------------- Press "Enter" key to continue: Step no: 4 of 7 | Intel(R) Software Improvement Program -------------------------------------------------------------------------------- Help improve your experience with Intel(R) software Participate in the design of future Intel software. Select 'Yes' to give us permission to learn about how you use your Intel software and we will do the rest. - No Personal contact information is collected - There are no surveys or additional follow-up emails by opting in - You can stop participating at any time Learn more about Intel(R) Software Improvement Program http://software.intel.com/en-us/articles/software-improvement-program With your permission, Intel may automatically receive anonymous information about how you use your current and future Intel software. -------------------------------------------------------------------------------- 1. Yes, I am willing to participate and improve Intel software. (Recommended) 2. No, I don't want to participate in the Intel(R) Software Improvement Program at this time. b. Back to the previous menu q. Quit -------------------------------------------------------------------------------- Please type a selection: Step no: 5 of 7 | Options -------------------------------------------------------------------------------- You are now ready to begin installation. You can use all default installation settings by simply choosing the "Start installation Now" option or you can customize these settings by selecting any of the change options given below first. You can view a summary of the settings by selecting "Show pre-install summary". -------------------------------------------------------------------------------- 1. Start installation Now 2. Change install directory [ /opt/intel ] 3. Change components to install [ All ] 4. Change advanced options 5. Show pre-install summary h. Help b. Back to the previous menu q. Quit -------------------------------------------------------------------------------- Please type a selection or press "Enter" to accept default choice [1]: -------------------------------------------------------------------------------- Checking the prerequisites. It can take several minutes. Please wait... -------------------------------------------------------------------------------- Step no: 5 of 7 | Options > Missing Optional Pre-requisite(s) -------------------------------------------------------------------------------- There are one or more optional unresolved issues. It is highly recommended to resolve them all before you continue the installation. You can fix them without exiting from the installation and re-check. Or you can quit from the installation, fix them and run the installation again. -------------------------------------------------------------------------------- Missing optional pre-requisites -- Intel(R) VTune(TM) Amplifier XE 2013 Update 5: The system does not have an Intel Architecture processor -------------------------------------------------------------------------------- 1. Skip missing optional pre-requisites [default] 2. Show the detailed info about issue(s) 3. Re-check the pre-requisites h. Help b. Back to the previous menu q. Quit -------------------------------------------------------------------------------- Please type a selection or press "Enter" to accept default choice [1]: Step no: 7 of 7 | Complete -------------------------------------------------------------------------------- Thank you for installing and using the Intel(R) Parallel Studio XE 2013 Update 3 for Linux* Reminder: Intel(R) VTune(TM) Amplifier XE users must be members of the "vtune" permissions group in order to use Event-based Sampling. To register your product purchase, visit https://registrationcenter.intel.com/RegCenter/registerexpress.aspx?clientsn=N43 3-4XGWTJLB To get started using Intel(R) VTune(TM) Amplifier XE 2013 Update 5: - To set your environment variables: source /opt/intel/vtune_amplifier_xe_2013/amplxe-vars.sh - To start the graphical user interface: amplxe-gui - To use the command-line interface: amplxe-cl - For more getting started resources: /opt/intel/vtune_amplifier_xe_2013/ documentation/en/welcomepage/get_started.html. To get started using Intel(R) Inspector XE 2013 Update 5: - To set your environment variables: source /opt/intel/inspector_xe_2013/inspxe-vars.sh - To start the graphical user interface: inspxe-gui - To use the command-line interface: inspxe-cl - For more getting started resources: /opt/intel/inspector_xe_2013/ documentation/en/welcomepage/get_started.html. To get started using Intel(R) Advisor XE 2013 Update 2: - To set your environment variables: source /opt/intel/advisor_xe_2013/advixe-vars.sh - To start the graphical user interface: advixe-gui - To use the command-line interface: advixe-cl - For more getting started resources: /opt/intel/advisor_xe_2013/ documentation/en/welcomepage/get_started.html. To get started using Intel(R) Composer XE 2013 Update 3 for Linux*: - Set the environment variables for a terminal window using one of the following (replace "intel64" with "ia32" if you are using a 32-bit platform). For csh/tcsh: $ source /opt/intel/bin/compilervars.csh intel64 For bash: $ source /opt/intel/bin/compilervars.sh intel64 To invoke the installed compilers: For C++: icpc For C: icc For Fortran: ifort To get help, append the -help option or precede with the man command. - For more getting started resources: /opt/intel/composer_xe_2013/Documentation/en_US/get_started_lc.htm. /opt/intel/composer_xe_2013/Documentation/en_US/get_started_lf.htm. To view movies and additional training, visit http://www.intel.com/software/products. -------------------------------------------------------------------------------- q. Quit [default] -------------------------------------------------------------------------------- Please type a selection or press "Enter" to accept default choice [q]:
The Files and Setup:
The compiler binaries are now found in /opt/intel/composer_xe_2013.3.163/bin/intel64/ . Of particular interest are ifort, icc and icpc (fortran, c and c++).
In addition, you'll need the lib and include files, which are found in /opt/intel/composer_xe_2013.3.163/compiler/lib/intel64/ and /opt/intel/composer_xe_2013.3.163/compiler/include/intel64.
You can either simply add the libs using LD_LIBRARY_PATH, but a perhaps easier and better method is to create a file: /etc/ld.so.conf.d/intel.conf
Once that's done, run/opt/intel/composer_xe_2013.3.163/compiler/lib/intel64 /opt/intel/composer_xe_2013.4.183/mkl/lib/intel64
sudo ldconfig
Then do
echo 'PATH=$PATH:/opt/intel/composer_xe_2013.3.163/bin/intel64' >> ~/.bashrc source ~/.bashrc
Testing:
See this post for an example of how to compile nwchem using ifort: http://verahill.blogspot.com.au/2013/07/470-very-briefly-compiling-nwchem-63.html
468. Kernel 3.10 on Debian
NOTE I: As of 3.10.2 the nvidia module will still not build. I've also tried 3.10.5 and it also does not work.
NOTE II: I'm getting random slowdowns on my SL410 laptop with intel graphics. Not sure if it's the same issue as this: http://verahill.blogspot.com.au/2013/03/368-slow-mouse-and-keyboard-triggered.html
Once kworker shows up in top everything grinds to a slow crawl. I also notice that I never used 3.9 on that laptop, so the issue may be present there too.
There are several ways of building a kernel. The easiest (a purely subjective statement) is to use kernel-package i.e. make-kpkg. However, every now and again I see people writing that it's been deprecated.
Either way, start by doing
You can now enable any additional modules by navigating the menu structure. Note that most likely you don't have to enable anything in this step, but it can come in handy if there's a major transition (e.g. the way multimedia was handled changed between kernel 3.5 and 3.6) or if you want to enable a previously disabled option.
Then pick either method 1 or 2 below.
If you only want to compile modules that are currently in use (not a good idea if you want to use the same kernel on a range of computers, or have USB devices that aren't currently plugged in) you can do that by using make localmodconfig instead of make oldconfig. I wouldn't recommend it -- in most cases it won't make a faster kernel, and space and memory tends not to be much of in the way of issues these days.
Method 1. kernel-package
Below, change -j2 to -jX, where X is the number of cores in your CPU (not cores+1 or anything funny like that. See other posts on this blog for compilation performance tests)
Took 49 minutes on a 3-core AMD Athlon II, and used ca 7 Gb.
The files are shown below:
Method 2. make deb-pkg
It took ca 50 minutes, it used ca 7 Gb, and generated the following files:
Note the size of the linux-image-3.10.0_3.10.0-2_amd64.deb package.
Difference:
There are a few differences. One method, the kpgk one, is supposedly 'deprecated', but it's been working fine in the past and will work fine in the future (at least for some time).
The make dev-pkg method also generates a much bigger image file -- by a factor of 15 or so.
Otherwise, the chief difference, from what I can see, is if you want to uninstall the kernel. Remove the libc-dev package requires you to downgrade the package using apt-get, by specifying a version e.g.
I personally prefer the kernel-package approach.
NOTE II: I'm getting random slowdowns on my SL410 laptop with intel graphics. Not sure if it's the same issue as this: http://verahill.blogspot.com.au/2013/03/368-slow-mouse-and-keyboard-triggered.html
Once kworker shows up in top everything grinds to a slow crawl. I also notice that I never used 3.9 on that laptop, so the issue may be present there too.
There are several ways of building a kernel. The easiest (a purely subjective statement) is to use kernel-package i.e. make-kpkg. However, every now and again I see people writing that it's been deprecated.
Either way, start by doing
sudo apt-get install fakeroot build-essential ncurses-dev mkdir ~/tmp cd ~/tmp wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.10.tar.xz tar xvf linux-3.10.tar.xz cd linux-3.10/ cat /boot/config-`uname -r`>.config make oldconfigTimer tick handling 1. Periodic timer ticks (constant rate, no dynticks) (HZ_PERIODIC) (NEW) > 2. Idle dynticks system (tickless idle) (NO_HZ_IDLE) (NEW) 3. Full dynticks system (tickless) (NO_HZ_FULL) (NEW) choice[1-3]: Memory placement aware NUMA scheduler (NUMA_BALANCING) [N/y/?] (NEW) Simple CPU accounting cgroup subsystem (CGROUP_CPUACCT) [N/y/?] (NEW) Group CPU scheduler (CGROUP_SCHED) [N/y/?] (NEW) Automatic process group scheduling (SCHED_AUTOGROUP) [N/y/?] (NEW) Choose SLAB allocator 1. SLAB (SLAB) (NEW) > 2. SLUB (Unqueued Allocator) (SLUB) choice[1-2?]: Linux guest support (HYPERVISOR_GUEST) [N/y/?] (NEW) Timer frequency 1. 100 HZ (HZ_100) 2. 250 HZ (HZ_250) (NEW) 3. 300 HZ (HZ_300) > 4. 1000 HZ (HZ_1000) choice[1-4?]: Memory Hotplug (ACPI_HOTPLUG_MEMORY) [N/y/?] (NEW) AMD frequency sensitivity feedback powersave bias (X86_AMD_FREQ_SENSITIVITY) [N/m/?] (NEW) Kernel support for scripts starting with #! (BINFMT_SCRIPT) [Y/n/m/?] (NEW) InfiniBand media type support (TIPC_MEDIA_IB) [N/y/?] (NEW) Network Coding (BATMAN_ADV_NC) [N/y/?] (NEW) NETLINK: mmaped IO (NETLINK_MMAP) [N/y/?] (NEW) NETLINK: socket monitoring interface (NETLINK_DIAG) [N/m/y/?] (NEW) Dummy IRQ handler (DUMMY_IRQ) [N/m/y/?] (NEW) Generic on-chip SRAM driver (SRAM) [N/y/?] (NEW) ME Enabled Intel Chipsets (INTEL_MEI_ME) [N/m/y/?] (NEW) Block device as cache (BCACHE) [N/m/y/?] (NEW) Qualcomm Atheros AR816x/AR817x support (ALX) [N/m/y/?] (NEW) QLOGIC QLCNIC 83XX family SR-IOV Support (QLCNIC_SRIOV) [Y/n/?] (NEW) Realtek RTL8152 Based USB 2.0 Ethernet Adapters (USB_RTL8152) [N/m/?] (NEW) Atheros ath9k rate control (ATH9K_LEGACY_RATE_CONTROL) [N/y/?] (NEW) rt2800usb - Include support for rt55xx devices (EXPERIMENTAL) (RT2800USB_RT55XX) [N/y/?] (NEW) y Realtek RTL8188EE Wireless Network Adapter (RTL8188EE) [N/m/?] (NEW) IMS Passenger Control Unit driver (INPUT_IMS_PCU) [N/m/?] (NEW) Qualcomm Single-wire Serial Bus Interface (SSBI) (SSBI) [N/m/y/?] (NEW) Analog Devices ADT7310/ADT7320 (SENSORS_ADT7310) [N/m/y/?] (NEW) National Semiconductor LM95234 (SENSORS_LM95234) [N/m/?] (NEW) Nuvoton NCT6775F and compatibles (SENSORS_NCT6775) [N/m/y/?] (NEW) generic cpu cooling support (CPU_THERMAL) [N/y/?] (NEW) y ChromeOS Embedded Controller (MFD_CROS_EC) [N/m/y/?] (NEW) Silicon Laboratories 4761/64/68 AM/FM radio. (MFD_SI476X_CORE) [N/m/?] (NEW) System Controller Register R/W Based on Regmap (MFD_SYSCON) [N/y/?] (NEW) TI TPS65912 Power Management chip (MFD_TPS65912) [N/y/?] (NEW) Conexant cx25821 support (VIDEO_CX25821) [N/m/?] (NEW) Cypress firmware helper routines (CYPRESS_FIRMWARE) [N/m] (NEW) QXL virtual GPU (DRM_QXL) [N/m/?] (NEW) Apple infrared receiver (HID_APPLEIR) [N/m/?] (NEW) Enable USB persist by default (USB_DEFAULT_PERSIST) [Y/n/?] (NEW) USB-Wishbone adapter interface driver (USB_SERIAL_WISHBONE) [N/m/?] (NEW) USB Physical Layer drivers (USB_PHY) [N/y/?] (NEW) PXA 27x (USB_PXA27X) [N/m/?] (NEW) MARVELL PXA2128 USB 3.0 controller (USB_MV_U3D) [N/m/?] (NEW) LED Support for TI LP5562 LED driver chip (LEDS_LP5562) [N/m/?] (NEW) LED Camera Flash/Torch Trigger (LEDS_TRIGGER_CAMERA) [N/m/y/?] (NEW) y iSCSI Extentions for RDMA (iSER) target support (INFINIBAND_ISERT) [N/m/?] (NEW) Set system time from RTC on startup and resume (RTC_HCTOSYS) [Y/n/?] (NEW) Set the RTC time based on NTP synchronization (RTC_SYSTOHC) [Y/n/?] (NEW) RTC used to set the system time (RTC_HCTOSYS_DEVICE) [rtc0] (NEW) WIS GO7007 MPEG encoder support (VIDEO_GO7007) [N/m/?] (NEW) DesignWare USB2 DRD Core Support (USB_DWC2) [N/m/?] (NEW) pvpanic device support (PVPANIC) [N/m/y/?] (NEW) Reset Controller Support (RESET_CONTROLLER) [N/y/?] (NEW) XFS Verbose Warnings (XFS_WARN) [N/y/?] (NEW) Btrfs will run sanity tests upon loading (BTRFS_FS_RUN_SANITY_TESTS) [N/y/?] (NEW) Btrfs debugging support (BTRFS_DEBUG) [N/y/?] (NEW) EFI Variable filesystem (EFIVAR_FS) [N/m/y/?] (NEW) torture tests for RCU (RCU_TORTURE_TEST) [N/m/y/?] (NEW) Ring buffer startup self test (RING_BUFFER_STARTUP_TEST) [N/y/?] (NEW) Test functions located in the string_helpers module at runtime (TEST_STRING_HELPERS) [N/m/y] (NEW) CMAC support (CRYPTO_CMAC) [N/m/y/?] (NEW) SHA256 digest algorithm (SSSE3/AVX/AVX2) (CRYPTO_SHA256_SSSE3) [N/m/y/?] (NEW) y SHA512 digest algorithm (SSSE3/AVX/AVX2) (CRYPTO_SHA512_SSSE3) [N/m/y/?] (NEW) y Camellia cipher algorithm (x86_64/AES-NI/AVX2) (CRYPTO_CAMELLIA_AESNI_AVX2_X86_64) [N/m/y/?] (NEW) m Serpent cipher algorithm (x86_64/AVX2) (CRYPTO_SERPENT_AVX2_X86_64) [N/m/y/?] (NEW) m KVM legacy PCI device assignment support (KVM_DEVICE_ASSIGNMENT) [Y/n/?] (NEW) VHOST_SCSI TCM fabric driver (VHOST_SCSI) [N/m/?] (NEW)make menuconfig
You can now enable any additional modules by navigating the menu structure. Note that most likely you don't have to enable anything in this step, but it can come in handy if there's a major transition (e.g. the way multimedia was handled changed between kernel 3.5 and 3.6) or if you want to enable a previously disabled option.
Then pick either method 1 or 2 below.
If you only want to compile modules that are currently in use (not a good idea if you want to use the same kernel on a range of computers, or have USB devices that aren't currently plugged in) you can do that by using make localmodconfig instead of make oldconfig. I wouldn't recommend it -- in most cases it won't make a faster kernel, and space and memory tends not to be much of in the way of issues these days.
Method 1. kernel-package
Below, change -j2 to -jX, where X is the number of cores in your CPU (not cores+1 or anything funny like that. See other posts on this blog for compilation performance tests)
sudo apt-get install kernel-package make-kpkg clean time fakeroot make-kpkg -j2 --initrd kernel_image kernel_headers sudo dpkg -i ../linux-image-3.10.0_3.10.0-10.00.Custom_amd64.deb ../linux-headers-3.10.0_3.10.0-10.00.Custom_amd64.deb
Took 49 minutes on a 3-core AMD Athlon II, and used ca 7 Gb.
The files are shown below:
-rw-r--r-- 1 me me 8.4M Jul 1 16:02 linux-headers-3.10.0_3.10.0-10.00.Custom_amd64.deb -rw-r--r-- 1 me me 32M Jul 1 16:00 linux-image-3.10.0_3.10.0-10.00.Custom_amd64.deb
Method 2. make deb-pkg
make clean time make deb-pkg -j2 sudo dpkg -i ../linux-firmware-image_3.10.0-2_amd64.deb ../linux-headers-3.10.0_3.10.0-2_amd64.deb ../linux-libc-dev_3.10.0-2_amd64.deb ../linux-image-3.10.0_3.10.0-2_amd64.deb
It took ca 50 minutes, it used ca 7 Gb, and generated the following files:
-rw-r--r-- 1 me me 1.1M Jul 1 16:58 ../linux-firmware-image_3.10.0-2_amd64.deb -rw-r--r-- 1 me me 9.7M Jul 1 16:58 ../linux-headers-3.10.0_3.10.0-2_amd64.deb -rw-r--r-- 1 me me 458M Jul 1 17:13 ../linux-image-3.10.0_3.10.0-2_amd64.deb -rw-r--r-- 1 me me 920K Jul 1 16:58 ../linux-libc-dev_3.10.0-2_amd64.deb
Note the size of the linux-image-3.10.0_3.10.0-2_amd64.deb package.
Difference:
There are a few differences. One method, the kpgk one, is supposedly 'deprecated', but it's been working fine in the past and will work fine in the future (at least for some time).
The make dev-pkg method also generates a much bigger image file -- by a factor of 15 or so.
Otherwise, the chief difference, from what I can see, is if you want to uninstall the kernel. Remove the libc-dev package requires you to downgrade the package using apt-get, by specifying a version e.g.
apt-cache policy linux-libc-devlinux-libc-dev: Installed: 3.10.0-1 Candidate: 3.10.0-1 Version table: *** 3.10.0-1 0 100 /var/lib/dpkg/status 3.9.6-1~bpo70+1 0 100 http://ftp.iinet.net.au/debian/debian/ wheezy-backports/main amd64 Packages 100 http://ftp.debian.org/debian/ wheezy-backports/main amd64 Packages 3.2.46-1 0 500 http://ftp.iinet.net.au/debian/debian/ wheezy/main amd64 Packages 3.2.41-2+deb7u2 0 500 http://security.debian.org/ wheezy/updates/main amd64 Packagessudo apt-get install linux-libc-dev=3.9.6-1~bpo70+1
I personally prefer the kernel-package approach.
467 wget and tor issue
Ever since I set up Tor on my debian workstation I've been having issues using wget:
I haven't configured /etc/wgetrc and so I was a bit surprised that this kept on happening.
The solution:
edit /etc/wgetrc and put
anywhere. And you're done.
wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.10.tar.xzFor fun I also tried torify although I don't want to download the kernel via Tor:--2013-07-01 11:32:35-- https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.10.tar.xz Connecting to 127.0.0.1:9050... connected. Proxy tunneling failed: Tor is not an HTTP ProxyUnable to establish SSL connection.
torify wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.10.tar.xzNote that I DON'T want to use wget with Tor. I don't want to eat up bandwidth on the Tor network for stuff like this. When I use wget I want to use a direct connection.--2013-07-01 11:36:20-- https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.10.tar.xz Connecting to 127.0.0.1:9050... 11:36:20 libtorsocks(26463): connect: Connection is to a local address (127.0.0.1), may be a TCP DNS request to a local DNS server so have to reject to be safe. Please report a bug to http://code.google.com/p/torsocks/issues/entry if this is preventing a program from working properly with torsocks. failed: No such file or directory. Retrying.
I haven't configured /etc/wgetrc and so I was a bit surprised that this kept on happening.
The solution:
edit /etc/wgetrc and put
use_proxy=off
anywhere. And you're done.
28 June 2013
466. morph xyz -- python script to morph .xyz files
Rather naively I was hoping that by comparing two molecule .xyz files and generating an average of them I would be able to conveniently generate a half-decent transition state guess.
Turns out that it's not quite as simple. However, I've written the software, so I might as well share it.
Note that it's written in python 2.7 (i.e. not python 3)
Run the script without arguments for help. General usage is
So here it is:
morphxyz:
Turns out that it's not quite as simple. However, I've written the software, so I might as well share it.
Note that it's written in python 2.7 (i.e. not python 3)
Run the script without arguments for help. General usage is
morphxyz -i 1.xyz 2.xyz -o morph.xyz
So here it is:
morphxyz:
#!/usr/bin/python
import sys
def getvars(arguments):
switches={}
version='0.1'
try:
if "-i" in arguments:
switches['in_one']=arguments[arguments.index('-i')+1]
switches['in_two']=arguments[arguments.index('-i')+2]
print 'Input: %s and %s'% (switches['in_one'],switches['in_two'])
else:
arguments="--help";
except:
arguments="--help";
try:
if "-o" in arguments:
switches['o']=arguments[arguments.index('-o')+1].lower()
print 'Output: %s'% switches['o']
else:
arguments="--help";
except:
arguments="--help";
try:
if "-w" in arguments:
switches['w']=float(arguments[arguments.index('-w')+1])
print 'Weighting: %i'% switches['w']
else:
print 'Assuming no weighting'
switches['w']=1.0;
except:
switches['w']=1.0;
doexit=0
try:
if ("-h" in arguments) or ("--help" in arguments):
print '\t\t bytes2words version %s' % version
print '\t-i\t two xyz files to morph'
print '\t-o\t output file'
print '\t-w\t weight one structure vs the other (1=average; 0=start; 2=end)'
print 'Exiting'
doexit=1
except:
a=0 # do nothing
if doexit==1:
sys.exit(0)
return switches
def getcmpds(switches):
cmpds={}
g=open(switches['in_one'],'r')
n=0
xyz=[]
atoms=[]
for line in g:
n+=1
line=line.rstrip('\n')
if n==1:
cmpds['atoms_one']=int(line)
elif n==2:
cmpds['title_one']=line
else:
line=line.split(' ')
line=filter(None,line)
xyz+=[[float(line[1]),float(line[2]),float(line[3])]]
atoms+=[line[0].capitalize()]
cmpds['coords_one']=xyz
cmpds['elements_one']=atoms
g.close
g=open(switches['in_two'],'r')
n=0
xyz=[]
atoms=[]
for line in g:
n+=1
line=line.rstrip('\n')
if n==1:
cmpds['atoms_two']=int(line)
elif n==2:
cmpds['title_two']=line
else:
line=line.split(' ')
line=filter(None,line)
xyz+=[[float(line[1]),float(line[2]),float(line[3])]]
atoms+=[line[0].capitalize()]
cmpds['coords_two']=xyz
cmpds['elements_two']=atoms
g.close
cmpds['w']=switches['w']
return cmpds
def morph(cmpds):
coords_one=cmpds['coords_one']
coords_two=cmpds['coords_two']
coords_morph=[]
coords_diff=[]
for n in range(0,cmpds['atoms_one']):
morph_x=coords_one[n][0]+cmpds['w']*(coords_two[n][0]-coords_one[n][0])/2.0
morph_y=coords_one[n][1]+cmpds['w']*(coords_two[n][1]-coords_one[n][1])/2.0
morph_z=coords_one[n][2]+cmpds['w']*(coords_two[n][2]-coords_one[n][2])/2.0
diff_x=coords_two[n][0]-coords_one[n][0]
diff_y=coords_two[n][1]-coords_one[n][1]
diff_z=coords_two[n][2]-coords_one[n][2]
coords_morph+=[[morph_x,morph_y,morph_z]]
coords_diff+=[[diff_x,diff_y,diff_z]]
cmpds['coords_morph']=coords_morph
cmpds['coords_diff']=coords_diff
return cmpds
def genxyzstring(coords,element):
x_str='%10.5f'% coords[0]
y_str='%10.5f'% coords[1]
z_str='%10.5f'% coords[2]
xyz_string=element+(3-len(element))*' '+10*' '+\
(8-len(x_str))*' '+x_str+10*' '+(8-len(y_str))*' '+y_str+10*' '+(8-len(z_str))*' '+z_str+'\n'
return xyz_string
def writemorph(cmpds,outfile):
g=open(outfile,'w')
h=open('diff.xyz','w')
g.write(str(cmpds['atoms_one'])+'\n'+'\n')
h.write(str(cmpds['atoms_one'])+'\n'+'\n')
for n in range(0,cmpds['atoms_one']):
coords=cmpds['coords_morph'][n]
diffcoords=cmpds['coords_diff'][n]
g.write(genxyzstring(coords, cmpds['elements_one'][n]))
h.write(genxyzstring(diffcoords, cmpds['elements_one'][n]))
g.close
h.close
return 0
if __name__=="__main__":
arguments=sys.argv[1:len(sys.argv)]
switches=getvars(arguments)
cmpds=getcmpds(switches)
if cmpds['atoms_one']!=cmpds['atoms_two']:
print 'The number of atoms differ. Exiting'
sys.exit(1)
elif cmpds['elements_one']!=cmpds['elements_two']:
print 'The types of atoms differ. Exiting'
sys.exit(1)
cmpds=morph(cmpds)
success=writemorph(cmpds,switches['o'])
if success==0:
print 'Conversion seems successful'
27 June 2013
465. The Intel MKL (Math Kernel Library) on Linux (Debian) -- for free
I've been living under the impression that the Intel MKL weren't free.
In all fairness, since I'm using AMD almost exclusively and the ACML is free -- and OpenBlas has worked fine on my i5-2400-based node -- I haven't had enough motivation to really dig into this. However, as part of a post on compiling GAMESS US, Kirill Berezovsky mentions in passing that you can get both the Intel MKL and ifortran for free for non-commercial use.
To be fair, that's free in the Windows sense, not in the Linux sense. You've still got enough restrictions to make Stallman weep, but it's free enough that we have a fighting chance at evaluation the software -- and use it if it's good enough.
See here for ACML: http://verahill.blogspot.com.au/2013/05/422-set-up-acml-on-linux.html
See here for OpenBlas: http://verahill.blogspot.com.au/2013/05/423-openblas-on-debian-wheezy.html
Anyway, this is about MKL:
0. Register a request
Go to http://software.intel.com/en-us/non-commercial-software-development
Click on what you want to download -- in this case the Intel MKL. Fill out the form and hit submit. Note that you'll need to enable cookies for this to work.
[while I was at it I got the intel parallel studio xe as well -- that's 2 Gb though, so don't get it unless you want it and have a lot of bandwidth to spare]
1. Download
You'll get an email with a link. Click it:
The MKL file is 609 MB, which is a bit bigger than the ACML files which tend to be around 67-70 Mb (each).
Note: the email you get has a serial number. From what I understand it's valid for one year if you want to download updates and new releases. However, nowhere does it say that any installed software will expire, so I presume that you can continue using the MKL libraries indefinitely.
2. Install
In my case the file is called l_mkl_11.0.4.183.tgz and it was downloaded to ~/Downloads
3. Usage
I'll demonstrate using nwchem. I've repeated the build instructions over and over on this blog, but here it goes again:
I had a look at http://www.ichec.ie/support/tutorials/mkl.pdf to better understand what the different libraries do. Still not quite clear though, so I might not be getting optimal performance.
4. Performance
I basically repeated the test job shown here: http://verahill.blogspot.com.au/2013/05/430-briefly-crude-comparison-of.html
The machine has 16 gb ram and an intel i5-2400 cpu (4 cores)
ACML: Total times cpu: 1644.3s wall: 1655.5s
MKL: Total times cpu: 1550.0s wall: 1563.8s
It certainly looks good. While 100 seconds in less than two minutes, and 1550 is 94 % of 1644, it's a difference of about one day per fortnight, which doesn't make much of a difference on a small single-user cluster, it would make a bit of cumulative difference on our 1400 core multi-user cluster.
In all fairness, since I'm using AMD almost exclusively and the ACML is free -- and OpenBlas has worked fine on my i5-2400-based node -- I haven't had enough motivation to really dig into this. However, as part of a post on compiling GAMESS US, Kirill Berezovsky mentions in passing that you can get both the Intel MKL and ifortran for free for non-commercial use.
To be fair, that's free in the Windows sense, not in the Linux sense. You've still got enough restrictions to make Stallman weep, but it's free enough that we have a fighting chance at evaluation the software -- and use it if it's good enough.
See here for ACML: http://verahill.blogspot.com.au/2013/05/422-set-up-acml-on-linux.html
See here for OpenBlas: http://verahill.blogspot.com.au/2013/05/423-openblas-on-debian-wheezy.html
Anyway, this is about MKL:
0. Register a request
Go to http://software.intel.com/en-us/non-commercial-software-development
Click on what you want to download -- in this case the Intel MKL. Fill out the form and hit submit. Note that you'll need to enable cookies for this to work.
[while I was at it I got the intel parallel studio xe as well -- that's 2 Gb though, so don't get it unless you want it and have a lot of bandwidth to spare]
1. Download
You'll get an email with a link. Click it:
The MKL file is 609 MB, which is a bit bigger than the ACML files which tend to be around 67-70 Mb (each).
Note: the email you get has a serial number. From what I understand it's valid for one year if you want to download updates and new releases. However, nowhere does it say that any installed software will expire, so I presume that you can continue using the MKL libraries indefinitely.
2. Install
In my case the file is called l_mkl_11.0.4.183.tgz and it was downloaded to ~/Downloads
mkdir ~/tmp cd ~/tmp cp ~/Downloads/l_mkl_11.0.4.183.tgz . tar xvf l_mkl_11.0.4.183.tgz cd l_mkl_11.0.4.183 sudo sh install.shStep no: 1 of 7 | Welcome -------------------------------------------------------------------------------- Welcome to the Intel(R) Math Kernel Library 11.0 Update 4 for Linux* installation program. The Flagship of HPC Math Libraries. This library contains highly optimized, extensively threaded, mathematical functions for engineering, scientific, and financial applications that require maximum performance. -------------------------------------------------------------------------------- You will complete the steps below during this installation: Step 1 : Welcome Step 2 : License Step 3 : Activation Step 4 : Intel(R) Software Improvement Program Step 5 : Options Step 6 : Installation Step 7 : Complete -------------------------------------------------------------------------------- Step no: 1 of 7 | Options > Missing Optional Pre-requisite(s) -------------------------------------------------------------------------------- There are one or more optional unresolved issues. It is highly recommended to resolve them all before you continue the installation. You can fix them without exiting from the installation and re-check. Or you can quit from the installation, fix them and run the installation again. -------------------------------------------------------------------------------- Missing optional pre-requisites -- unsupported OS -------------------------------------------------------------------------------- 1. Skip missing optional pre-requisites [default] 2. Show the detailed info about issue(s) 3. Re-check the pre-requisites h. Help b. Back to the previous menu q. Quit -------------------------------------------------------------------------------- Please type a selection or press "Enter" to accept default choice [1]: 1 [..] 13. THIRD PARTY PROGRAMS. The Materials may include third party programs or materials that are governed by the third party's license terms, including without limitation, open source software. The license terms associated with such third party programs or materials govern your use of same, and Intel is not liable for such third party programs or materials. * Other names and brands may be claimed as the property of others -------------------------------------------------------------------------------- Do you agree to be bound by the terms and conditions of this license agreement? Type "accept" to continue or "decline" to back to the previous menu: accept Step no: 3 of 7 | Activation -------------------------------------------------------------------------------- If you have purchased this product and have the serial number and a connection to the internet you can choose to activate the product at this time. Activation is a secure and anonymous one-time process that verifies your software licensing rights to use the product. Alternatively, you can choose to evaluate the product or defer activation by choosing the evaluate option. Evaluation software will time out in about one month. Also you can use license file, license manager, or remote activation if the system you are installing on does not have internet access activation options. -------------------------------------------------------------------------------- 1. I want to activate my product using a serial number [default] 2. I want to evaluate my product or activate later 3. I want to activate either remotely, or by using a license file, or by using a license manager h. Help b. Back to the previous menu q. Quit -------------------------------------------------------------------------------- Please type a selection or press "Enter" to accept default choice [1]: 1 Note: Press "Enter" key to back to the previous menu. Please type your serial number (the format is XXXX-XXXXXXXX): -------------------------------------------------------------------------------- Activation completed successfully. -------------------------------------------------------------------------------- Press "Enter" key to continue: Step no: 4 of 7 | Intel(R) Software Improvement Program -------------------------------------------------------------------------------- Help improve your experience with Intel(R) software Participate in the design of future Intel software. Select 'Yes' to give us permission to learn about how you use your Intel software and we will do the rest. - No Personal contact information is collected - There are no surveys or additional follow-up emails by opting in - You can stop participating at any time Learn more about Intel(R) Software Improvement Program http://software.intel.com/en-us/articles/software-improvement-program With your permission, Intel may automatically receive anonymous information about how you use your current and future Intel software. -------------------------------------------------------------------------------- 1. Yes, I am willing to participate and improve Intel software. (Recommended) 2. No, I don't want to participate in the Intel(R) Software Improvement Program at this time. b. Back to the previous menu q. Quit -------------------------------------------------------------------------------- Please type a selection: 2 Step no: 5 of 7 | Options -------------------------------------------------------------------------------- You are now ready to begin installation. You can use all default installation settings by simply choosing the "Start installation Now" option or you can customize these settings by selecting any of the change options given below first. You can view a summary of the settings by selecting "Show pre-install summary". -------------------------------------------------------------------------------- 1. Start installation Now 2. Change install directory [ /opt/intel/composer_xe_2013.4.183 ] 3. Change components to install [ All ] 4. Show pre-install summary h. Help b. Back to the previous menu q. Quit -------------------------------------------------------------------------------- Please type a selection or press "Enter" to accept default choice [1]: 1 Step no: 6 of 7 | Installation -------------------------------------------------------------------------------- Each component will be installed individually. If you cancel the installation, components that have been completely installed will remain on your system. This installation may take several minutes, depending on your system and the options you selected. -------------------------------------------------------------------------------- Installing Intel Math Kernel Library 11.0 Update 4 on IA-32 component... done -------------------------------------------------------------------------------- Installing Intel Math Kernel Library 11.0 Update 4 on Intel(R) 64 component... done -------------------------------------------------------------------------------- Finalizing installation... done -------------------------------------------------------------------------------- Press "Enter" key to continue Step no: 7 of 7 | Complete -------------------------------------------------------------------------------- Thank you for installing and for using the Intel(R) Math Kernel Library 11.0 Update 4 for Linux*. Support services start from the time you install or activate your product, so please create your support account now in order to take full advantage of your product purchase. Your Subscription Service support account provides access to free product updates interactive issue management, technical support, sample code, and documentation. To create your support account, please visit the Subscription Services web site https://registrationcenter.intel.com/RegCenter/registerexpress.aspx?clientsn=NBJ N-87S2RM3P To get started using Intel(R) Math Kernel Library 11.0 Update 4 located in /opt/intel/composer_xe_2013.4.183 visit: install-dir/Documentation/en_US/mkl/get_started.html. -------------------------------------------------------------------------------- q. Quit [default] -------------------------------------------------------------------------------- Please type a selection or press "Enter" to accept default choice [q]:
3. Usage
I'll demonstrate using nwchem. I've repeated the build instructions over and over on this blog, but here it goes again:
sudo apt-get install build-essential gfortran python2.7-dev libopenmpi-dev openmpi-bin sudo mkdir /opt/nwchem -p sudo chown $USER:$USER /opt/nwchem cd /opt/nwchem wget http://www.nwchem-sw.org/download.php?f=Nwchem-6.3.revision1-src.2013-05-28.tar.gz -O Nwchem-6.3.revision1-src.2013-05-28.tar.gz tar xvf Nwchem-6.3.revision1-src.2013-05-28.tar.gz cd nwchem-6.3-src.2013-05-28/ export NWCHEM_TOP=`pwd` export LARGE_FILES=TRUE export TCGRSH=/usr/bin/ssh export NWCHEM_TOP=`pwd` export NWCHEM_TARGET=LINUX64 export NWCHEM_MODULES="all python" export PYTHONVERSION=2.7 export PYTHONHOME=/usr export BLASOPT="-L/opt/intel/composer_xe_2013.4.183/mkl/lib/intel64/ -lmkl_sequential -lmkl_core -lmkl_intel_ilp64" export LIBRARY_PATH="$LIBRARY_PATH:/usr/lib/openmpi/lib:/opt/intel/composer_xe_2013.4.183/mkl/lib/intel64/" export USE_MPI=y export USE_MPIF=y export USE_MPIF4=y export MPI_LOC=/usr/lib/openmpi/lib export MPI_INCLUDE=/usr/lib/openmpi/include export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread" export ARMCI_NETWORK=SOCKETS cd $NWCHEM_TOP/src make clean make nwchem_config make FC=gfortran 1> make.log 2>make.err cd $NWCHEM_TOP/contrib export FC=gfortran ./getmem.nwchem
I had a look at http://www.ichec.ie/support/tutorials/mkl.pdf to better understand what the different libraries do. Still not quite clear though, so I might not be getting optimal performance.
4. Performance
I basically repeated the test job shown here: http://verahill.blogspot.com.au/2013/05/430-briefly-crude-comparison-of.html
The machine has 16 gb ram and an intel i5-2400 cpu (4 cores)
ACML: Total times cpu: 1644.3s wall: 1655.5s
MKL: Total times cpu: 1550.0s wall: 1563.8s
It certainly looks good. While 100 seconds in less than two minutes, and 1550 is 94 % of 1644, it's a difference of about one day per fortnight, which doesn't make much of a difference on a small single-user cluster, it would make a bit of cumulative difference on our 1400 core multi-user cluster.
26 June 2013
464. bytes2words -- python script
This script does something I could easily do myself in e.g. bc, and so is a bit of a waste of time. However, because I enjoy writing short python scripts, I did it anyway.
Usage:
Usage:
/bytes2words -i 8gib -o MWInput: 8gib Output: mw Assuming 64 bit word size Result: 8 gib is 1073 mwbytes2words:
#!/usr/bin/python
# Converts bytes to words
# The impetus comes from the use of MW as the default memory unit in many computational pieces of software.
import sys
def split_text(s): # from http://stackoverflow.com/questions/12409894
from itertools import groupby
for k,g in groupby(s, str.isalpha):
yield ''.join(list(g))
def getargs(arguments):
switches={}
version='0.1'
try:
if "-i" in arguments:
switches['i']=arguments[arguments.index('-i')+1]
print 'Input: %s'% switches['i']
inputted=list(split_text(switches['i']))
if len(inputted)>1:
switches['i']=inputted[0]
switches['unit']=inputted[len(inputted)-1].lower()
if switches['unit'] == 'kib':
switches['if']=1024
elif switches['unit'] == 'kb':
switches['if']=1000
elif switches['unit'] == 'mb':
switches['if']=1000*1000
elif switches['unit'] == 'mib':
switches['if']=1024*1024
elif switches['unit'] == 'gb':
switches['if']=1000*1000*1000
elif switches['unit'] == 'gib':
switches['if']=1024*1024*1024
else:
switches['if']=1
if len(inputted)>2:
print 'Illegal input: %s'% inputted
arguments="--help"
else:
arguments="--help";
except:
arguments="--help";
try:
if "-o" in arguments:
switches['o']=arguments[arguments.index('-o')+1].lower()
print 'Output: %s'% switches['o']
if switches['o'] == 'w':
switches['of']=1
elif switches['o'] == 'kiw':
switches['of']=1024
elif switches['o'] == 'kw':
switches['of']=1000
elif switches['o'] == 'mw':
switches['of']=1000*1000
elif switches['o'] == 'miw':
switches['of']=1024*1024
elif switches['o'] == 'gw':
switches['of']=1000*1000*1000
elif switches['o'] == 'giw':
switches['of']=1024*1024*1024
else:
print 'illegal output argument' % switches['o']
else:
arguments="--help";
except:
arguments="--help";
version='0.1'
try:
if "-b" in arguments:
switches['b']=int(arguments[arguments.index('-b')+1])
print 'Word size: %i bits'% switches['b']
else:
print 'Assuming 64 bit word size'
switches['b']=64;
except:
switches['b']=64;
doexit=0
try:
if ("-h" in arguments) or ("--help" in arguments):
print '\t\t bytes2words version %s' % version
print ' \t-i\t input in words with units, e.g. 200kb'
print ' \t-o\t output unit (w,kw,mw,gw)'
print ' \t-b\t word size (32 or 64 (bit))'
print 'Exiting'
doexit=1
except:
a=0 # do nothing
if doexit==1:
sys.exit(0)
return switches
if __name__=="__main__":
bitsperbyte=8
arguments=sys.argv[1:len(sys.argv)]
switches=getargs(arguments)
print 'Result: %s %s is %s %s' % ( (switches['i']),switches['unit'],int(switches['if']*(float(switches['i'])/float(switches['b']/bitsperbyte))/float(switches['of'])),switches['o'])
463. Very Briefly: Installing ia32-libs
Since the skype and bankid posts on this blog rely on ia32-libs, and the question has popped up a few times:
To install ia32-libs on debian wheezy and up you need to enable multiarch:
To install ia32-libs on debian wheezy and up you need to enable multiarch:
sudo dpkg --add-architecture i386 sudo apt-get update sudo apt-get install ia32-libs
24 June 2013
462. Olex2 (1.1) on Debian
Olex2 is an open-source program for solving and refining crystal structures. I am not a crystallographer -- I have never (successfully= publication quality output) solved a crystal structure in my life -- and I have no idea whether Olex2 is an alternative or a complement to SHELX.
What I do now is that Olex2 is more of a point-and-click type program with a snazzy GUI, and so it would seem that the learning curve may be shallower than that of SHELX. However, as always you must always be cautious when doing science -- just because it's easy to do, doesn't mean that you're doing it right (I'm looking at you, biologists/physicians/social scientists using statistical software).
But maybe Olex2 makes getting started just easy enough that you can slowly work your way towards actually slowly get to the point where you know and understand what you are doing?
That's my hope anyway.
Quick note: there's no source code on the http://www.olex2.org/ website. However, there's a sourceforge SVN repository: http://sourceforge.net/projects/olex2/
Ideally I should show you how to download and compile and make a .deb package from the source, but sadly I haven't managed to figure it out (I'll add troubleshooting data to this post later). I even tried using the deb rules by http://jupiter.plymouth.edu/~jsduncan/software/olex2.php (again, I might add troubleshooting data later)
Anyway, using the pre-built binaries is easy enough. Only the older version 1.1 seems to be working ok on Debian Wheezy.
Installation:
Note that I've generated a list of dependencies the lazy way -- by looking at the ldd output.
Anyway.
You'll need a raft of libraries, including libGL. You can get this from different sources depending on your graphics card (or lack thereof). Install libgl1-mesa-glx or libgl1-fglrx-glx (ati) or libgl1-nvidia-glx (nvidia). Then continue.
Again, note that you may want to change the versions of the packages below that you install. Anyway, here's a list that may work:
Get the compiled binary:
Create the file ~/.local/share/applications/olex2.desktop
Olex2 is very pretty, but feels a bit incomplete and buggy though (again, I don't know if these are real issues or not -- and I'm no expert). The issue is that the program seems to throw errors every now and again. For example when clicking on electron density map in the Solve window:
Problem with version 1.2:
Start Olex2. 1.2 takes a lot longer than 1.1 to start, but that's fine. Click on Tutorials. Click on Maps and Masks. CLick Next five times (the step where you actually create the mask). It triggers the following error:
Problems with compiling:
There's a CMakeLists.txt file, but when doing e.g.
I get
What I do now is that Olex2 is more of a point-and-click type program with a snazzy GUI, and so it would seem that the learning curve may be shallower than that of SHELX. However, as always you must always be cautious when doing science -- just because it's easy to do, doesn't mean that you're doing it right (I'm looking at you, biologists/physicians/social scientists using statistical software).
But maybe Olex2 makes getting started just easy enough that you can slowly work your way towards actually slowly get to the point where you know and understand what you are doing?
That's my hope anyway.
Quick note: there's no source code on the http://www.olex2.org/ website. However, there's a sourceforge SVN repository: http://sourceforge.net/projects/olex2/
Ideally I should show you how to download and compile and make a .deb package from the source, but sadly I haven't managed to figure it out (I'll add troubleshooting data to this post later). I even tried using the deb rules by http://jupiter.plymouth.edu/~jsduncan/software/olex2.php (again, I might add troubleshooting data later)
Anyway, using the pre-built binaries is easy enough. Only the older version 1.1 seems to be working ok on Debian Wheezy.
Installation:
Note that I've generated a list of dependencies the lazy way -- by looking at the ldd output.
Anyway.
You'll need a raft of libraries, including libGL. You can get this from different sources depending on your graphics card (or lack thereof). Install libgl1-mesa-glx or libgl1-fglrx-glx (ati) or libgl1-nvidia-glx (nvidia). Then continue.
Again, note that you may want to change the versions of the packages below that you install. Anyway, here's a list that may work:
sudo apt-get install libpng3 libglu1-mesa libgtk2.0-0 libatk1.0-0 libgdk-pixbuf2.0-0 libpango1.0-0 libglib2.0-0 libfreetype6 libxrender1 libfontconfig1 libx11-6 libxext6 libpng12-0 libxinerama1 libxxf86vm1 libsm6 lib32z1 zlib1g libc6 libpython2.6 libstdc++6 libgcc1 libxcomposite1 libxdamage1 libxfixes3 libcairo2 libxi6 libxrandr2 libxcursor1 libffi5 libexpat1 libpcre3 libxcb1 libice6 libuuid1 libssl1.0.0 libpixman-1-0 libxcb-shm0 libxcb-render0 libselinux1 libxau6 libxdmcp6
Get the compiled binary:
cd ~ wget http://www.olex2.org/olex2-distro/1.1/olex2-linux64.zip unzip olex2-linux64.zip cd ~/olex2 sed -i 's./work/distro/olex2-new.$HOME/olex2.g' start
Create the file ~/.local/share/applications/olex2.desktop
You're now ready to use Olex2. If you launch it by hand, use the start script. Otherwise just launch is from your desktop.[Desktop Entry] Name=Olex2 GenericName=Olex2 Comment=Software for refinement of crystal structures Exec=sh /home/verahill/olex2/start Terminal=false Type=Application Categories=Science Version=1.1
Olex2 is very pretty, but feels a bit incomplete and buggy though (again, I don't know if these are real issues or not -- and I'm no expert). The issue is that the program seems to throw errors every now and again. For example when clicking on electron density map in the Solve window:
Map sigma 0.000 : CalcFourier {diff=}{r=0.1}{m=} N4esdl25TInvalidArgumentExceptionE mask size at [xlib/fracmask.cpp(Init):22] calcFourier -diff -r=0.1 -m
Problem with version 1.2:
Start Olex2. 1.2 takes a lot longer than 1.1 to start, but that's fine. Click on Tutorials. Click on Maps and Masks. CLick Next five times (the step where you actually create the mask). It triggers the following error:
An error occured running the function/macro next_demo_item Traceback (most recent call last): File "olexFunctions.py", line 480, in func retVal = f(*args, **kwds) File "/home/verahill/olex2_1.2/etc/scripts/Tutorials.py", line 138, in next_demo_item self.run_demo_item() File "/home/verahill/olex2_1.2/etc/scripts/Tutorials.py", line 318, in run_demo_item flash_gui_control(control) File "gui/tools/__init__.py", line 123, in flash_gui_control OV.Refresh() File "guiFunctions.py", line 61, in Refresh olx.Refresh() File "/home/verahill/.olex2/data/4f11cd71424c9e8484c8c4f91644e3b6/olx/__init__.py", line 1319, in Refresh for arg in args: RuntimeError: [repository/pyext.cpp(runOlexFunctionEx):354]: Function 'html.SetImage' failed: wrong html object name: 'IMG_H3-H3-MASKS'Key variable values: args = () kwds = {} al = []
Problems with compiling:
sudo apt-get install libwxgtk2.8-dev mkdir ~/tmp/svn_co cd ~/tmp/svn_co svn checkout svn://svn.code.sf.net/p/olex2/code/branches/1.2 olex2-1.2 cd olex2-1.2/ make/home/verahill/tmp/svn_co/olex2-1.2/gxlib/gxapp.cpp: In member function ‘esdl::TUndoData* gxlib::TGXApp::Name(gxlib::TXAtom&, const olxstr&, bool)’: /home/verahill/tmp/svn_co/olex2-1.2/gxlib/gxapp.cpp:2003:73: error: no matching function for call to ‘gxlib::TGXApp::SynchroniseBonds(gxlib::TXAtomPList)’ /home/verahill/tmp/svn_co/olex2-1.2/gxlib/gxapp.cpp:2003:73: note: candidate is: [..] /home/verahill/tmp/svn_co/olex2-1.2/gxlib/gxapp.cpp: In member function ‘esdl::TUndoData* gxlib::TGXApp::Name(const olxstr&, const olxstr&, bool, bool)’: /home/verahill/tmp/svn_co/olex2-1.2/gxlib/gxapp.cpp:2136:71: error: no matching function for call to ‘gxlib::TGXApp::SynchroniseBonds(gxlib::TXAtomPList)’ /home/verahill/tmp/svn_co/olex2-1.2/gxlib/gxapp.cpp:2136:71: note: candidate is: [..[ make[1]: *** [/home/verahill/tmp/svn_co/olex2-1.2/obj/gxapp.s] Error 1 make[1]: *** Waiting for unfinished jobs.... make[1]: *** [/home/verahill/tmp/svn_co/olex2-1.2/obj/gxmacro.s] Error 1 make[1]: Leaving directory `/home/verahill/tmp/svn_co/olex2-1.2' make: *** [all] Error 2
There's a CMakeLists.txt file, but when doing e.g.
mkdir build_olex cd build_olex/ cmake ../olex2-1.2
I get
CMake Error: Error in cmake code at /home/verahill/tmp/svn_co/olex2-1.2/CMakeLists.txt:49: Parse error. Expected a command name, got unquoted argument with text "${olex2_html_SRCS}". -- Configuring incomplete, errors occurred!
461. Briefly: setting up SHELX on linux (crystallography)
Nothing difficult, but putting up instructions won't hurt anyone.
SHELX is THE crystal structure refinement software. I'm not a crystallographer, but it never hurts familiarising yourself with the tools of your collaborators.
Download
Register using this page (if you're an academic user): http://shelx.uni-ac.gwdg.de/SHELX/register.php
To find the answer to the xtal question, use google.
You'll then receive an email with a password. Now go to
http://shelx.uni-ac.gwdg.de/SHELX/download.php
where you'll find instructions.
Download all the files:
Several of my files became corrupted the first time I downloaded them for some reason: anode.bz2, shelx[cde].bz2, shredcif.bz2.
I'm presuming that you're downloading the files to ~/Downloads
Here are the 'good' md5sums:
While you're at it, download the testdata from http://shelx.uni-ac.gwdg.de/~gsheldr/bin/test_data/: cdetutorial_andrea.zip, ciftab_templates.zip, difficult_sad.zip, ltests.zip, pn1a.zip
Installation
If you downloaded the test data:
And you're done. Now, learning how to use SHELX, and how to use it properly, is a different matter on which I am not qualified to write.
SHELX is THE crystal structure refinement software. I'm not a crystallographer, but it never hurts familiarising yourself with the tools of your collaborators.
Download
Register using this page (if you're an academic user): http://shelx.uni-ac.gwdg.de/SHELX/register.php
To find the answer to the xtal question, use google.
You'll then receive an email with a password. Now go to
http://shelx.uni-ac.gwdg.de/SHELX/download.php
where you'll find instructions.
Download all the files:
Several of my files became corrupted the first time I downloaded them for some reason: anode.bz2, shelx[cde].bz2, shredcif.bz2.
I'm presuming that you're downloading the files to ~/Downloads
Here are the 'good' md5sums:
219183542ada47a17e5528bf217f9261 anode.bz2 61335e6b9cf2e654242db80822f32681 ciftab.bz2 918fe0a04e59589938a81a93d8e3eaff shelxc.bz2 e65580af087989aa4958eb53dcd8a473 shelxd.bz2 bc5cad6e4129fa61bbde49207cd4d244 shelxe.bz2 5390146a4b516425fb7b326533443ba7 shelxl.bz2 95617863be917743df55bd94509504fb shelxs.bz2
While you're at it, download the testdata from http://shelx.uni-ac.gwdg.de/~gsheldr/bin/test_data/: cdetutorial_andrea.zip, ciftab_templates.zip, difficult_sad.zip, ltests.zip, pn1a.zip
Installation
sudo apt-get install bunzip2 xargs
mkdir ~/tmp/shelx-2013 -p
cd ~/Downloads
cp shredcif.bz2 shelxe.bz2 shelxd.bz2 shelxc.bz2 ciftab.bz2 anode.bz2 shelxl.bz2 shelxs.bz2 ~/tmp/shelx-2013
cd ~/tmp/shelx-2013/
ls *.bz2|xargs -I {} bunzip2 {}
chmod +x *
sudo cp * /usr/local/bin
If you downloaded the test data:
mkdir ~/tmp/shelx_examples
cd ~/Downloads
cp cdetutorial_andrea.zip ciftab_templates.zip difficult_sad.zip ltests.zip pn1a.zip ~/tmp/shelx_examples
cd ~/shelx_examples
ls *.zip |xargs -I {} unzip {}
And you're done. Now, learning how to use SHELX, and how to use it properly, is a different matter on which I am not qualified to write.
21 June 2013
460. Briefly: Crystallography software: CCSD Mercury
The Cambridge Crystallographic Data Centre (CCDC)/Crystal Structure Database (CSD) has a free structure viewer called Mercury. Downloading and installing it is pretty straightforward, but still makes for a reasonable post.
To install
Go to http://www.ccdc.cam.ac.uk/SupportandResources/Downloads/pages/ProtectedDownloadProductList.aspx and click on Mercury. It'll take you to a license agreement page, Click accept to continue. Note that it won't work if you are blocking cookies.
Download Mecury 3.1 for Linux, and the 3.1.1 patch for linux. I'll presume that you downloaded the files to ~/Downloads.
Finally, create a file called
~/.local/share/applications/mercury.desktop
And you are done! Note that you will want to have working OpenGL for this to look ok.
To install
Go to http://www.ccdc.cam.ac.uk/SupportandResources/Downloads/pages/ProtectedDownloadProductList.aspx and click on Mercury. It'll take you to a license agreement page, Click accept to continue. Note that it won't work if you are blocking cookies.
Download Mecury 3.1 for Linux, and the 3.1.1 patch for linux. I'll presume that you downloaded the files to ~/Downloads.
cd ~/Downloads chmod +x mercurystandalone-3.1-linux-installer.run ./mercurystandalone-3.1-linux-installer.run
~/.local/share/applications/mercury.desktop
Update to 3.1.1[Desktop Entry] Name=Mercury GenericName=CCDC Mercury Comment=Visualization of crystal structures Exec=/home/verahill/.Mercury_3.1/bin/mercury Icon=/home/verahill/.Mercury_3.1/icons/mercury_48x48.png Terminal=false Type=Application Categories=Science Version=3.1
cd ~/Downloads chmod +x csdsystempatch-5.34.2-linux-installer.run ./csdsystempatch-5.34.2-linux-installer.run
And you are done! Note that you will want to have working OpenGL for this to look ok.
20 June 2013
459. Briefly: Proxies, browsing and paranoia
It's easy to configure Chrome to use Tor to preserve a semblance of privacy online (http://verahill.blogspot.com.au/2013/06/450-tor-and-chrome-on-debian.html). There are a few, simple things you can do to make your life with a proxy easier to manage.
This post presumes that you've followed this post first: http://verahill.blogspot.com.au/2013/06/450-tor-and-chrome-on-debian.html. In particular, that you have turned off pre-fetching.
In addition, you may want to think about the following:
Incognito mode
On the lower end of the scale, you may or may not want to use incognito mode consistently. This has little bearing on privacy online, but it depends on whether you want to leave traces on your computer of your browsing history. Although that should only be an issue if someone gets physical access to your computer, you never know if the next browser bug will give someone complete access to your history. Most likely it'll only provide metadata (which is what the NSA brouhaha has been mostly about).
Anyway, if you feel this is an important issue then you should probably be encrypting your disks with encfs as well.
Search engine
It's probably more important to rethink how you are using search engines in Chrome. First of all, you should turn off instant search. Secondly, you will want to consider whether you want to use google as the default search engine for queries in the URL field. Two main search engines come to mind: duckduckgo.com, and startpage.com. While duckduckgo.com has a higher profile, startpage.com is a bit more full-featured, and that's because it takes your query, anonymizes it, and passes it on to google. It's also based in Europe, which I (probably naively) feel is safer.
Go to startpage.com, and click on 'add to chrome' under the search box. Then set Startpage HTTPS as the default in Chrome:
Also consider making sure that google.com isn't your home page in chrome.
Proxy
Even though Tor works fine in general, it can be a bit slow, and you don't want to use it for everything anyway. There are times when you don't want to use a proxy. In my case, that's when I visit journal websites or my university websites. Also, I have set up a reverse proxy via my home router, and it's faster than Tor, so for a lot of things I'm fine with using that.
Switch ProxySharp supports the creation of rule-based proxy switching. In my case, I've set it so that if I use google, I use Tor. If I go to RSC, ACS, Wiley or Elsevier journals, I use my university connection, and for everything else, I use my home router.
You then just need to click your way through to the proxyswitcher alternative:
The icon will change colour depending on which proxy is active. Pretty neat!
This post presumes that you've followed this post first: http://verahill.blogspot.com.au/2013/06/450-tor-and-chrome-on-debian.html. In particular, that you have turned off pre-fetching.
In addition, you may want to think about the following:
Incognito mode
On the lower end of the scale, you may or may not want to use incognito mode consistently. This has little bearing on privacy online, but it depends on whether you want to leave traces on your computer of your browsing history. Although that should only be an issue if someone gets physical access to your computer, you never know if the next browser bug will give someone complete access to your history. Most likely it'll only provide metadata (which is what the NSA brouhaha has been mostly about).
Anyway, if you feel this is an important issue then you should probably be encrypting your disks with encfs as well.
Search engine
It's probably more important to rethink how you are using search engines in Chrome. First of all, you should turn off instant search. Secondly, you will want to consider whether you want to use google as the default search engine for queries in the URL field. Two main search engines come to mind: duckduckgo.com, and startpage.com. While duckduckgo.com has a higher profile, startpage.com is a bit more full-featured, and that's because it takes your query, anonymizes it, and passes it on to google. It's also based in Europe, which I (probably naively) feel is safer.
Go to startpage.com, and click on 'add to chrome' under the search box. Then set Startpage HTTPS as the default in Chrome:
Also consider making sure that google.com isn't your home page in chrome.
Proxy
Even though Tor works fine in general, it can be a bit slow, and you don't want to use it for everything anyway. There are times when you don't want to use a proxy. In my case, that's when I visit journal websites or my university websites. Also, I have set up a reverse proxy via my home router, and it's faster than Tor, so for a lot of things I'm fine with using that.
Switch ProxySharp supports the creation of rule-based proxy switching. In my case, I've set it so that if I use google, I use Tor. If I go to RSC, ACS, Wiley or Elsevier journals, I use my university connection, and for everything else, I use my home router.
You then just need to click your way through to the proxyswitcher alternative:
19 June 2013
458. Briefly: Converting GRAMS ASP ascii data to two-column ascii data
We have a couple of CARY 630 FT-IR /ATR instruments.
I hate them. Apart from being the Mac equivalent of spectrometers (if you try to do anything remotely creative you'll have a bad day. Point and click works well, most of the time), they aren't able to output data in any reasonable format.
At least not the way I'd define 'reasonable' i.e. simple x-y ascii data file and/or JCAMP-DX and/or even .csv. The default output is a binary .a2r file.
The only ascii-type format is a proprietary GRAMS ASP ascii file, for which I haven't been able to get the formal specs. Using google it seems as if the German arm of agilent did publish it, but when clicking on the links I'm told the file no longer exists, and google cache isn't playing ball.
Anyway. Luckily the format seems pretty simple.
Here are the first ten lines of an .asp file;
* The second line is the highest reciprocal wavelength in cm-1.
* The third line is the lowest reciprocal wavelength in cm_1.
* I don't know what the fourth and fifth lines signify. It could be dynamic resolution in the Y axis.
* The sixth line is the native resolution, i.e. 4 cm-1/data point. However, the data seems to be zero-filled, i.e. it seems the resolution is really ca 1.86 cm-1/pt.
Knowing the above, we can write a simple python script, which we'll call asp2asc, which will allow us to generate files suitable for gnuplot.
Example usage:
asp2asc:
Of course you could do this easily in a spreadsheet too, but I honestly find myself avoiding spreadsheet programmes like the plague ever since I learned how to use sed, gawk, and python.
Also, WHY do they make it so unnecessarily difficult to export your own data?
I hate them. Apart from being the Mac equivalent of spectrometers (if you try to do anything remotely creative you'll have a bad day. Point and click works well, most of the time), they aren't able to output data in any reasonable format.
At least not the way I'd define 'reasonable' i.e. simple x-y ascii data file and/or JCAMP-DX and/or even .csv. The default output is a binary .a2r file.
The only ascii-type format is a proprietary GRAMS ASP ascii file, for which I haven't been able to get the formal specs. Using google it seems as if the German arm of agilent did publish it, but when clicking on the links I'm told the file no longer exists, and google cache isn't playing ball.
Anyway. Luckily the format seems pretty simple.
Here are the first ten lines of an .asp file;
* The first line is the number of acquired data points1798 4000.41016197344 650.579285428114 1 128 4 98.4862110783457 98.4183476284596 98.4587565715995 98.5660576694946
* The second line is the highest reciprocal wavelength in cm-1.
* The third line is the lowest reciprocal wavelength in cm_1.
* I don't know what the fourth and fifth lines signify. It could be dynamic resolution in the Y axis.
* The sixth line is the native resolution, i.e. 4 cm-1/data point. However, the data seems to be zero-filled, i.e. it seems the resolution is really ca 1.86 cm-1/pt.
Knowing the above, we can write a simple python script, which we'll call asp2asc, which will allow us to generate files suitable for gnuplot.
Example usage:
./asp2asc -i data.asp -o data.dat
asp2asc:
#!/usr/bin/python
#converts GRAMS ascii (asp) output from an CARY 630 FT-ATR-IR to a two-column ascii dat file
import sys
def getvars(arguments):
exit=0
ver=0.1
try:
if "-o" in arguments:
theoutput=arguments[arguments.index('-o')+1]
print 'Output: %s.'%theoutput
elif "--output" in arguments:
theoutput=arguments[arguments.index('--output')+1]
print 'Output: %s.'%theoutput
else:
print ''
print 'Error -- no output file defined.'
print ''
arguments="--help"
except:
arguments="--help"
try:
if "-i" in arguments:
theinput=arguments[arguments.index('-i')+1]
print 'Input: %s.'%theinput
elif "--input" in arguments:
theinput=arguments[arguments.index('--input')+1]
print 'Input: %s.'%theinput
else:
print ''
print 'Error -- no input file defined.'
print ''
arguments="--help"
except:
arguments="--help"
try:
if ("-h" in arguments) or ("--help" in arguments):
print " "
print "\t\tThis is asp2asc, a tool for generating converting"
print "\t\tGRAMS ASP ascii files to two-column ascii files"
print "\t\tThis is version",ver
print "\tUsage:"
print "\t-h\t--help \tYou're looking at it."
print "\t-i\t--input \tInput file, e.g. data.asp"
print "\t-o\t--output \tOutput file, e.g. data.dat"
print ""
exit=1
except:
a=1 #do nothing
if exit==1:
sys.exit(0)
print ''
switches={'i':theinput,'o':theoutput}
return switches
def getparams(datafile):
params=[]
n=1
for line in datafile:
try:
params+=[int(line.rstrip('\n'))]
except:
params+=[float(line.rstrip('\n'))]
if n==6:
break
n+=1
return params
def getydata(datafile):
ydata=[]
for line in datafile:
ydata+=[float(line.rstrip('\n'))]
return ydata
def makexdata(xpts,xmax,increment):
n=0
xdata=[]
while n < xpts:
xdata+=[xmax-n*increment]
n+=1
return xdata
def writexydata(outfile,xdata,ydata):
for n in range(0,len(xdata)):
outfile.write(str(xdata[n])+'\t'+str(ydata[n])+'\n')
return 0
if __name__ == "__main__":
arguments=sys.argv[1:len(sys.argv)]
switches=getvars(arguments)
infile=open(switches['i'],'r')
params=getparams(infile)
ydata=getydata(infile) # needs getparams to have parked file reading at the 7th line
infile.close()
xdata=makexdata(params[0],params[1],(params[1]-params[2])/(params[0]-1))
if len(xdata)==len(ydata):
outfile=open(switches['o'],'w')
success=writexydata(outfile,xdata,ydata)
outfile.close()
else:
print 'Something bad happened:'
print 'Number of X data points not equal to number of Y data points'
print 'x pts: %i, y pts: %i'%(len(xdata),len(ydata))
Of course you could do this easily in a spreadsheet too, but I honestly find myself avoiding spreadsheet programmes like the plague ever since I learned how to use sed, gawk, and python.
Also, WHY do they make it so unnecessarily difficult to export your own data?
457. Very Briefly: Microsoft has a Tor exit node?
Whenever I play around with Tor I use ipchicken.com or whatsmyip.org to make sure that I'm indeed using a proxy. I also normally do a whois on the IP address, so see who's running the exit node.
Today I ended up with the IP address 168.61.8.22.
Microsoft as a company isn't exactly known for doing things out of the goodness of their hearts. Oh well.
Today I ended up with the IP address 168.61.8.22.
whois 168.61.8.22That Microsoft is listed as the organisation doesn't necessarily mean that they are running the node (could be a hosting company) but it still seems that this might actually be MS running this one. Maybe it's just for research purposes, but it still seemed a bit surprising.NetRange: 168.61.0.0 - 168.63.255.255 CIDR: 168.62.0.0/15, 168.61.0.0/16 OriginAS: NetName: MSFT-EP NetHandle: NET-168-61-0-0-1 Parent: NET-168-0-0-0-0 NetType: Direct Assignment RegDate: 2011-06-22 Updated: 2012-10-16 Ref: http://whois.arin.net/rest/net/NET-168-61-0-0-1 OrgName: Microsoft Corp OrgId: MSFT-Z Address: One Microsoft Way City: Redmond StateProv: WA PostalCode: 98052 Country: US RegDate: 2011-06-22 Updated: 2013-04-12 Ref: http://whois.arin.net/rest/org/MSFT-Z OrgTechHandle: MSFTP-ARIN OrgTechName: MSFT-POC OrgTechPhone: +1-425-882-8080 OrgTechEmail: iprrms@microsoft.com OrgTechRef: http://whois.arin.net/rest/poc/MSFTP-ARIN OrgAbuseHandle: HOTMA-ARIN OrgAbuseName: Hotmail Abuse OrgAbusePhone: +1-425-882-8080 OrgAbuseEmail: abuse@hotmail.com OrgAbuseRef: http://whois.arin.net/rest/poc/HOTMA-ARIN OrgAbuseHandle: MSNAB-ARIN OrgAbuseName: MSN ABUSE OrgAbusePhone: +1-425-882-8080 OrgAbuseEmail: abuse@msn.com OrgAbuseRef: http://whois.arin.net/rest/poc/MSNAB-ARIN OrgNOCHandle: ZM23-ARIN OrgNOCName: Microsoft Corporation OrgNOCPhone: +1-425-882-8080 OrgNOCEmail: noc@microsoft.com OrgNOCRef: http://whois.arin.net/rest/poc/ZM23-ARIN OrgAbuseHandle: ABUSE231-ARIN OrgAbuseName: Abuse OrgAbusePhone: +1-425-882-8080 OrgAbuseEmail: abuse@microsoft.com OrgAbuseRef: http://whois.arin.net/rest/poc/ABUSE231-ARIN
Microsoft as a company isn't exactly known for doing things out of the goodness of their hearts. Oh well.
17 June 2013
456. Adding NWChem basis sets to ECCE. Part 2. A solution: nwchem2ecce.py
UPDATED!
I've moved the finished scripts to here:
https://sourceforge.net/projects/nwbas2ecce/
They work! I've also added a number of converted basis sets to the sourceforge repo under 'examples'. You'll also find example ecp and ECPOrbital files.
Phew...
Here's the README:
I've moved the finished scripts to here:
https://sourceforge.net/projects/nwbas2ecce/
They work! I've also added a number of converted basis sets to the sourceforge repo under 'examples'. You'll also find example ecp and ECPOrbital files.
Phew...
Here's the README:
The programmes are not 'intelligent' -- they won't check that you are doing something reasonable. Bad input = bad output. __Installation__: Download eccepag and nwbas2ecce They are both python (2.7) programmes, so you will need to install python to run them. On linux, this is normally very easy. E.g. on debian, run 'sudo apt-get install python2.7' and you are done. If you want, you can put the files in /usr/local/bin and do 'sudo chmod +x /usr/local/bin/eccepage' 'sudo chmod +x /usr/local/bin/nwbas2ecce' and you will be able to call the scripts from any directory. __Usage__ nwbas2ecce can turn a full basis set, or a, ECP basis set, into an ECCE compatible set of basis set files. Typically, an nwchem basis set consists of a single file, e.g. 3-21g. It can also be divided into several files, e.g. def2-svp and def-ecp, where the effective core potentials (ecps) are in def2-ecp. Other basis set files, like lanl2dz_ecp, contains both the orbital and the contraction parts. Typically, a ECCE basis set suite consists of: basis.BAS basis.BAS.meta basis.POT (for ECP) basis.POT.meta (for ECP) Sometimes polarization and diffuse functions are separated from the main .BAS file. E.g. 3-21++G* consists of 3-21G.BAS 3-21GS.BAS POPLDIFF.BAS , in addition to the meta files. The meta files are just markup-language type files with e.g. references. Note that you don't HAVE to break up the basis set components like this. Since the basis set data can be broken up into smaller files, the overall basis set is defined as an entry in a category file. For example, 3-21G is defined in the category file 'pople', and points to 3-21G.BAS. 3-21G* is also defined in pople, but point to both 3-21G.BAS and 3-21GS.BAS. ECP works in a similar way, by combining a .BAS and a .POT file. Note that the .POT files look different from the .BAS files. nwbas2ecce generates .BAS and .POT files based on whether there are basis/end or ecp/end sections in the nwchem basis set file. If there are both, both POT and BAS files are generated. All these files are contained in server/data/Ecce/system/GaussianBasisSetLibrary Finally, you need to generate .pag and .dir files that go into the server/data/Ecce/system/GaussianBasisSetLibrary/.DAV directory. The .dir file is always empty, while the .pag file is unfortunately a binary file. eccepag can, however, generate it with the right input. See e.g. http://verahill.blogspot.com.au/2013/06/455-adding-nwchem-basis-sets-to-ecce.html for more detailed information __Example__ We'll use def2-svp as an example. The nwchem basis set file def2-svp contains the basis set, while def2-ecp contains the core potentials. Use def2-svp to generate DEF2_SVP.BAS, DEF2_SVP.BAS.meta. Use def2-ecp to generate DEF2_ECP.POT, DEF2_ECP.POT.meta. As part of the generation, .descriptor files are also generated. These contain information that should go into the category file(s). Then generate the .pag files for both the POT and the BAS files, and touch the .dir files into existence. Do like this: nwbas2ecce -i def2-svp -o DEF2_SVP.BAS -n 'def2-svp' nwbas2ecce -i def2-ecp -p DEF2_ECP.POT -n 'def2-ecp' eccepag -n def2-svp -t ECPOrbital -c ORBITAL -y Segmented -s Y -o DEF2_SVP.BAS.pag eccepag -n def2-ecp -t ecp -c AUXILIARY -o DEF2_ECP.POT.pag NOTE: I don't actually know if def2-svp is segmented, and spherical. I don't think it matters for the .pag file generation. Also note that most inputs are case sensitive. Look at a similar .pag file for hints. You now have the following files: DEF2_ECP.POT DEF2_ECP.POT.descriptor DEF2_ECP.POT.meta DEF2_ECP.POT.pag DEF2_SVP.BAS DEF2_SVP.BAS.descriptor DEF2_SVP.BAS.meta DEF2_SVP.BAS.pag Copy the files. Note that you need to select the correct target directory, and that will vary with where you installed ECCE. I'll assume it's in /opt/ecce cp DEF2* /opt/ecce/server/data/Ecce/system/GaussianBasisSetLibrary cd /opt/ecce/server/data/Ecce/system/GaussianBasisSetLibrary mv *.pag .DAV/ touch .DAV/DEF2_SVP.BAS.dir .DAV/DEF2_ECP.POT.dir cat DEF2_SVP.BAS.descriptor >> ECPOrbital cat DEF2_ECP.POT.descriptor >> ECPOrbital cat DEF2_ECP.POT.descriptor >> ecp Edit ECPOrbital so that it reads: name= def2-svp files= DEF2_SVP.BAS DEF2_ECP.POT atoms= H He Li Be B C N O F Ne Na Mg Al Si P S Cl Ar K Ca Sc Ti V Cr Mn Fe Co Ni Cu Zn Ga Ge As Se Br Kr Rb Sr Y Zr Nb Mo Tc Ru Rh Pd Ag Cd In Sn Sb Te I Xe Cs Ba La Hf Ta W Re Os Ir Pt Au Hg Tl Pb Bi Po At Rn atoms= Rb Sr Y Zr Nb Mo Tc Ru Rh Pd Ag Cd In Sn Sb Te I Xe Cs Ba La Hf Ta W Re Os Ir Pt Au Hg Tl Pb Bi Po At Rn/pre>
Subscribe to:
Posts (Atom)