Many, many years ago I learned basic programming using BASIC (the version that came with PC DOS 5, I think). I even wrote the odd game, but it was all pretty awful. A few years later I learned Turbo Pascal, which was a fantastic experience compared to Basic. It felt all sciency and grown up, and it was my first experience with a real IDE. I even ended up buying a TP book, and became somewhat proficient. This must've been when I was around 18-19.I then stopped programming completely.
At around 30 years of age I decided it was time to get serious about programming again -- I was doing mass spectrometry and needed a simple program that could generate a series of solutions to the identity of a mass/charge ratio given a range of elements. I probably had a quick look at C and C++, but ended up getting a Python book and have been happy Python programmer ever since.
The problem is that I've never been a /good/ python programmer -- and in all these years I've never fully understood the use for (or, in all fairness, use OF) OOP. And at the moment it seems to be holding me back -- all the examples that I find of the use the threading module as well as writing GUIs (using e.g. wxPython) involve using classes. And I just don't understand them well enough to sort out what I need done.
Anyway, long story short: I've written a basic program for communicating with a Metrohm Titrino 736 GP via RS 232. It's found here: https://sourceforge.net/projects/pytitrino/
Currently:
* the code is a mess (see above)
* it works fine for doing monotonic and dynamic end point titrations (MET and DET)
* it saves data to a file, but does so silently (i.e. when you run you won't get any feedback that things are working properly...)
* it uses the thread (not threading) module
* I've managed to pass parameters back and forth between the thread and the main loop using Queue
There are probably much better solutions. One day I hope to be able to stick a GUI on top of it, but the more I look at it I get the impression that one writes the GUI first, then the engine...not that I'd know.
Anyway. That's what I've been up to. Anyone with a bit of programming experience, whom is in possession of an old-school Titrino (i.e. using RS 232) and wants to save $1.5k in software licenses may be interested in taking the sources and turning them into something useful.
Lindqvist - a blog about Linux and Science. Mostly.
Showing posts with label serial. Show all posts
Showing posts with label serial. Show all posts
11 September 2012
231. Compiling john the ripper: single/serial, parallel/OMP and MPI
Update: updated for v1.7.9-jumbo-7 since hccap2john in 1.7.9-jumbo-6 was broken
For no particular reason at all, here's how to compile John the Ripper on Debian Testing (Wheezy). It's very easy, and this post is probably a bit superfluous. The standard version only supports serial and parallel (OMP). See below for MPI.
The regular version:
mkdir ~/tmp
cd ~/tmp
wget http://www.openwall.com/john/g/john-1.7.9.tar.gz
tar xvf john-1.7.9.tar.gz
cd john-1.7.9/src
If you don't edit the Makefile you build a serial/single-threaded binary.
If you want to build a threaded version for a single node with a multicore processor (OMP) do:
Edit Makefile and uncomment row 19 or 20
You now have a binary called john in your ../run folder.
Edit the Makefile
and do
make clean linux-x86-64-native
cd ../run
I had a look at the passwords on one of our lab boxes -- it immediately discovered that someone had used 'password' as the password...
These test were run on my old AMD II X3 445. Processes which don't speed up with MP are highlighted in red. LM DES is borderline -- it's faster, but doesn't scale well.
Here's the single thread/serial version:
./john --test
And here's the MPI version:
mpirun -n 3 ./john --test
(note that this includes a great many more tests than the default version)
I don't quite understand the Kerberos results.
Other targets of interest are:
linux-x86-64-avx Linux, x86-64 with AVX (2011+ Intel CPUs)
linux-x86-64-xop Linux, x86-64 with AVX and XOP (2011+ AMD CPUs)
linux-x86-64 Linux, x86-64 with SSE2 (most common)
linux-x86-avx Linux, x86 32-bit with AVX (2011+ Intel CPUs)
linux-x86-xop Linux, x86 32-bit with AVX and XOP (2011+ AMD CPUs)
linux-x86-sse2 Linux, x86 32-bit with SSE2 (most common, if 32-bit)
linux-x86-mmx Linux, x86 32-bit with MMX (for old computers)
linux-x86-any Linux, x86 32-bit (for truly ancient computers)
The FX 8150 does AVX and XOP, while my 1055T doesn't.
The community version has more options:
linux-x86-64-native Linux, x86-64 'native' (all CPU features you've got)
linux-x86-64-gpu Linux, x86-64 'native', CUDA and OpenCL (experimental)
linux-x86-64-opencl Linux, x86-64 'native', OpenCL (experimental)
linux-x86-64-cuda Linux, x86-64 'native', CUDA (experimental)
linux-x86-64-avx Linux, x86-64 with AVX (2011+ Intel CPUs)
linux-x86-64-xop Linux, x86-64 with AVX and XOP (2011+ AMD CPUs)
linux-x86-64[i] Linux, x86-64 with SSE2 (most common)
linux-x86-64-icc Linux, x86-64 compiled with icc
linux-x86-64-clang Linux, x86-64 compiled with clang
linux-x86-gpu Linux, x86 32-bit with SSE2, CUDA and OpenCL (experimental)
linux-x86-opencl Linux, x86 32-bit with SSE2 and OpenCL (experimental)
linux-x86-cuda Linux, x86 32-bit with SSE2 and CUDA (experimental)
linux-x86-sse2[i] Linux, x86 32-bit with SSE2 (most common, 32-bit)
linux-x86-native Linux, x86 32-bit, with all CPU features you've got (not necessarily best)
linux-x86-mmx Linux, x86 32-bit with MMX (for old computers)
linux-x86-any Linux, x86 32-bit (for truly ancient computers)
linux-x86-clang Linux, x86 32-bit with SSE2, compiled with clang
linux-alpha Linux, Alpha
linux-sparc Linux, SPARC 32-bit
linux-ppc32-altivec Linux, PowerPC w/AltiVec (best)
linux-ppc32 Linux, PowerPC 32-bit
linux-ppc64 Linux, PowerPC 64-bit
linux-ia64 Linux, IA-64
For no particular reason at all, here's how to compile John the Ripper on Debian Testing (Wheezy). It's very easy, and this post is probably a bit superfluous. The standard version only supports serial and parallel (OMP). See below for MPI.
The regular version:
mkdir ~/tmp
cd ~/tmp
wget http://www.openwall.com/john/g/john-1.7.9.tar.gz
tar xvf john-1.7.9.tar.gz
cd john-1.7.9/src
If you don't edit the Makefile you build a serial/single-threaded binary.
If you want to build a threaded version for a single node with a multicore processor (OMP) do:
Edit Makefile and uncomment row 19 or 20
18 # gcc with OpenMP
19 OMPFLAGS = -fopenmp
20 OMPFLAGS = -fopenmp -msse2
make clean linux-x86-64
cd ../runYou now have a binary called john in your ../run folder.
The Jumbo version:
If you want to build a distributed version with MPI (can split jobs across several nodes) you need the enhanced, community version:
If you want to build a distributed version with MPI (can split jobs across several nodes) you need the enhanced, community version:
sudo apt-get install openmpi-bin libopenmpi-dev
cd ~/tmp
wget http://www.openwall.com/john/g/john-1.7.9-jumbo-7.tar.gz
tar xvf john-1.7.9-jumbo-7.tar.gz
cd john-1.7.9-jumbo-7/src
Edit the Makefile
20 ## Uncomment the TWO lines below for MPI (can be used together with OMP as well)
21 ## For experimental MPI_Barrier support, add -DJOHN_MPI_BARRIER too.
22 ## For experimental MPI_Abort support, add -DJOHN_MPI_ABORT too.
23 CC = mpicc -DHAVE_MPI
24 MPIOBJ = john-mpi.o
and do
cd ../run
I had a look at the passwords on one of our lab boxes -- it immediately discovered that someone had used 'password' as the password...
These test were run on my old AMD II X3 445. Processes which don't speed up with MP are highlighted in red. LM DES is borderline -- it's faster, but doesn't scale well.
Here's the single thread/serial version:
./john --test
Benchmarking: Traditional DES [128/128 BS SSE2-16]... DONE
Many salts: 2906K c/s real, 2918K c/s virtual
Only one salt: 2796K c/s real, 2807K c/s virtual
Benchmarking: BSDI DES (x725) [128/128 BS SSE2-16]... DONE
Many salts: 95564 c/s real, 95948 c/s virtual
Only one salt: 93593 c/s real, 93781 c/s virtual
Benchmarking: FreeBSD MD5 [32/64 X2]... DONE
Raw: 14094 c/s real, 14122 c/s virtual
Benchmarking: OpenBSD Blowfish (x32) [32/64 X2]... DONE
Raw: 918 c/s real, 919 c/s virtual
Benchmarking: Kerberos AFS DES [48/64 4K]... DONE
Short: 474316 c/s real, 475267 c/s virtual
Long: 1350K c/s real, 1356K c/s virtual
Benchmarking: LM DES [128/128 BS SSE2-16]... DONE
Raw: 39843K c/s real, 39923K c/s virtual
Benchmarking: generic crypt(3) [?/64]... DONE
Many salts: 262867 c/s real, 263393 c/s virtual
Only one salt: 260121 c/s real, 260642 c/s virtual
Benchmarking: Tripcode DES [48/64 4K]... DONE
Raw: 369843 c/s real, 370584 c/s virtual
Benchmarking: dummy [N/A]... DONE
Raw: 99512K c/s real, 99712K c/s virtual
Here's the OMP version:
Benchmarking: Traditional DES [128/128 BS SSE2-16]... DONE
Many salts: 6706K c/s real, 2555K c/s virtual
Only one salt: 5015K c/s real, 2091K c/s virtual
Benchmarking: BSDI DES (x725) [128/128 BS SSE2-16]... DONE
Many salts: 205670 c/s real, 85411 c/s virtual
Only one salt: 238524 c/s real, 86720 c/s virtual
Benchmarking: FreeBSD MD5 [32/64 X2]... DONE
Raw: 38400 c/s real, 13812 c/s virtual
Benchmarking: OpenBSD Blowfish (x32) [32/64 X2]... DONE
Raw: 2306 c/s real, 845 c/s virtual
Benchmarking: Kerberos AFS DES [48/64 4K]... DONE
Short: 474675 c/s real, 476581 c/s virtual
Long: 1332K c/s real, 1335K c/s virtual
Benchmarking: LM DES [128/128 BS SSE2-16]... DONE
Raw: 49046K c/s real, 16785K c/s virtual
Benchmarking: generic crypt(3) [?/64]... DONE
Many salts: 721670 c/s real, 246640 c/s virtual
Only one salt: 699168 c/s real, 239605 c/s virtual
Benchmarking: Tripcode DES [48/64 4K]... DONE
Raw: 367444 c/s real, 369657 c/s virtual
Benchmarking: dummy [N/A]... DONE
Raw: 100351K c/s real, 100552K c/s virtual
mpirun -n 3 ./john --test
(note that this includes a great many more tests than the default version)
Benchmarking: Traditional DES [128/128 BS SSE2-16]... (3xMPI) DONE
Many salts: 8533K c/s real, 8707K c/s virtual
Only one salt: 7705K c/s real, 8110K c/s virtual
Benchmarking: BSDI DES (x725) [128/128 BS SSE2-16]... (3xMPI) DONE
Many salts: 279808 c/s real, 282634 c/s virtual
Only one salt: 273362 c/s real, 276096 c/s virtual
Benchmarking: FreeBSD MD5 [128/128 SSE2 intrinsics 12x]... (3xMPI) DONE
Raw: 65124 c/s real, 65781 c/s virtual
Benchmarking: OpenBSD Blowfish (x32) [32/64 X2]... (3xMPI) DONE
Raw: 2722 c/s real, 2749 c/s virtual
Benchmarking: Kerberos AFS DES [48/64 4K]... (3xMPI) DONE
Short: 1387K c/s real, 1415K c/s virtual
Long: 3880K c/s real, 3959K c/s virtual
Benchmarking: LM DES [128/128 BS SSE2-16]... (3xMPI) DONERaw: 114781K c/s real, 115940K c/s virtual
I don't quite understand the Kerberos results.
Other targets of interest are:
linux-x86-64-avx Linux, x86-64 with AVX (2011+ Intel CPUs)
linux-x86-64-xop Linux, x86-64 with AVX and XOP (2011+ AMD CPUs)
linux-x86-64 Linux, x86-64 with SSE2 (most common)
linux-x86-avx Linux, x86 32-bit with AVX (2011+ Intel CPUs)
linux-x86-xop Linux, x86 32-bit with AVX and XOP (2011+ AMD CPUs)
linux-x86-sse2 Linux, x86 32-bit with SSE2 (most common, if 32-bit)
linux-x86-mmx Linux, x86 32-bit with MMX (for old computers)
linux-x86-any Linux, x86 32-bit (for truly ancient computers)
The FX 8150 does AVX and XOP, while my 1055T doesn't.
The community version has more options:
linux-x86-64-native Linux, x86-64 'native' (all CPU features you've got)
linux-x86-64-gpu Linux, x86-64 'native', CUDA and OpenCL (experimental)
linux-x86-64-opencl Linux, x86-64 'native', OpenCL (experimental)
linux-x86-64-cuda Linux, x86-64 'native', CUDA (experimental)
linux-x86-64-avx Linux, x86-64 with AVX (2011+ Intel CPUs)
linux-x86-64-xop Linux, x86-64 with AVX and XOP (2011+ AMD CPUs)
linux-x86-64[i] Linux, x86-64 with SSE2 (most common)
linux-x86-64-icc Linux, x86-64 compiled with icc
linux-x86-64-clang Linux, x86-64 compiled with clang
linux-x86-gpu Linux, x86 32-bit with SSE2, CUDA and OpenCL (experimental)
linux-x86-opencl Linux, x86 32-bit with SSE2 and OpenCL (experimental)
linux-x86-cuda Linux, x86 32-bit with SSE2 and CUDA (experimental)
linux-x86-sse2[i] Linux, x86 32-bit with SSE2 (most common, 32-bit)
linux-x86-native Linux, x86 32-bit, with all CPU features you've got (not necessarily best)
linux-x86-mmx Linux, x86 32-bit with MMX (for old computers)
linux-x86-any Linux, x86 32-bit (for truly ancient computers)
linux-x86-clang Linux, x86 32-bit with SSE2, compiled with clang
linux-alpha Linux, Alpha
linux-sparc Linux, SPARC 32-bit
linux-ppc32-altivec Linux, PowerPC w/AltiVec (best)
linux-ppc32 Linux, PowerPC 32-bit
linux-ppc64 Linux, PowerPC 64-bit
linux-ia64 Linux, IA-64
Subscribe to:
Posts (Atom)