Please note that Kirill Berezovsky has published a series of posts on GAMESS US, including how to compile it for both CPU and GPU use. See
http://biochemicalmatters.blogspot.com.au/2013/06/gamess-us-frequently-asked-questions_26.html
http://biochemicalmatters.blogspot.ru/2013/06/gamess-us-frequently-asked-questions_1687.html
http://biochemicalmatters.blogspot.ru/2013/06/gamess-us-frequently-asked-questions_1447.html
http://biochemicalmatters.blogspot.com.au/2013/06/gamess-us-frequently-asked-questions.html
Original post:
A new version of GAMESS is out now (2013 R1): https://groups.google.com/forum/?fromgroups#!topic/gamess-announce/8j1esKifzEo
GPU support will be a later post.
0. Install a math library
You can use e.g. acml or atlas. See http://verahill.blogspot.com.au/2013/05/422-set-up-acml-on-linux.html for acml (which I've only had luck with on AMD machines).
To get the debian ATLAS libs do
sudo apt-get install libatlas3-base libatlas-dev
If you want to compile your own ATLAS libs, see e.g. http://verahill.blogspot.com.au/2012/09/rocks-543-atlas-and-gromacs-on-xeon.html or http://verahill.blogspot.com.au/2012/09/compile-atlas-gromacs-nwchem-on-amd-fx.html
You can also link to openblas, as shown in this post: http://verahill.blogspot.com.au/2012/09/compiling-and-testing-gamess-us-on.html
1. Get GAMESS US
Go to http://www.msg.chem.iastate.edu/GAMESS/download/register/
Check the tick boxes next to the architectures you intend to use GAMESS on. Fill out your email address, and hit Submit. You'll receive an email with instructions almost immediately.
The email will contain a URL to an archive with the source code, and a password for downloading it.
I'll presume that you downloaded the file, gamess-current.tar.gz, to ~/Downloads.
2. Untar and prepare
I'm presuming that you don't already have any /opt/gamess or /opt/gamess/gamess directories. In my earlier write-ups (e.g. this) I put everything in /opt/gamess, which isn't a good long-term strategy since you often want to keep earlier versions of computational software alongside newer ones.
sudo apt-get install build-essential gfortran openmpi-bin libopenmpi-dev libboost-all-dev sudo mkdir /opt/gamess -p sudo chown $USER:$USER /opt/gamess cd /opt/gamess cp ~/Downloads/gamess-current.tar.gz gamess-2013r1.tar.gz tar xvf gamess-2013r1.tar.gz mv gamess gamess-2013r1 cd gamess-2013r1/
3. Configure
NOTE: Even if you may have gfortran 4.7 (e.g. Wheezy) you should give the version as 4.6 during configure (see below).
./configACML:please enter your target machine name: linux64 GAMESS directory? [/opt/gamess/gamess-2013r1] GAMESS build directory? [/opt/gamess/gamess-2013r1] Version? [00] 13 Please enter your choice of FORTRAN: gfortran Please enter only the first decimal place, such as 4.1 or 4.6: 4.6
Atlas:Enter your choice of 'mkl' or 'atlas' or 'acml' or 'none': acml enter this full pathname: /opt/acml/acml5.3.1 Math library 'acml' will be taken from /opt/acml/acml5.3.1/gfortran64_int64/lib
I've compiled ATLAS myself in the past, but the libs have not worked with all programs. Here we use the debian libs instead.
mpi:Enter your choice of 'mkl' or 'atlas' or 'acml' or 'none': atlas Please enter the Atlas subdirectory on your system: /usr/lib/atlas-base Math library 'atlas' will be taken from /usr/lib/atlas-base
communication library ('sockets' or 'mpi')? mpi Enter MPI library (impi, mvapich2, mpt, sockets): openmpi Please enter your openmpi's location: /usr/lib/openmpi
4. Build
Edit comp and change
to1663 # -fno-whole-file suppresses argument's data type checking 1664 set OPT='-O2' 1665 if (".$GMS_DEBUG_FLAGS" != .) set OPT="$GMS_DEBUG_FLAGS"
or exam44.inp will fail.1663 # -fno-whole-file suppresses argument's data type checking 1664 set OPT='-O0' 1665 if (".$GMS_DEBUG_FLAGS" != .) set OPT="$GMS_DEBUG_FLAGS"
cd ddi/ ./compddi cd ../ ./compall
If you are building with openmpi, edit lked and change
to958 case openmpi: 959 set MPILIBS="-L$GMS_MPI_PATH/lib64" 960 set MPILIBS="$MPILIBS -lmpi" 961 breaksw
Make the proper symlinks if you are using ATLAS:958 case openmpi: 959 set MPILIBS="-L$GMS_MPI_PATH/lib" 960 set MPILIBS="$MPILIBS -lmpi" 961 breaksw
sudo ln -s /usr/lib/atlas-base/libatlas.so.3 /usr/lib/atlas-base/libatlas.so sudo ln -s /usr/lib/atlas-base/libf77blas.so.3 /usr/lib/atlas-base/libf77blas.so
./lked gamess 13r1
I use my own script called gmrun:
#!/bin/csh set TARGET=mpi set SCR=$HOME/scratch set USERSCR=/scratch set GMSPATH=/opt/gamess/gamess-2013r1 set JOB=$1 set VERNO=$2 set NCPUS=$3 if ( $JOB:r.inp == $JOB ) set JOB=$JOB:r echo "Copying input file $JOB.inp to your run's scratch directory..." cp $JOB.inp $SCR/$JOB.F05 setenv TRAJECT $USERSCR/$JOB.trj setenv RESTART $USERSCR/$JOB.rst setenv INPUT $SCR/$JOB.F05 setenv PUNCH $USERSCR/$JOB.dat if ( -e $TRAJECT ) rm $TRAJECT if ( -e $PUNCH ) rm $PUNCH if ( -e $RESTART ) rm $RESTART source $GMSPATH/gms-files.csh setenv LD_LIBRARY_PATH /usr/lib/openmpi/lib:$LD_LIBRARY_PATH mpiexec -n $NCPUS $GMSPATH/gamess.$VERNO.x|tee $JOB.out cp $PUNCH .
To run, do e.g.
gmrun exam01.inp 13r1 2
Thank you for the link (and the series of posts!).
ReplyDeleteI'll try following the instructions next week to see if I have any more luck with libcchem.
I added a comment to http://biochemicalmatters.blogspot.com.au/2013/06/gamess-us-frequently-asked-questions_26.htm pointing out potential issues with gfortran 4.7, the optimization level, and test job exam44.inp.
I'd be interested in seeing the GUI. To date I'm only aware of http://www.chemsoft.ch/qc/gameix.htm
Although to be fair my 'dream' would be for ECCE to support GAMESS US -- ECCE handles job preparation, submission, monitoring, and everything in between. Unfortunately, it
1) 'only' handles g03, g98, nwchem, amica and gamess UK
and
2) the scripts that prepares the input and reads the output from different software packages are written in Perl, and while I find regex as compact and elegant as the next person, I have a hard time reading and understanding Perl.
Hi I was tested the performances of GAMESS-US using various compilers(ifort, gfortran, pgfortran). but the OPEN64 compiler was not tested because no support of GAMESS-US.
ReplyDeleteThe OPEN64 compiler was best performance about 100% for Gaussian03 at the AMD CPU computers in my experiences.
Do you have a plan to compile the GAMESS-US with OPEN64 compiler ?
I am now editing the compile scripts and testing the result having a lot of problems.
Hi I am thinking GAMESS-US compiling with OPEN64 compiler for best performance at AMD CPUs.
ReplyDeleteDo you have a plan of GAMESS compiling with OPEN64 compiler ?
I don't have any plans on compiling with open64. I used open64 a few years back to try to compile nwchem, and had little luck. If you do sort it out and write it up, feel free to post a link here.
DeleteHi,
ReplyDeleteThank you so much for all these topics. I am very-very basic with linux command since I am a pure chemist... but trying to develop calculations.
Thanks to you, I was able to compile gamess with opempi and atlas. This topic work on ubuntu 13.10. My machine is a HP z600 (2x4 cores, hypertreaded --> 16 cores with 24Gb) and calculations work fine. I have two similr machines, each contains a compiled version according to your topic.
But now I want to go further and I am trying to built cluster using kestrelHPC (the only one where a full topic help me to buit one) but in this case only ubuntu lucid works or debian 6.0.x.. I install it on a third computer as frontend machine.
I choose lucid in which the gfortran version is 4.4. Almost everything is ok during compilation. Kestrehpc is correctly installed : ganglia returns me 2 nodes (32processes and 48Gb).
I have 2 problems for gamess compilation and 1 question about cluster:
1.1 problem1
the symlink you put in the topic doesn't work (file not found). Effectively, I looked at the directory and no such file, even not with similar name (libatlas.so.3 and libf77blas.so.3). Do I need to symlink something ?
1.2. problem 2
in your gmrun, i seem that the line :
setenv LD_LIBRARY_PATH /usr/lib/openmpi/lib:$LD_LIBRARY_PATH
is not correct. message is that the variable LD_LIBRARY_PATH is not set and program cannot find library. Which library ? altas or mpi ? the folder /usr/lib/openmpi/lib exist on my ubuntu lucid.
2. question : about cluster and compilation
do I need to specify somewhere the presence of nodes or if I run directly with 32 processes (ie gmrun test.inp 13r1 32 >test.out), the gamess/opempi will automatically send to nodes ?
Thank you in advance for your help/comments. if you need to contact me, use phaubert2@wanadoo.fr to email me.
sincerly,
ph
1.1 Are you using ATLAS? Have you installed ATLAS? Note also that this was written for debian -- canonical has a way of doing weird things with libraries for no discernable reason e.g. ECCE won't compile on the latest versions of ubuntu but works just fine on debian and fedora derivatives.
ReplyDeleteEither way, to figure out where e.g. libatlas.so.3 is do
locate libatlas|grep so
1.2 The line is correct, and that's for openmpi. Again, make sure you've install openmpi and that the location is the same under ubuntu as under debian. Do e.g.
locate libmpi.so
If LD_LIBRARY_PATH isn't set you can try shortening the line to
setenv LD_LIBRARY_PATH /usr/lib/openmpi/lib
2. I'm not sure whether gamess is node-aware. I've basically never done any production work with it. However, you can set up openmpi to split jobs across nodes as it sees fit. See e.g. http://www.open-mpi.org/faq/?category=running#mpirun-scheduling
Hope that it helps
I should add that splitting a job across nodes often incurs a heavy performance penalty, depending on how they are interconnected (e.g. inifinibad vs gigabit ethernet and whether you use e.g. NFS) so it might not be worth the hassle.
Delete