447. Multiuser ECCE

A recent comment lead me to make this post:

[..] Can you advise a correct way of setting up ECCE so that everyone can have an individual account and run on the same machine?

It's a valid question. I haven't had any need for it since I'm running my own cluster, which I currently don't share with anyone. I also manage a multi-user cluster overseas, but I am the only one using ECCE, and so I'm fine running the ECCE server here.

Anyway. I built ECCE 6.4 as shown e.g. here and here.

Below I install the ECCE server as the user verahill, and then install the client software as the users lindqvst and me. In the former case I configure the server using a copied config_from_server script, and in the latter case I use config_with_server. Both methods worked fine. All users are on the same machine, but by editing apps/siteconfig/DataServers you can quite easily configure the ECCE client to connect to a remote server.

---

Server:

hostname

helium

sudo adduser verahill
sudo adduser lindqvst
./install_ecce.v6.4.csh

Main ECCE installation menu                                                                                                                  
===========================                                            1) Help on main menu options 
2) Prerequisite software check 
3) Full install  
4) Full upgrade  
5) Application software install  
6) Application software upgrade  
7) Server install  
8) Server upgrade 

IMPORTANT: If you are uncertain about any aspect of installing 
or running ECCE at your site, please refer to the detailed 
ECCE Installation and Administration Guide at 
http://ecce.pnl.gov/docs/installation/2864B-Installation.pdf 
                                                                                                                                             
Hit  at prompts to accept the default value in brackets.
Selection: [1] 7
Host name: [helium]
Server installation directory: [/home/verahill/ecce-v6.4/server]
Enter the path to the ECCE application software directory that  
will use this server (even if this directory is not accessible  
from the current machine) or [return] if you have not installed 
application software yet or don't want to update it: 
ECCE v6.4 will be installed using the settings:                                                                                              
 
  Installation type: [server install]      
  Host name: [helium]            
  Server installation directory: [/home/verahill/ecce-v6.4/server]  
 
Are these choices correct (yes/no/quit)? [yes]

ECCE installation succeeded.                                
                                                             
*************************************************************** 
!! You MUST perform the following steps in order to use ECCE !! 
-- Transfer the script:                                          
     /home/verahill/ecce-v6.4/server/ecce-admin/config_from_server 
   to the machine where the application software is installed.    
   Run this script as the same user as the application            
   software installation.                                         
   (This step can be skipped if you have not installed the        
   application software and the server name is specified during   
   that install)                                                  
                                                                  
-- Start the ECCE server as 'verahill' by running:                
     /home/verahill/ecce-v6.4/server/ecce-admin/start_ecce_server 
                                                                  
***************************************************************

cp /home/verahill/ecce-v6.4/server/ecce-admin/config_from_server ~
chmod ugo+r config_from_server

---

First client

su lindqvst
cd ~
cp /home/verahill/config_from_server .
cp /home/verahill/install_ecce.v6.4.csh .
./install_ecce.v6.4.csh

Main ECCE installation menu                                                                                                                  
===========================                                                                                                                  1) Help on main menu options 
2) Prerequisite software check 
3) Full install  
4) Full upgrade  
5) Application software install  
6) Application software upgrade  
7) Server install  
8) Server upgrade 

IMPORTANT: If you are uncertain about any aspect of installing 
or running ECCE at your site, please refer to the detailed 
ECCE Installation and Administration Guide at 
http://ecce.pnl.gov/docs/installation/2864B-Installation.pdf 
                                                                                                                                             
Hit  at prompts to accept the default value in brackets.
Selection: [1] 5
Host name: [helium]
Server installation directory: [/home/lindqvst/ecce-v6.4/apps]

ECCE installation succeeded.                                
                                                             
*************************************************************** 
!! You MUST perform the following steps in order to use ECCE !! 
-- Configure the application software to use the desired ECCE  
   server by running the script:                               
     /home/lindqvst/ecce-v6.4/apps/scripts/config_with_server 
   as the same user as the application software installation. 
   (This step can be skipped if you prefer to copy over and run 
   the config_from_server script created during the server 
   installation in the ecce-admin directory)              
                                                              
-- To register machines to run computational codes, please see 
   the installation and compute resource registration manuals 
   at http://ecce.pnl.gov/using/installguide.shtml          
                                                             
-- Before running ECCE each user must source an environment 
   setup script.  For csh/tcsh users add this to ~/.cshrc: 
     if ( -e /home/lindqvst/ecce-v6.4/apps/scripts/runtime_setup ) then 
       source /home/lindqvst/ecce-v6.4/apps/scripts/runtime_setup      
     endif                                                             
   For sh/bash users, add this to ~/.profile or ~/.bashrc:             
     if [ -e /home/lindqvst/ecce-v6.4/apps/scripts/runtime_setup.sh ]; then 
       . /home/lindqvst/ecce-v6.4/apps/scripts/runtime_setup.sh           
     fi                                                                 
***************************************************************

Next, configure:

./config_from_server

Enter the ECCE application software installation home directory: /home/lindqvst/ecce-v6.4/apps 
 
ECCE application software home directory is /home/lindqvst/ecce-v6.4/apps 
Is this correct? [yes] 
Adding data server URL for helium to siteconfig/DataServers
Adding URL for online help to siteconfig/site_runtime
Adding message server URL for helium to siteconfig/jndi.properti

And then do

echo 'export ECCE_HOME=/home/lindqvst/ecce-v6.4/apps' >> ~/.bashrc
echo 'export PATH=${ECCE_HOME}/scripts:${ECCE_HOME}/scripts/parsers:${PATH}' >> ~/.bashrc

---

Second client:
I set up a third user, installed ECCE as shown above for lindqvst, but instead of running config_from_server
I did:

/home/me/tmp/ecce-v6.4/ecce-v6.4/apps/scripts/config_with_server

Enter the full host name of the machine where the ECCE server
is installed: helium
ECCE server host name is helium 
Is this correct? [yes]

And then do

echo 'export ECCE_HOME=/home/me/ecce-v6.4/apps' >> ~/.bashrc
echo 'export PATH=${ECCE_HOME}/scripts:${ECCE_HOME}/scripts/parsers:${PATH}' >> ~/.bashrc
source ~/.bashrc

---

Tying it all together:

Start the server: As user verahill, start the server.

su verahill
/home/verahill/ecce-v6.4/server/ecce-admin/start_ecce_server

/home/verahill/ecce-v6.4/server/httpd/bin/apachectl start: httpd started
[1] 13951
INFO  BrokerService    - ActiveMQ 5.1.0 JMS Message Broker (localhost) is starting
INFO  BrokerService    - ActiveMQ JMS Message Broker (localhost, ID:helium-39358-1370944321599-0:0) started

ps a|grep 13951 

13951 pts/3    S      0:00 grep -v -e ACTIVEMQ -e Loading -e AMQ -e Kaha -e help -e Transport

You should be able to autostart ecce using something along the lines of the following in your /etc/rc.local:

su verahill '/home/verahill/ecce-v6.4/server/ecce-admin/start_ecce_server' &

As a user, start ecce:

su lindqvst
/home/lindqvst/ecce-v6.4/apps/scripts/ecce

Create a password

Create a job

Then log out and log in as another user, e.g. me

su me
ecce

me is in reality called something else

Note that as user me you can't see any of the files in lindqvst's folder -- nor access them without inputting the correct (ECCE) username and password.

And again, note that there are alternative ways of setting this up -- you could have everyone log in as the same linux user, but still retain different ECCE identities. I think what I show here is more in line with how most people would want to use ECCE though.

Also, note that you can quite easily be a client on a different computer too -- the key lies in editing the apps/siteconfig/DataServers file.

446. B3LYP and WAH -- the confusion

Quite a while back I was looking at the WAH (Wilson-Amos-Handy) functional, and while it turned out to be a bit more complicated than I had hoped, it led me to type up a brief discussion about b3lyp in different computational packages.

The issue is that there are several different definitions, and that even if a paper is kind enough to provide the Becke 1993 communication as a reference, this is rarely the actual form of b3lyp used. In fact, the LYP part would speak directly against it. As someone whom isn't well-versed in the computational and theoretical arts I do think it would be nice if we could get to the point where we can get useful information by doing point-and-click computations, but as exchange-correlation functionals essentially are fudge-factors, this probably won't happen for some time.  In other words, 6-31G/B3LYP may be a winning combination for the computation of electronic energies of a limited range of (mostly organic) small molecules in the gas phase, it doesn't always yield anything useful about real-world systems.

NOTE: I wrote the original text as I was trying to figure out what WAH was -- and I thought at that point that it was a simple form of B3PW91 with tweaked prefactors. It isn't -- it's requires changes in the way the GIAOs are computed (I think). So don't focus on the WAH discussion.

Anyway, here's the story as a bench chemist (i.e. not a computational or theoretical chemist) understands it, in the context of trying to understand what the WAH exchange-correlation functional looks like.

The WAH functional is a hybrid exchange correlation functional which was developed to provide accurate NMR shift calculations.

Note that the WAH functional is correctly implemented in PQS -- see the manual.

But the story is really about B3LYP...

---

1. Definition of the WAH exchange-correlation functional:
The definition[1] consists of

We found that by using hybrid Kohn-Sham orbitals and eigenvalues with an adjusted 'exact-exchange' coefficient C_x, the NMR shielding parameters gave an accuracy approaching the best coupled-cluster calculations for molecules containing first and second row atoms. For B3LYP[2] we find C_x=0.05[..] give(s) best values (note that the coefficient of Local Density Exchange is (1-C_x) in the amended B3LYP. We name the resulting NMR values B3LYP_GGA^0.05[..].

This is the paper which is cited by the PQS manual. The definition is brief, but not unreasonable.

---

2 Definition of B3LYP
[The first-person account of the background to B3LYP is found here: http://www.ccl.net/chemistry/resources/messages/2002/05/22.008-dir/]

In the paper which is cited as a source of B3LYP, Becke defined[2] a functional as

E_XC=E_XC^LSDA+a₀ (E_x^HF-E_x^LSDA )+a_xΔ E_x^Becke88+a_cΔ E_c^PW91   (eq 1)

where E_XC denotes exchange-correlation functional, E_x denotes exchange functional, E_c denotes correlation functional, Δ denotes non-local contribution, LSDA is the local spin-density approximation, Becke88[3] is the gradient-corrected LDA and PW91 is the Perdew-Wang 1991 gradient correction.[4]. The B3 in B3LYP refers to the three parameters it involves: a₀=0.2, a_x=0.72 and a_c=0.81. Note that a₀ is the same as C_x above. We'll refer to equation 1 as B3PW91.

LSDA is poorly defined but is normally taken to be the SVWN of the form

E_XC^LSDA=E_X^LSDA+E_C^LSDA  (eq 2)
=E_x^Slater+E_c^VWN   (eq 3)

although there's a slew of Vosko-Wilk-Nusair (VWN) functionals -- most sources suggest that Becke referred to VWN5, while my reading of the literature is a bit different (Becke states he uses the electron-gas parametrization in [4]). Either way, equation 1 now becomes

E_XC=a₀ E_x^HF+(1-a₀ )E_x^Slater+E_c^VWN +a_xΔ E_x^Becke88+a_cΔ E_c^PW91 (eq 4)

This is implemented as the hybrid exchange-correlation functional acm in NWChem (the adibatic connection method) using VWN5. A very brief summary of Becke '93 vs Gaussian '92 (don't ask me about the chronology) is also available by Stephens et al. in J. Phys. Chem. 1994, 98(45), p. 11624.

2.1 Gaussian '92
You may at this point be forgiven for asking yourself why it is called B3LYP and not B3PW91. In 1991 Gaussian hadn't yet implemented PW91 (fair enough) and substituted it with the Lee-Yang-Parr (LYP) correlation functional (ΔE_c^LYP). Since it's difficult to separate the local component (and we want the non-local component as indicated by Δ), they wrote

Δ E_c^LYP=E_c^LYP-E_c^VWN (eq 5)

which turns equation 4 into

E_XC=a₀ E_x^HF+(1-a₀)E_x^Slater+a_xΔ E_x^Becke88+a_cE_c^LYP +(1-a_c)E_c^VWN (eq 6)

In addition, in the original Gaussian implementation VWN_1_RPA was used, which sources tell me is 100\% wrong when taking Becke's intentions into account.

E_XC=a₀ E_x^HF+(1-a₀)E_x^Slater +a_xΔ E_x^Becke88+a_cE_c^LYP +(1-a_c)E_c^VWN_1_RPA (eq 7)

To make matters worse, today Gaussian uses VWN_3 and it seems they know how to get the nonlocal component of LYP directly (see below). Equation 7 is what you use if you use B3LYP in most software packages (though not all -- e.g. Gamess US uses VWN5 instead of VWN_1_RPA) So in G09 it's now

E_XC=a₀ E_x^HF+(1-a₀)E_x^Slater+a_xΔ E_x^Becke88+E_c^VWN_3+a_cE_c^LYP (eq 8)

2.2 PQS
PQS uses the old gaussian version (eq. 7) as the b3lyp functional, but it doesn't explicitly state which form -- b3lyp or b3pw91 -- is used for the WAH functional.

---

3 So what definition did WAH use?
All we really care about is reproducing the original paper by Handy et al. -- not whether Becke would approve or not. But here's where the problem of citing papers you may not have read becomes an issue.

Wilson, Amos and Handy cite the 1993 paper by Becke which defines the canonical version of B3LYP (i.e. B3PW91), which should settle it in favour of WAH being defined as shown in equation 4.

However, they used CADPACK, which implements it as in equation 7. Reading the CADPACK manual I can see what Handy et al. probably did: the way you define custom parameters for hybrid functionals in CADPACK is by doing

hybrid a0 ax ac

so that they during the development of their functional most likely typed in

b3lyp 0.05 0.72 0.81

which meant they probably used

E_XC=0.05 E_x^HF+0.95 E_x^Slater+ +0.72 Δ E_x^Becke88+0.81 E_c^LYP +0.19 E_c^VWN_1_RPA (eq 9)

---

4 Implementing it in your package of choice
[NOTE: this will NOT set up WAH correctly -- I'm leaving it as it shows how to set up custom XCs in nwchem, G09 and Dalton]

4.1 NWCHEM
The canonical version of Becke's functional, B3PW91, is implemented as acm in NWChem and which is manually entered as

xc HFexch 0.2 slater 0.8 becke88 nonlocal 0.72 vwn_5 1 Perdew91 0.81

while the Gaussian '92 form is manually entered as

xc HFexch 0.2 slater 0.8 becke88 nonlocal 0.72 vwn_1_rpa 0.19 lyp 0.81

This means that the two possible forms of WAH are:

xc HFexch 0.05 slater 0.95 becke88 nonlocal 0.72 vwn_5 1 Perdew91 0.81

and

 xc HFexch 0.05 slater 0.95 becke88 nonlocal 0.72 vwn_1_rpa 0.19 lyp 0.81

We'll refer to them as B3PW91_0.05 and B3LYP_0.05, respectively.

4.2 Gaussian 09
Gaussian is a lot less elegant. The canonical version of Becke's functional, B3PW91, is implemented as acm in Gaussian as B3PW91, which is manually entered as

BPW91 IOp(3/76=1000002000) IOp(3/77=0720008000) IOp(3/78=0810010000)

while the old Gaussian form is manually entered as

BLYP IOp(3/76=1000002000) IOp(3/77=0720008000) IOp(3/78=0810001900

This means that the two forms of WAH are:

BPW91 IOp(3/76=1000000500) IOp(3/77=0720009500) IOp(3/78=0810010000)

and

BLYP IOp(3/76=1000000500) IOp(3/77=0720009500) IOp(3/78=0810001900)

We'll refer to them as B3PW91_0.05 and B3LYP_0.05, respectively.

4.3 Dalton
The notations are (more or less)

Combine HF=0.20 Slater=0.80 Becke=0.72 PW91c=0.81 VWN5=1
Combine HF=0.20 Slater=0.80 Becke=0.72 LYP=0.81 VWN=0.19
Combine HF=0.05 Slater=0.95 Becke=0.72 PW91c=0.81 VWN5=1
Combine HF=0.05 Slater=0.95 Becke=0.72 LYP=0.81 VWN=0.19

As an aside, I don't think anyone at this point would be surprised to learn that B3PW91 in Dalton and Gaussian are two completely different exchange-correlation functionals...

---

Performance
NOTE: the results don't quite make any sense to me anymore -- using the same XCs and basis sets and structures one would expect to get the exact same results for all three packages. I don't know why that wasn't the case, assuming that I implemented the XCs correctly, and assuming that the basis sets really are the same (which often they actually aren't...). So keep that in mind.

Original:
I used the same equilibrium structures as Handy (i.e. those of Cybulski), and used the aug-cc-pVTZ basis set with a fine DFT grid (not the same as Handy -- but should be as good. Also, I've done this with def2-qzvp and 6-31+g* as well). All calculations are for the gas phase. Handy's values are for the Huzinaga IV basis set in their GIAO paper[5], but it doesn't really matter much which basis set is chosen. The results are tabulated in table 1.

Note that Handy also saw chemical shifts for the oxygen in carbon monoxide which were around -80 ppm. Basically, I can reproduce everything except for his B3LYP^0.05_gga. I did a few test-runs with Huz-IV in Dalton and it was just as bad as the other methods.

Table 1: Tabulated calculated gas phase NMR shifts using different combinations of exchange-correlation functionals and the aug-cc-pvtz basis set. Note that Handy's experimental data are not in agreement with my sources, which are 1.0, -42.3 and 344 ppm for CO, CO and H₂O, respectively.

Package b3lypa gb3lypb acmc b3pw91b b3lyp0.05 b3pw910.05 Handy Expt. Carbon in CO (ppm) NWChem -10.32 -10.32 -8.80 -8.80 -8.08 -6.55 5.6 2.8 G09 -7.71 -7.71 -9.95 -9.95 -5.66 -7.74 Dalton -10.21 -10.21 N/A -12.65 -7.97 -10.38 Oxygen in CO (ppm) NWChem -71.51 -71.51 -72.34 -72.34 -70.06 -70.93 -40 -36.7 G09 -78.82 -78.82 -77.89 -77.89 -77.57 -76.86 Dalton -71.50 -71.50 N/A -72.03 -70.04 -70.89 Oxygen in water NWChem 328.45 328.45 329.28 329.28 329.79 330.48 327 358 G09 N/A 328.76 328.39 328.39 328.95 328.31 Dalton 328.50 328.50 N/A 328.02 329.85 329.07 aNote that G09 uses eq. 8 by default (gives -11.70, -77.63, and 328.34 ppm), but I forced it to use eq. 7. b These were my manual implementations of 'b3lyp' and 'acm' to make sure I got things right. c This is acm in NWChem and B3PW91 in Gaussian 09.

---

References:
[1] P. J. Wilson, R. D. Amos, H. N. C., Chem. Phys. Letters 1999, 321, 475-484.
[2] A. D. Becke, J. Chem. Phys. 1993, 98, 5648-5652.
[3] A. D. Becke, Phys. Rev. A 1988, 38, 3098-3100.
[4] J. P. Perdew, Y. Wang, Phys. Rev. B 1991, 45, 13244-13249.
[5] T. Helgaker, P. J. Wilson, R. D. Amos, N. C. Handy, J. Chem. Phys. 2000, 113, 2983-2989.

445. GAMESS US 2013 R1 on Debian (Wheezy) -- w/o GPU

Update 27/6/2013:
Please note that Kirill Berezovsky has published a series of posts on GAMESS US, including how to compile it for both CPU and GPU use. See
http://biochemicalmatters.blogspot.com.au/2013/06/gamess-us-frequently-asked-questions_26.html
http://biochemicalmatters.blogspot.ru/2013/06/gamess-us-frequently-asked-questions_1687.html
http://biochemicalmatters.blogspot.ru/2013/06/gamess-us-frequently-asked-questions_1447.html
http://biochemicalmatters.blogspot.com.au/2013/06/gamess-us-frequently-asked-questions.html


Original post:
A new version of GAMESS is out now (2013 R1): https://groups.google.com/forum/?fromgroups#!topic/gamess-announce/8j1esKifzEo

GPU support will be a later post.

---

0. Install a math library
You can use e.g. acml or atlas. See http://verahill.blogspot.com.au/2013/05/422-set-up-acml-on-linux.html for acml (which I've only had luck with on AMD machines).

To get the debian ATLAS libs do

sudo apt-get install libatlas3-base libatlas-dev

If you want to compile your own ATLAS libs, see e.g. http://verahill.blogspot.com.au/2012/09/rocks-543-atlas-and-gromacs-on-xeon.html or http://verahill.blogspot.com.au/2012/09/compile-atlas-gromacs-nwchem-on-amd-fx.html

You can also link to openblas, as shown in this post: http://verahill.blogspot.com.au/2012/09/compiling-and-testing-gamess-us-on.html

---

1. Get GAMESS US
Go to http://www.msg.chem.iastate.edu/GAMESS/download/register/
Check the tick boxes next to the architectures you intend to use GAMESS on. Fill out your email address, and hit Submit. You'll receive an email with instructions almost immediately.

The email will contain a URL to an archive with the source code, and a password for downloading it.
I'll presume that you downloaded the file, gamess-current.tar.gz, to ~/Downloads.

---

2. Untar and prepare
I'm presuming that you don't already have any /opt/gamess or /opt/gamess/gamess directories. In my earlier write-ups (e.g. this) I put everything in /opt/gamess, which isn't a good long-term strategy since you often want to keep earlier versions of computational software alongside newer ones.

sudo apt-get install build-essential gfortran openmpi-bin libopenmpi-dev libboost-all-dev
sudo mkdir /opt/gamess -p
sudo chown $USER:$USER /opt/gamess
cd /opt/gamess
cp ~/Downloads/gamess-current.tar.gz gamess-2013r1.tar.gz
tar xvf gamess-2013r1.tar.gz
mv gamess gamess-2013r1
cd gamess-2013r1/

---

3. Configure
NOTE: Even if you may have gfortran 4.7 (e.g. Wheezy) you should give the version as 4.6 during configure (see below).

./config

please enter your target machine name: linux64
GAMESS directory? [/opt/gamess/gamess-2013r1] 
GAMESS build directory? [/opt/gamess/gamess-2013r1]
Version? [00] 13
Please enter your choice of FORTRAN: gfortran
Please enter only the first decimal place, such as 4.1 or 4.6:  
4.6

ACML:

Enter your choice of 'mkl' or 'atlas' or 'acml' or 'none': acml
enter this full pathname: /opt/acml/acml5.3.1
Math library 'acml' will be taken from /opt/acml/acml5.3.1/gfortran64_int64/lib

Atlas:
I've compiled ATLAS myself in the past, but the libs have not worked with all programs. Here we use the debian libs instead.

Enter your choice of 'mkl' or 'atlas' or 'acml' or 'none': atlas
Please enter the Atlas subdirectory on your system: /usr/lib/atlas-base
Math library 'atlas' will be taken from /usr/lib/atlas-base

mpi:

communication library ('sockets' or 'mpi')? mpi
Enter MPI library (impi, mvapich2, mpt, sockets): openmpi
Please enter your openmpi's location: /usr/lib/openmpi

---

4. Build
Edit comp and change

1663 #           -fno-whole-file suppresses argument's data type checking
1664       set OPT='-O2'
1665       if (".$GMS_DEBUG_FLAGS" != .) set OPT="$GMS_DEBUG_FLAGS"

to

1663 #           -fno-whole-file suppresses argument's data type checking
1664       set OPT='-O0'
1665       if (".$GMS_DEBUG_FLAGS" != .) set OPT="$GMS_DEBUG_FLAGS"

or exam44.inp will fail.

cd ddi/
./compddi
cd ../
./compall

If you are building with openmpi, edit lked and change

958             case openmpi:
 959                set MPILIBS="-L$GMS_MPI_PATH/lib64"
 960                set MPILIBS="$MPILIBS -lmpi"
 961                breaksw

to

958             case openmpi:
 959                set MPILIBS="-L$GMS_MPI_PATH/lib"
 960                set MPILIBS="$MPILIBS -lmpi"
 961                breaksw

Make the proper symlinks if you are using ATLAS:

sudo ln -s /usr/lib/atlas-base/libatlas.so.3 /usr/lib/atlas-base/libatlas.so
sudo ln -s /usr/lib/atlas-base/libf77blas.so.3 /usr/lib/atlas-base/libf77blas.so

./lked gamess 13r1 

---

I use my own script called gmrun:

#!/bin/csh
set TARGET=mpi
set SCR=$HOME/scratch
set USERSCR=/scratch
set GMSPATH=/opt/gamess/gamess-2013r1
set JOB=$1
set VERNO=$2
set NCPUS=$3

if ( $JOB:r.inp == $JOB ) set JOB=$JOB:r
echo "Copying input file $JOB.inp to your run's scratch directory..."
cp $JOB.inp $SCR/$JOB.F05

setenv TRAJECT $USERSCR/$JOB.trj
setenv RESTART $USERSCR/$JOB.rst
setenv INPUT $SCR/$JOB.F05
setenv PUNCH $USERSCR/$JOB.dat
if ( -e $TRAJECT ) rm $TRAJECT
if ( -e  $PUNCH ) rm $PUNCH
if ( -e  $RESTART ) rm $RESTART
source $GMSPATH/gms-files.csh

setenv LD_LIBRARY_PATH /usr/lib/openmpi/lib:$LD_LIBRARY_PATH
mpiexec -n $NCPUS $GMSPATH/gamess.$VERNO.x|tee $JOB.out
cp $PUNCH .

To run, do e.g.

gmrun exam01.inp 13r1 2