19 October 2012

260. Compiling Dalton 2.0 QM on Debian -- in progress

UPDATE: I've since successfully compiled Dalton 2011 and lsdalton on both debian and CentOS/ROCKS:
http://verahill.blogspot.com.au/2012/11/compiling-lsdalton-on-debian.html
http://verahill.blogspot.com.au/2012/11/compiling-dalton-2011-on-debian.html
http://verahill.blogspot.com.au/2012/11/compiling-dalton-2011-on-rocks-543centos.html

I will revisit this post at a later date.

NOTE:  This is an update of an old post in which the compilation of Dalton was described, but I couldn't get the finished binary to run. Seems the main problem was that I don't know how to use dalton properly. I've got enough data now to show that dalton is working in serial mode though.

1. Sign up for a license.

2. Once you get approved you'll get an email with a link, a username and a password.
Download the cam-enabled package.

3.
cd ~/tmp
mv ~/Download/dalton-2.0-ca.tar.gz .
tar xvf dalton-2.0-cam.tar.gz
cd dalton-2.0-cam/
sudo mkdir /opt/dalton
sudo chown $USER:$USER /opt/dalton
mkdir /opt/dalton/basis
mkdir /opt/dalton/bin

Also, you may want to compile the netlib blas and lapack, and set them up. See here: http://verahill.blogspot.com.au/2012/09/compiling-netlibs-lapack-and-blas-on.html

Once you're done with that, start running configure:
./configure
------------------------------------------------------------------
   Configuring the DALTON Makefile.config and "dalton" run script
------------------------------------------------------------------

No architecture specified, attempting auto-configuration:
This appears to be a -linux architecture. Is this correct?  [y/n] y

-->Installing DALTON on a -linux computer

Do you want to install the program in a parallel MPI version? [y/n] y

-->WARNING: Makefiles for MPI architecture are difficult to guess
   Please compare the generated Makefile.config with local documentation.

   Checking for Fortran compiler ...
   from this list: mpif90 mpif77 ifort ifc efc pgf77 gfortran g77 

Compiler mpif90 found, use this compiler? [y/n] y

-->Compiler mpif90 found and accepted.

   Checking for C compiler ...
   from this list: mpicc  icc ecc pgcc gcc 

Compiler mpicc found, use this compiler? [y/n] y

-->Compiler mpicc found and accepted.

Testing existence of libraries in this order:
 libacml.a libmkl.so libmkl_p3.a libatlas.a libblas.a
Directory search list for libraries:
  /opt/gaussian/g03/ATLAS/compiled/lib /opt/gaussian/g03/ATLAS/build/lib /opt/ATLAS/lib /home/me/tmp/ATLAS/build/lib /lib /usr/local/lib /usr/lib /usr/local/lib/ATLAS /lib64 /usr/lib64 /usr/local/lib64 

Found /opt/gaussian/g03/ATLAS/compiled/lib/libatlas.a, use it ? [y/n] n
Found /opt/gaussian/g03/ATLAS/build/lib/libatlas.a, use it ? [y/n] n
Found /opt/ATLAS/lib/libatlas.a, use it ? [y/n] y

-->The following mathematical library(ies) will be used:
   -L/opt/ATLAS/lib -llapack -lf77blas -latlas

Does your file system allow files bigger than 2 Gb? [y/n] y

DALTON uses almost 7 million *8 words (56 Megabytes) of static
allocations, in addition to the dynamic allocation.

DALTON has the possibility to reserve an amount of static memory
for storing two-electron integrals in direct and parallel calculations
Storing some or all of the 2-el. integrals in memory will speed up
direct and parallel calculations (and in particular the latter).
NOTE: This will increase the static memory allocation used by DALTON

Would you like to activate the possibility of storing 2-el.int. in memory? [y/n] y
How many millions of *8 words to use for storing 2-el.integrals? 1000

-->Program will be installed with 1000000000 words used for storing 2-el.integrals

Maximum amount of work memory for dynamic allocations can be
changed at run time with the environment variable WRKMEM
or by using the -M option to the run script: "dalton -M mb ...".
We recommend at least 20 million *8 words work memory,
larger for correlated calculations, but it should for maximum
efficiency NOT exceed available physical memory.

How many millions of *8 words to use as default for work memory? 1000

-->Program will be installed with a default work memory of 1000000000 words

-->Current directory is /home/me/tmp/dalton-2.0-cam

Use ./bin as installation directory for binaries and scripts? [y/n] n
Please enter another installation directory: /opt/dalton/bin

-->DALTON executable and script will be placed in /opt/dalton/bin directory


-->Default basis set directory will be /home/me/tmp/dalton-2.0-cam/basis/


Use this directory as default basis set directory? [y/n] n

Please choose another default basis set directory (must end with /) /opt/dalton/basis/

-->Default basis set directory will be /opt/dalton/basis/


-->Job specific directories under $SCRATCH/$USER
-->will be used for temporary files when running DALTON

Use SCRATCH=/work as default root scratch space in "dalton" run script? [y/n] n
Please enter default root scratch directory: /scratch

-->Creating Makefile.config ...
Backup: renaming Makefile.config to Makefile.config.0
Is backend compiler gfortran ? [y/n] y
gfortran version 471 prc=x86_64
Proper 64-bit file access detected.

-->Creating the DALTON run-script in /opt/dalton/bin

   The configuration of DALTON has finished succesfully.
   Check compiler flags etc. in Makefile.config and run "make" to get executable.


So far so good. Ignore the math libs above.

The Makefile.config looks like this now:

ARCH        = linux
#
#
CPPFLAGS      = -DVAR_GFORTRAN -DSYS_LINUX -DVAR_MFDS -D'INSTALL_WRKMEM=1000000000' -D'INSTALL_MMWORK=1000000000' -D_FILE_OFFSET_BITS=64 -DVAR_MPI -DGFORTRAN=471 -DIMPLICIT_NONE -DINSTALL_BASDIR='"/opt/dalton/basis/"'
F77           = mpif90
CC            = mpicc
LOADER        = mpif90
RM            = rm -f
FFLAGS        = -march=native -O1 -ffast-math -funroll-loops -ftree-vectorize -fno-range-check -fno-whole-file
SAFEFFLAGS    = -march=native -O1 -ffast-math -funroll-loops -ftree-vectorize -fno-range-check -fno-whole-file
CFLAGS        = -march=native -O1 -ffast-math -funroll-loops -ftree-vectorize -std=c99 -DRESTRICT=restrict -DFUNDERSCORE=1
INCLUDES      = -I../include 
LIBS          = -L/opt/ATLAS/lib -llapack -llapack -lf77blas -latlas
INSTALLDIR    = /opt/dalton/bin
PDPACK_EXTRAS = linpack.o eispack.o gp_zlapack.o
GP_EXTRAS     = 
AR            = ar
ARFLAGS       = rvs
# flags for ftnchek on Dalton /hjaaj
CHEKFLAGS  = -nopure -nopretty -nocommon -nousage -noarray -notruncation -quiet  -noargumants -arguments=number  -usage=var-unitialized
# -usage=var-unitialized:arg-const-modified:arg-alias
# -usage=var-unitialized:var-set-unused:arg-unused:arg-const-modified:arg-alias
#
default : linuxparallel.x
SAFE2_FFLAGS = $(FFLAGS)
#
# Parallel initialization
#
MPI_INCLUDE_DIR = 
MPI_LIB_PATH    = 
MPI_LIB         = 
#
#
# Suffix rules
# hjaaj Oct 04: .g is a "cheat" suffix, for debugging.
#               'make x.g' will create x.o from x.F or x.c with -g debug flag set.
#
.SUFFIXES : .F .c .o .i .g .s

.F.o:
        $(F77) $(INCLUDES) $(CPPFLAGS) $(FFLAGS) -c $*.F 

.F.i:
        $(F77) $(INCLUDES) $(CPPFLAGS) -E $*.F > $*.i

.F.g:
        $(F77) $(INCLUDES) $(CPPFLAGS) $(FFLAGS) -g -c $*.F 

.F.s:
        $(F77) $(INCLUDES) $(CPPFLAGS) $(FFLAGS) -S -g -c $*.F 

.c.o:
        $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -c $*.c 

.c.i:
        $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -E $*.c > $.i

.c.g:
        $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -g -c $*.c 

.c.s:
        $(CC) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) -S -g -c $*.c 


Start the compilation by
make -f Makefile.in


Note that
LIBS   =-L/opt/acml5.1.0/gfortran64/lib -lacml
also works, while I couldn't get openblas or the debian lapack/blas libs to work.

If all goes well:
...
a - amfi.o
a - symtra.o
make[1]: Leaving directory `/home/me/tmp/dalton-2.0-cam/amfi'
---------------> Linking sequential dalton.x ...
mpif90 -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -fno-range-check -fno-whole-file \
        -o /opt/dalton/bin/dalton.x abacus/dalton.o cc/crayio.o abacus/linux_mem_allo.o \
        abacus/herpar.o eri/eri2par.o amfi/amfi.o amfi/symtra.o gp/mpi_dummy.o -Labacus -labacus -Lrsp -lrsp -Lsirius -lsirius -labacus -Leri -leri -Ldensfit -ldensfit -Lcc  -lcc -Ldft -ldft -Lgp -lgp -Lpdpack -lpdpack -L/opt/ATLAS/lib -llapack -lf77blas -latlas 
---------------> Linking parallel dalpar.x ...
mpif90 -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -fno-range-check -fno-whole-file \
        -o /opt/dalton/bin/dalpar.x abacus/dalton.o cc/crayio.o abacus/linux_mem_allo.o \
        abacus/herpar.o eri/eri2par.o amfi/amfi.o amfi/symtra.o -Labacus -labacus -Lrsp -lrsp -Lsirius -lsirius -labacus -Leri -leri -Ldensfit -ldensfit -Lcc  -lcc -Ldft -ldft -Lgp -lgp -Lpdpack -lpdpack -L/opt/ATLAS/lib -llapack -lf77blas -latlas  \
        -L/usr/lin/openmpi/lib -lmpi 



Copy the basis sets:
cp basis/* /opt/dalton/basis/

Throw
export PATH=$PATH:/opt/dalton/bin
in your ~/.bashrc and source it (source ~/.bashrc)

You're now ready to see how well it went.

cd test/
./dft_b3lyp_cart
mv dft_b3lyp_cart.dal DALTON.INP && mv dft_b3lyp_cart.mol MOLECULE.INP
mpirun -n 1 dalpar.x DALTON.INP MOLECULE.INP > DALTON.OUT

Looking at the output in DALTON.OUT:

 ***************************************************************************
 ************************ FINAL RESULTS FROM ABACUS ************************
 ***************************************************************************


 
     Date and time (Linux)  : Fri Oct 19 09:55:42 2012
     Host name              : beryllium                               



                         Molecular geometry (au)
                         -----------------------

 O          0.0000000000            0.0000000000           -0.1231063545
 H1   1     0.0000000000            1.4300000000            0.9768936455
 H1   2     0.0000000000           -1.4300000000            0.9768936455





                    Molecular wave function and energy
                    ----------------------------------

     Spin multiplicity  1     State number       1     Total charge       0


     Total energy        -73.9502446565 au (Hartrees)
                         -2012.28846267 eV
                           -194156.3395 kJ/mol




                         Relativistic corrections
                         ------------------------

     Darwin correction:                          0.1095380827 au
     Mass-velocity correction:                  -0.1391457352 au

     Total relativistic correction:             -0.0296076525 au (0.0400%)
     Non-relativistic + relativistic energy:   -73.9798523089 au




                              Dipole moment
                              -------------

                    0.783423 au           1.991263 Debye




                         Dipole moment components
                         ------------------------

                               au             Debye

                    z      0.78342341      1.99126348

                        1 a.u. =   2.54175 Debye 





   Interatomic separations (in Angstroms):
   ---------------------------------------

            O           H1   1      H1   2
            ------      ------      ------
 O     :    0.000000
 H1   1:    0.954707    0.000000
 H1   2:    0.954707    1.513447    0.000000


  Max interatomic separation is    1.5134 Angstroms
  between atoms "H1   2" and "H1   1".


  Bond distances (angstroms):
  ---------------------------

                  atom 1     atom 2       distance
                  ------     ------       --------
  bond distance:  H1   1     O            0.954707
  bond distance:  H1   2     O            0.954707


  Bond angles (degrees):
  ----------------------

                  atom 1     atom 2     atom 3         angle
                  ------     ------     ------         -----
  bond angle:     H1   1     O          H1   2       104.863




 CPU time statistics for ABACUS
 ------------------------------





 >>>> Total CPU  time used in ABACUS:   0.01 seconds
 >>>> Total wall time used in ABACUS:   0.00 seconds

- End of Static Property Section

 >>>> Total CPU  time used in DALTON:   1.56 seconds
 >>>> Total wall time used in DALTON:   0.00 seconds

 
     Date and time (Linux)  : Fri Oct 19 09:55:42 2012
     Host name              : beryllium         



So it's clearly working. However, -n 2 leads to a string of errors:

2abe8f67d000-2abe8f87c000 ---p 00005000 08:01 1839615                    /usr/lib/openmpi/lib/openmpi/mca_dpm_orte.so
2abe8f87c000-2abe8f87d000 rw-p 00004000 08:01 1839615                    /usr/lib/openmpi/lib/openmpi/mca_dpm_orte.so
2abe8f87d000-2ac06c5e3000 rw-p 00000000 00:00 0 
2ac070000000-2ac070021000 rw-p 00000000 00:00 0 
2ac070021000-2ac074000000 ---p 00000000 00:00 0 
7fffc3ce9000-7fffc3d0c000 rw-p 00000000 00:00 0                          [stack]
7fffc3dff000-7fffc3e00000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

Program received signal SIGABRT: Process abort signal.

Backtrace for this error:
^Cmpirun: killing job...

--------------------------------------------------------------------------
mpirun noticed that process rank 0 with PID 2846 on node beryllium exited on signal 0 (Unknown signal 0).
--------------------------------------------------------------------------
mpirun: clean termination accomplished




Errors:


Error:
Warning: Type mismatch in argument 'indsq' at (1); passed REAL(8) to INTEGER(4)

cc3_omeg3.F:3322.33:
                  CALL CC3_RACC3(DUMMY,OMEGA2M,WORK(KRMAT4),ISYMB,B,  
                                 1
Warning: Rank mismatch in argument 'omega2p' at (1) (rank-1 and scalar)
cc3_omeg3.F:3352.27:
            CALL CC3_RACC3(DUMMY,OMEGA2M,WORK(KRMAT3),ISYMD,D,ISYRES,
                           1
Warning: Rank mismatch in argument 'omega2p' at (1) (rank-1 and scalar)
cc3_omeg3.F: In function ‘cc3_racc3_.constprop.0’:cc3_omeg3.F:1319:0: internal compiler error: in expand_assignment, at expr.c:4731Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.7/README.Bugs> for instructions.
make[1]: *** [cc3_omeg3.o] Error 1
make[1]: Leaving directory `/home/me/tmp/dalton-2.0-cam/cc'
make: *** [CC_OBJ] Error 2

Solution:
add -fno-whole-file to FFLAGS in Makefile.config

Error:
make[1]: Leaving directory `/home/me/tmp/dalton-2.0-cam/cc'
cd dft && make all
make[1]: Entering directory `/home/me/tmp/dalton-2.0-cam/dft'
gfortran -I../include  -DVAR_GFORTRAN -DSYS_LINUX -DVAR_MFDS -D'INSTALL_WRKMEM=1000000000' -D'INSTALL_MMWORK=1000000000' -D_FILE_OFFSET_BITS=64 -DVAR_MPI -DGFORTRAN=471 -DIMPLICIT_NONE -DINSTALL_BASDIR='"/opt/dalton/basis/"' -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -fno-range-check -fno-whole-file -c dft_den.F
gfortran -I../include  -DVAR_GFORTRAN -DSYS_LINUX -DVAR_MFDS -D'INSTALL_WRKMEM=1000000000' -D'INSTALL_MMWORK=1000000000' -D_FILE_OFFSET_BITS=64 -DVAR_MPI -DGFORTRAN=471 -DIMPLICIT_NONE -DINSTALL_BASDIR='"/opt/dalton/basis/"' -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -fno-range-check -fno-whole-file -c dft_grid.F
gfortran -I../include  -DVAR_GFORTRAN -DSYS_LINUX -DVAR_MFDS -D'INSTALL_WRKMEM=1000000000' -D'INSTALL_MMWORK=1000000000' -D_FILE_OFFSET_BITS=64 -DVAR_MPI -DGFORTRAN=471 -DIMPLICIT_NONE -DINSTALL_BASDIR='"/opt/dalton/basis/"' -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -fno-range-check -fno-whole-file -c dft_ksm.F
dft_ksm.F:239:0: fatal error: mpif.h: No such file or directorycompilation terminated.
make[1]: *** [dft_ksm.o] Error 1
make[1]: Leaving directory `/home/me/tmp/dalton-2.0-cam/dft'
make: *** [DFT_OBJ] Error 2
Solution: 
Make sure you set
MPI_INCLUDE_DIR = -I/usr/lib/openmpi/include

It may not be necessary for the mpiXXX compilers, but e.g. gfortran would not know where to look for the mpi headers.

On ROCKS/CentOS:

Error:
mpif90 -I../include  -DVAR_GFORTRAN -DSYS_LINUX -DVAR_MFDS -D'INSTALL_WRKMEM=100000000' -D'INSTALL_MMWORK=100000000' -D_FILE_OFFSET_BITS=64 -DVAR_MPI -DGFORTRAN=412 -DIMPLICIT_NONE -DINSTALL_BASDIR='"/home/me/.dalton/basis"' -march=x86-64 -O3 -ffast-math -funroll-loops -ftree-vectorize -fno-range-check -c gphjj.F
 In file gphjj.F:416
      WRITE (LUPRIN,'(T6,2A)') 'Date and time (Linux)  : ',FDATE()    
                                                          1
 In file gphjj.F:89
                 
                   2
Error: Global name 'fdate' at (1) is already being used as a SUBROUTINE at (2)
 In file gphjj.F:934
            IF(IR-IB) 340,310,320                                    
                                                                       1
Warning: Obsolete: arithmetic IF statement at (1)
 In file gphjj.F:1390
      DATA C/8H********/                                              
              1
Warning: Extension: Hollerith constant at (1)
 In file gphjj.F:1390
      DATA C/8H********/                                              
                      1
Warning: Extension: Conversion from HOLLERITH to REAL(8) at (1)
make[1]: *** [gphjj.o] Error 1
make[1]: Leaving directory `/home/me/tmp/dalton-2.0-cam/gp'
make: *** [GP_OBJ] Error 2
Solution:
???

No comments:

Post a Comment