Showing posts with label gfortran. Show all posts
Showing posts with label gfortran. Show all posts

24 July 2012

215. Compiling gcc 4.7.1/gfortran 4.7.1 on Centos 5.6/ROCKS 4.5.3 (and gmp, mpfr, mpc, binutils (ld,as), glibc, libunistring, libtools...)

Update June 2013:
See flakrat's post for a working example: http://flakrat.blogspot.com.au/2013/06/building-gcc-481-on-centos-59.html

I haven't updated this post for a long time, and I haven't been using the GCC I compiled this way. See flakrat's post for an up-to-date working version.

Update(s): This is the first time I compile the GCC and because of this I will like be going back, updating this document over the coming days. Make sure to 1) check back in a week and 2) hit refresh.

Updated (given in Melb./Au time zone): 24/7, 25/7

The reason for this post is the outdated version (4.1) of gcc on our ROCKS 5.4.3/CentOS 5.6 cluster.

I looked at installing GCC 4.4.6 using rpm packages I found online, but I'm not used to the Red Hat way of doing things, and e.g. openmpi-devel was requiring a version of libgomp I wanted to update. Ultimately I got scared of messing up a production cluster and decided that compiling, while slower, is a whole lot safer -- especially if you're not comfortable with the local package manager.

So, here's the alternative route of compiling your own.

I'm really not a friend of CentOS or ROCKS. On the other hand, I freely admit that this is probably in large part due to not being used to it. But...during the course of this compilation my feelings have gone from mild annoyance to active, fiery hate. Mostly it has to do with how old everything is, and the difficult in updating anything in ROCKS.

This is probably my most massive compilation, owing to the number of packages I ended up compiling (a lot of these are probably optional). Guile in particular is very, very demanding.

The order in which things are done is not random


1. Look at 
http://gcc.gnu.org/install/configure.html
http://gcc.gnu.org/install/prerequisites.html

2. Download and untar all the sources
NOTE: you should, if possible, select mirrors based on where you are. However, sometimes you're stuck in a shell and just need to get those files downloaded.

mkdir ~/tmp/gcc
cd ~/tmp/gcc
wget http://www.netgull.com/gcc/releases/gcc-4.7.1/gcc-4.7.1.tar.gz
wget http://www.mpfr.org/mpfr-current/mpfr-3.1.1.tar.gz
wget ftp://ftp.gmplib.org/pub/gmp-5.0.5/gmp-5.0.5.tar.bz2
wget http://www.multiprecision.org/mpc/download/mpc-1.0.tar.gz

tar xvf gcc-4.7.1.tar.gz
tar xvf gmp-5.0.5.tar.bz2
tar xvf mpfr-3.1.1.tar.gz
tar xvf mpc-1.0.tar.gz

That was the easy bit.

I've also set up a directory called /share/apps/tools/gcc/ with proper permissions already (i.e. whoever is doing the compiling should have write access)

3. Build gmp
cd gmp-5.0.5/
mkdir build
cd build/
.././configure --prefix=/share/apps/tools/gcc/gmp --program-suffix=-gcc47
make
make install
make check


4. Build mpfr
cd ../../mpfr-3.1.1/
mkdir build
cd build/
.././configure --prefix=/share/apps/tools/gcc/mpfr --program-suffix=-gcc-4.7 --with-gmp=/share/apps/tools/gcc/gmp
make
make install

Libraries have been installed in:
   /share/apps/tools/gcc/mpfr/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the `LD_RUN_PATH' environment variable
     during linking
   - use the `-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.

make check

It looks like your compile is starting from scratch, but then it ends with:

====================
All 160 tests passed
(1 test was not run)
====================

5. Build mpc
cd ../../mpc-1.0/
mkdir build
cd build/
.././configure --prefix=/share/apps/tools/gcc/mpc --program-suffix=-gcc-4.7 --with-gmp=/share/apps/tools/gcc/gmp --with-mpfr=/share/apps/tools/gcc/mpfr
make
make install
make check

===================
All 64 tests passed
===================


6. Binutils
https://www.gnu.org/software/binutils/
'Often' (judging from forum postings...) you can use an older version of the binutils (ld, as) with a newer version of GCC. However, I need crt1.o when compilicing, it's part of glibc, and glibc requires newer versions of ld and as. So, here we go.
wget http://ftp.gnu.org/gnu/binutils/binutils-2.22.tar.gz
tar xvf binutils-2.22.tar.gz
cd binutils-2.22/
mkdir build
cd build/
.././configure --prefix=/share/apps/tools/gcc/binutils --program-suffix=-gcc-4.7
make
make install
make check

At some point I did:
cd /share/apps/tools/gcc/binutils/bin
ln -s ld-gcc-4.7 ld
ln -s as-gcc-4.7 as
ln -s ar-gcc-4.7 ar
but I don't think it's essential

7. Build gcc
cd ~/tmp/gcc-4.7.1
mkdir build
cd build/
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/share/apps/tools/gcc/gmp/lib:/share/apps/tools/gcc/mpc/lib:/share/apps/tools/gcc/mpfr/lib
.././configure --prefix=/share/apps/tools/gcc/gcc47 --program-suffix=-gcc-4.7 --with-gmp=/share/apps/tools/gcc/gmp --with-mpfr=/share/apps/tools/gcc/mpfr --with-mpc=/share/apps/tools/gcc/mpc LD_FOR_TARGET=ld-gcc-4.7 AS_FOR_TARGET=as-gcc-4.7 --with-ld=/share/apps/tools/gcc/binutils/bin/ld-gcc-4.7 --with-as=/share/apps/tools/gcc/binutils/bin/as-gcc-4.7 --with-ar=/share/apps/tools/gcc/binutils/bin/ar-gcc-4.7 AR_FOR_TARGET=ar-gcc-4.7

make

This takes A LONG TIME.

make install

To do 'make check' you need to have done the compilations of the optional packages (autogen and dependencies) below.
make check

You now have a shiny new compiler.

Using it is another matter. I'm working on that post at the moment...
You may find the following informative:

gcc-gcc-4.7 --print-search-dirs

install: /share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1/

programs: =/share/apps/tools/gcc/gcc47/libexec/gcc/x86_64-unknown-linux-gnu/4.7.1/:/share/apps/tools/gcc/gcc47/libexec/gcc/x86_64-unknown-linux-gnu/4.7.1/:/share/apps/tools/gcc/gcc47/libexec/gcc/x86_64-unknown-linux-gnu/:/share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1/:/share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/:/share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1/../../../../x86_64-unknown-linux-gnu/bin/x86_64-unknown-linux-gnu/4.7.1/:/share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1/../../../../x86_64-unknown-linux-gnu/bin/

libraries: =/share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1/:/share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1/../../../../x86_64-unknown-linux-gnu/lib/x86_64-unknown-linux-gnu/4.7.1/:/share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1/../../../../x86_64-unknown-linux-gnu/lib/../lib64/:/share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1/../../../x86_64-unknown-linux-gnu/4.7.1/:/share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1/../../../../lib64/:/lib/x86_64-unknown-linux-gnu/4.7.1/:/lib/../lib64/:/usr/lib/x86_64-unknown-linux-gnu/4.7.1/:/usr/lib/../lib64/:/share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1/../../../../x86_64-unknown-linux-gnu/lib/:/share/apps/tools/gcc/gcc47/lib/gcc/x86_64-unknown-linux-gnu/4.7.1/../../../:/lib/:/usr/lib/


8. glibc
cd ~/tmp/gcc
wget http://ftp.gnu.org/gnu/glibc/glibc-2.14.tar.gz
tar xvf blic-2.14.tar.gz
cd glibc-2.14/
mkdir build
cd build/
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/share/apps/tools/gcc/mpc/lib:/share/apps/tools/gcc/mpfr/lib:/share/apps/tools/gcc/gmp/lib
.././configure --prefix=/share/apps/tools/gcc/glibc CC=gcc-gcc-4.7 --with-headers=/usr/src/kernels/2.6.18-238.19.1.el5-x86_64/include
make
make install

Using the current glibc 2.16 requires kernel>2.6.19 which is why I used 2.14. I don't want to fiddle with installing a new kernel on a production system which is shared between several users (and time zones...):
configure: error: GNU libc requires kernel header files from
Linux 2.6.19 or later to be installed before configuring.
The kernel header files are found usually in /usr/include/asm and
/usr/include/linux; make sure these directories use files from
Linux 2.6.19 or later.  This check uses <linux/version.h>, so
make sure that file was built correctly when installing the kernel header
files.  To use kernel headers not from /usr/include/linux, use the
configure option --with-headers.




Optional -- libtool, libunistring, libffi, bdw-gc, autogen and guile

I'm still stuck on Guile.

If you want to run make check on your gcc build, you need autogen, which needs guile, which needs libtool and libunistring

cd ~/tmp/gcc
wget http://ftpmirror.gnu.org/libtool/libtool-2.4.2.tar.gz
tar xvf libtool-2.4.2.tar.gz
cd libtool-2.4.2/
mkdir build
cd build/
.././configure --prefix=/share/apps/tools/gcc/libtool --program-suffix=-2.4.2
make
make install

make check
(this is very slow)
======================
All 122 tests passed
(2 tests were not run)
======================
and
## ------------- ##
## Test results. ##
## ------------- ##

104 tests behaved as expected.
22 tests were skipped.

cd ~/tmp/gcc
wget http://ftp.gnu.org/gnu/libunistring/libunistring-0.9.3.tar.gz
tar xvf libunistring-0.9.3.tar.gz 
cd libunistring-0.9.3/
mkdir build
cd build/
.././configure --prefix=/share/apps/tools/gcc/libunistring
make
make install

make check
====================
All 418 tests passed
====================

cd ~/tmp/gcc
wget ftp://sourceware.org/pub/libffi/libffi-3.0.11.tar.gz
tar xvf libffi-3.0.11.tar.gz 
cd libffi-3.0.11/
mkdir build
cd build/
.././configure --prefix=/share/apps/tools/gcc/libffi
make
make install
cp include -R /share/apps/tools/gcc/libffi/
cp ../src/x86/ffitarget.h /share/apps/tools/gcc/libffi/include/

cd ~/tmp/gcc
wget http://www.hpl.hp.com/personal/Hans_Boehm/gc/gc_source/gc.tar.gz
tar xvf gc.tar.gz
cd gc-7.2/
mkdir build
cd build/
 .././configure --prefix=/share/apps/tools/gcc/bdw-gc
make

make check
===================
All 10 tests passed
===================

wget ftp://ftp.gnu.org/gnu/guile/guile-2.0.6.tar.gz
tar xvf guile-2.0.6.tar.gz
cd guile-2.0.6
mkdir build
cd build/
.././configure --prefix=/share/apps/tools/gcc/guile --with-libltdl-prefix=/share/apps/tools/gcc/libtool --with-libgmp-prefix=/share/apps/tools/gcc/gmp --with-libunistring-prefix=/share/apps/tools/gcc/libunistring LIBFFI_CFLAGS=-I/share/apps/tools/gcc/libffi/include LIBFFI_LIBS=-L/share/apps/tools/gcc/libffi/lib BDW_GC_CFLAGS=-I/share/apps/tools/gcc/bdw-gc/include BDW_GC_LIBS=-L/share/apps/tools/gcc/bdw-gc/lib
make

Stuck:

../.././libguile/finalizers.c:167: error: static declaration of 'GC_set_finalizer_notifier' follows non-static declaration
/share/apps/tools/gcc/bdw-gc/include/gc/gc.h:177: error: previous declaration of 'GC_set_finalizer_notifier' was here
make[3]: *** [libguile_2.0_la-finalizers.lo] Error 1
make[3]: *** Waiting for unfinished jobs....
make[3]: Leaving directory `/home/me/tmp/gcc/guile-2.0.6/build/libguile'
make: *** [all] Error 2


wget http://ftp.gnu.org/gnu/autogen/rel5.16/autogen-5.16.tar.gz
tar xvf autogen-5.16.tar.gz
cd autogen-5.16/
mkdir build
cd build/






Errors:
checking for x86_64-unknown-linux-gnu-gcc... /home/me/tmp/gcc/gcc-4.7.1/build/./gcc/xgcc -B/home/me/tmp/gcc/gcc-4.7.1/build/./gcc/ -B/share/apps/tools/gcc/gcc47/x86_64-unknown-linux-gnu/bin/ -B/share/apps/tools/gcc/gcc47/x86_64-unknown-linux-gnu/lib/ -isystem /share/apps/tools/gcc/gcc47/x86_64-unknown-linux-gnu/include -isystem /share/apps/tools/gcc/gcc47/x86_64-unknown-linux-gnu/sys-include
checking for suffix of object files... configure: error: in `/home/me/tmp/gcc/gcc-4.7.1/build/x86_64-unknown-linux-gnu/libgcc':
configure: error: cannot compute suffix of object files: cannot compile
See `config.log' for more details.
make[2]: *** [configure-stage1-target-libgcc] Error 1
make[2]: Leaving directory `/home/me/tmp/gcc/gcc-4.7.1/build'
make[1]: *** [stage1-bubble] Error 2
make[1]: Leaving directory `/home/me/tmp/gcc/gcc-4.7.1/build'
make: *** [all] Error 2

Solution: export LD_LIBRARY_PATH to include the mpc, mpfr and gmp lib dirs (see above)

Links to this post:
http://blog.naver.com/PostView.nhn?blogId=myunggyu&logNo=60173986972&categoryNo=0&parentCategoryNo=0&viewDate&currentPage=1&postListTopCurrentPage=1&userTopListOpen=true&userTopListCount=30&userTopListManageOpen=false&userTopListCurrentPage=1
http://flakrat.blogspot.com/2013/06/building-gcc-481-on-centos-59.html

18 May 2012

156. Compiling nwchem 6.1 with internal libs, openmpi, python on debian testing/wheezy

I followed this thread: http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id435/segmentation_fault--running_smal....html

See here for build with external blas: http://verahill.blogspot.com.au/2012/05/building-nwchem-61-on-debian.html

--Start here --
sudo apt-get install libopenmpi-dev python-dev

sudo mkdir /opt/nwchem
sudo chown ${USER} /opt/nwchem
cd /opt/nwchem
wget http://www.nwchem-sw.org/images/Nwchem-6.1-2012-Feb-10.tar.gz
tar xvf Nwchem-6.1-2012-Feb-10.tar.gz
mv nwchem-6.1 nwchem-6.1_internal
cd nwchem-6.1_internal/

Edit src/config/makefile.h and add -lz -lssl to the end of line 1914 (needed by python)

Edit src/tools/GNUmakefile and add two lines as shown below in red (line numbers added by me for illustration only) -- editing needed for compile with internal libs.
355 ifneq ($(BLAS_LIB),)
356     ifeq ($(BLAS_SIZE),4)
357         MAYBE_BLAS = --with-blas4="$(strip $(BLAS_LIB))"
358     else
359         MAYBE_BLAS = --with-blas8="$(strip $(BLAS_LIB))"
360     endif
361 endif
to
355 ifneq ($(BLAS_LIB),)
356     ifeq ($(BLAS_SIZE),4)
357         MAYBE_BLAS = --with-blas4="$(strip $(BLAS_LIB))"
358     else
359         MAYBE_BLAS = --with-blas8="$(strip $(BLAS_LIB))"
360     endif
361 else
362     MAYBE_BLAS = --without-blas
363 endif
Then continue as normal:


export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export PYTHONVERSION=2.7
export PYTHONHOME=/usr
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openmpi/lib
export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"
cd $NWCHEM_TOP/src
make clean
make nwchem_config
make FC=gfortran

**If you get errors relating to openssl etc. at the end of compile, try building without python support by removing anything in blue. You won't need to do make clean so the rebuild will be quick. 

Building takes ages

Edit your ~/.bashrc and add
export NWCHEM_EXECUTABLE=/opt/nwchem/nwchem-6.1_internal/bin/LINUX64/nwchem
export NWCHEM_BASIS_LIBRARY=/opt/nwchem/nwchem-6.1_internal/src/basis/libraries/
export PATH=$PATH:/opt/nwchem/nwchem-6.1_internal/bin/LINUX64


Links to this post:
http://chemport.ru/forum/viewtopic.php?f=71&t=98589

16 May 2012

151. Building nwchem 6.1 on debian wheezy/testing with openblas

Build without external libs in a later post.

Openblas:

sudo apt-get install build-essential gfortran gpp
sudo mkdir /opt/openblas
sudo chown ${USER} /opt/openblas
cd ~/tmp
wget http://nodeload.github.com/xianyi/OpenBLAS/tarball/v0.1.1
tar xvf v0.1.1
cd xianyi-OpenBLAS-e6e87a2/
wget http://www.netlib.org/lapack/lapack-3.4.1.tgz
make all BINARY=64 CC=/usr/bin/gcc FC=/usr/bin/gfortran USE_THREAD=0 INTERFACE64=1 1> make.log 2>make.err

make PREFIX=/opt/openblas install
cp lib*.*  /opt/openblas/lib

add
export LD_LIBRARY_PATH:$LD_LIBRARY_PATH:/opt/openblas/lib
to your ~/.bashrc

[for later use with nwchem and ecce, add /opt/openblas/lib to /etc/ld.so.conf and do sudo ldconfig]



Nwchem:
sudo apt-get install libopenmpi-dev python-dev

sudo mkdir /opt/nwchem
sudo chown ${USER} /opt/nwchem
cd /opt/nwchem
wget http://www.nwchem-sw.org/images/Nwchem-6.1-2012-Feb-10.tar.gz
tar xvf Nwchem-6.1-2012-Feb-10.tar.gz
cd nwchem-6.1/

Edit line 1914 in src/config/makefile.h and add -lz -lssl for python support
Then continue

export LARGE_FILES=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export PYTHONVERSION=2.7
export PYTHONHOME=/usr
export BLASOPT="-L/opt/openblas/lib -lopenblas"
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export LIBRARY_PATH=$LIBRARY_PATH:/usr/lib/openmpi/lib
export LIBMPI="-lmpi -lopen-rte -lopen-pal -ldl -lmpi_f77 -lpthread"
cd $NWCHEM_TOP/src
make clean
make nwchem_config
make FC=gfortran
export FC=gfortran
cd ../contrib
./getmem.nwchem

Building takes ages

Edit your ~/.bashrc and add
export NWCHEM_EXECUTABLE=/opt/nwchem/nwchem-6.1/bin/LINUX64/nwchem
export NWCHEM_BASIS_LIBRARY=/opt/nwchem/nwchem-6.1/src/basis/libraries/
export PATH=$PATH:/opt/nwchem/nwchem-6.1/bin/LINUX64

To make ecce play nice you also need to edit /etc/ld.so.conf and add
/opt/openblas/lib
then do 
sudo ldconfig

Links to this post:
http://chemport.ru/forum/viewtopic.php?f=71&t=98589