add --with-openmp support to ./configure

[hamish]

Hi,

lib/gpde already supports OpenMP (but requires Makefile hacks to enable) and hopefully v.lidar's tcholDec() will support OpenMP soon as well, so it would be nice if ./configure could do checks & sets for --with-openmp as well as --with-pthread. It would mean we have two ways of parallelizing stuff, which is sort of redundant, but maybe each can be used to its own strength? i.e. we have a large enough array of algorithms in the modules so that some will be both well and poorly suited to either method.

see also ​http://grass.osgeo.org/wiki/OpenMP

my philosophy would be to only add it to known bottlenecks, ie fight the urge to add it to every for loop just because it may be possible to do so. Hit the low hanging fruit and leave the rest of the code stable.

also it would be good to add some Doxygen documentation to lib/gis/worker.c for educational purposes.

thanks, Hamish

[hamish]

Hi,

wrt trunk/lib/gpde/Makefile, does -lgomp have to be before $(GMATHLIB) and -lm or could we do

EXTRA_LIBS += -lgomp

later?

should we do the same "+=" trick for EXTRA_CFLAGS=-fopenmp, to stop it from clobbering any earlier setting?

hopefully when this wish is fulfilled these things could be added as needed with a permanent ifdef in the Makefile(s). (do that centrally in grass's libmath or in the couple of library/modules that would use it?)

thanks, Hamish

[hamish]

see also ​http://thread.gmane.org/gmane.comp.gis.grass.devel/37962/focus=38004

[glynn]

Replying to hamish:

wrt trunk/lib/gpde/Makefile, does -lgomp have to be before $(GMATHLIB) and -lm or could we do

EXTRA_LIBS += -lgomp

later?

lib/gpde/Makefile should just have $(OMPLIB) at the appropriate place in EXTRALIBS, probably immediately before $(MATHLIB).

should we do the same "+=" trick for EXTRA_CFLAGS=-fopenmp, to stop it from clobbering any earlier setting?

EXTRA_CFLAGS = $(OMPCFLAGS)

Once the configure checks are written, Platform.make.in would define these. Until then, you can use e.g.

make OMPCFLAGS=-fopenmp OMPLIB=-lgomp ...

[huhabla]

OpenMP support is implemented in the gpde and the gmath libraries. In the gmath lib many linear equation solver and all blas level 2 and 3 functions are parallelized with OpenMP.

OpenMP flags are compiler dependent, thus OpenMP support should be set using C- and linker-flags bevor configure. I.e: for gcc > 4.2:

CFLAGS="-O3 -Wall -Werror-implicit-function-declaration -fno-common -fopenmp"
LDFLAGS="-lgomp"

This should enable OpenMP support in the libraries and ALL depending modules.

You can test the OpenMP support when compiling the gpde and gmath tests by hand (switch into the test directories in the lib dirs and type make). The test library modules "test.gmath.lib" and "test.gpde.lib" should be available in the path after starting grass.

The gmath lib test module "test.gmath.lib" provides additionally benchmarks for blas level 2 and 3 functions and for many solver.

gmath/test> test.gmath.lib help

Description:
 Performs benchmarks, unit and integration tests for the gmath library

Usage:
 test.gmath.lib [-uia] [unit=string] [integration=string] [rows=value]
   [solverbench=string] [blasbench=string] [--verbose] [--quiet]

Flags:
  -u   Run all unit tests
  -i   Run all integration tests
  -a   Run all unit and integration tests
 --v   Verbose module output
 --q   Quiet module output

Parameters:
         unit   Choose the unit tests to run
                options: blas1,blas2,blas3,solver,ccmath,matconv
  integration   Choose the integration tests to run
                options:
         rows   The size of the matrices and vectors for benchmarking
                default: 1000
  solverbench   Choose solver benchmark
                options: krylov,direct
    blasbench   Choose blas benchmark
                options: blas2,blas3

I.e testing the speedup of the blas level 2 and 3 functions of the latest svn trunk of grass7, compiled with the flags mentioned above on a 8 core intel xeon system:

gmath/test> setenv OMP_NUM_THREADS 1
gmath/test> test.gmath.lib blasbench=blas2 rows=5000

++ Running blas level 2 benchmark ++
Computation time G_math_Ax_sparse: 0.244123
Computation time G_math_Ax_sband: 0.280636
Computation time G_math_d_Ax: 0.134494
Computation time G_math_d_Ax_by: 0.18556
Computation time G_math_d_x_dyad: 0.268684

-- gmath lib tests finished successfully --

gmath/test> setenv OMP_NUM_THREADS 4
gmath/test> test.gmath.lib blasbench=blas2 rows=5000

++ Running blas level 2 benchmark ++
Computation time G_math_Ax_sparse: 0.072549
Computation time G_math_Ax_sband: 0.192712
Computation time G_math_d_Ax: 0.036652
Computation time G_math_d_Ax_by: 0.047904
Computation time G_math_d_x_dyad: 0.080534

-- gmath lib tests finished successfully --

gmath/test> setenv OMP_NUM_THREADS 1
gmath/test> test.gmath.lib blasbench=blas3 rows=1000

++ Running blas level 3 benchmark ++
Computation time G_math_d_aA_B: 0.013263
^[[AComputation time G_math_d_AB: 18.729

-- gmath lib tests finished successfully --

gmath/test> setenv OMP_NUM_THREADS 4
gmath/test> test.gmath.lib blasbench=blas3 rows=1000

++ Running blas level 3 benchmark ++
Computation time G_math_d_aA_B: 0.006946
Computation time G_math_d_AB: 4.80446

-- gmath lib tests finished successfully --

The gmath library should work fine with OpenMP support. In the gpde library is still a bug (race condition) in the linear equation system creator.

[hamish]

patch adding OpenMP support to the build system

[hamish]

Hi,

patch attached for review.

missing from it, but done locally: autoconf'd ./configure (too big), added $(OMPLIBPATH) to Grass.make, added $(OMP..) to lib/gmath/test and lib/gpde/test Makefiles.

missing from it, and TODO (pls help): only set OMPCFLAGS=-fopenmp for gcc >= 4.2.1

seems to work very nicely indeed. race condition for lib/gpde mentioned in comment:4 still exists.

see also ​http://grass.osgeo.org/wiki/OpenMP

Hamish

[hamish]

updated patch, fills in a few bits missing from the last one

[hamish]

added in trunk with r49652 with support for: gcc (>=4.2.1), Intel's C compiler, AMD's x86 Open64 compiler, and PGI's compiler. Additional support for Solaris cc, AIX xlc, and SGI IRIX 6.5 MIPSpro C is known but left commented out for now.

todo:

# try to build a test program with "$CC $OMPCLFAGS" to confirm it works
#...how?
#echo "int main() {return 0;}" > test.c 
#$CC $OMPCLFAGS test.c -o test

I think the test compile is better than explicitly checking the gcc version,

GCC_VERSION=`${CC} -dumpversion`
GCC_VERSION_MAJOR=$(echo $GCC_VERSION | cut -d'.' -f1)
GCC_VERSION_MINOR=$(echo $GCC_VERSION | cut -d'.' -f2)
GCC_VERSION_PATCH=$(echo $GCC_VERSION | cut -d'.' -f3)

for a gcc version test, see also:

​http://git.savannah.gnu.org/gitweb/?p=autoconf-archive.git;a=blob_plain;f=m4/ax_gcc_version.m4

a positive but reasonably inefficient speedup for v.surf.rst has been added in r49653.

Hamish

[martinl]

Still relevant?, there is --with-openmp available in configure script.

[hamish]

Hamish wrote:

added in trunk with r49652

yes, it's done in the configure script, which is all for this ticket. the wider subject of using OpenMP well is an ongoing project.

Hamish