Platform.make odd variables settings

[frankie]

A recent issue reported on debian:

,-----[ /usr/lib/grass64/include/Make/Platform.make (lines: 66 - 70) ] | # GRASS dirs | GRASS_HOME = /build/buildd/grass-6.4.0~rc4 | RUN_GISBASE = /build/buildd/grass-6.4.0~rc4/dist.x86_64-pc-linux-gnu | RUN_GISRC = ${ARCH_DISTDIR}/demolocation/.grassrc${GRASS_VERSION_MAJOR}${GRASS_VERSION_MINOR} `-----

It seems to me that the grass building system is creating inconsistent vars in Platform.make.

A build log  https://buildd.debian.org/fetch.cgi?pkg=grass;ver=6.4.0~rc4-3;arch=amd64;stamp=1242167209

[frankie]

Sorry, a better view:

,-----[ /usr/lib/grass64/include/Make/Platform.make (lines: 66 - 70) ]
| # GRASS dirs
| GRASS_HOME          = /build/buildd/grass-6.4.0~rc4
| RUN_GISBASE         = /build/buildd/grass-6.4.0~rc4/dist.x86_64-pc-linux-gnu
| RUN_GISRC           = ${ARCH_DISTDIR}/demolocation/.grassrc${GRASS_VERSION_MAJOR}${GRASS_VERSION_MINOR}
`----- 

[glynn]

Replying to frankie:

A recent issue reported on debian:

It seems to me that the grass building system is creating inconsistent vars in Platform.make.

What exactly is the problem here?

[hamish]

Glynn:

What exactly is the problem here?

Debian ships a grass-dev package with include/Make, .h and friends.

This way people with grass installed from the binary packages can compile addon modules by using:

 make MODULE_TOPDIR=/usr/lib/grass

instead of having to maintain & build the entire source tree.

because Platform.make does not get modified to use --prefix=/usr during the 'make install' step, the debian package is left with remnant chroot build directory names there.

For anyone building from source, and those of us running directly from the src/dist.i686* dir (never ran 'make install') the given build dir is appropriate.

see

 http://thread.gmane.org/gmane.comp.gis.grass.user/29683/focus=29684

Hamish

[glynn]

Replying to hamish:

What exactly is the problem here?

Debian ships a grass-dev package with include/Make, .h and friends.

I see now.

"make install" should probably modify the GRASS_HOME setting in Platform.make similar to the modification of GISBASE in the grassNN[.bat] scripts and etc/fontcap.

At the very least, RUN_GISBASE should probably be set to either $(GRASS_HOME)/dist.$(ARCH) or $(ARCH_DISTDIR) in Grass.make, rather than being substituted by configure.

But there's still a lot more work to be done in disentangling the GRASS, source and output directories so that building against an installed version of GRASS works correctly. E.g. both BIN and INC are defined relative to ARCH_DISTDIR, so you can't re-build a module against installed headers without installing the resulting module.

[mlennert]

Replying to hamish:

Glynn:

What exactly is the problem here?

Debian ships a grass-dev package with include/Make, .h and friends. This way people with grass installed from the binary packages can compile addon modules by using: {{{ make MODULE_TOPDIR=/usr/lib/grass }}} instead of having to maintain & build the entire source tree. because Platform.make does not get modified to use --prefix=/usr during the 'make install' step, the debian package is left with remnant chroot build directory names there. For anyone building from source, and those of us running directly from the src/dist.i686* dir (never ran 'make install') the given build dir is appropriate. see  http://thread.gmane.org/gmane.comp.gis.grass.user/29683/focus=29684

While waiting for a solution to this, what is the recommended way of compiling addons without downloading the entire source tree ?

Moritz

[neteler]

Replying to mlennert:

what is the recommended way of compiling addons without downloading the entire source tree ?

Won't the new g.extension script do the job?

Markus

[hamish]

Replying to mlennert:

While waiting for a solution to this, what is the recommended way of compiling addons without downloading the entire source tree ?

try

svn checkout --non-recursive https://svn.osgeo.org/grass/grass-addons/path/to/addon/
cd /path/to/addon/
make MODULE_TOPDIR=/path/to/grass_source_tree/

you probably would not use --non-recursive in this case, but it may give you an idea how it can be made to be flexible.

Hamish

[mlennert]

Replying to hamish:

Replying to mlennert:

While waiting for a solution to this, what is the recommended way of compiling addons without downloading the entire source tree ?

try {{{ svn checkout --non-recursive  https://svn.osgeo.org/grass/grass-addons/path/to/addon/ cd /path/to/addon/ make MODULE_TOPDIR=/path/to/grass_source_tree/ }}}

So this means that there is no way to avoid downloading the entire source tree ?

Moritz

[hamish]

Replying to mlennert:

So this means that there is no way to avoid downloading the entire source tree ?

no, just the opposite.

e.g. to download v.strahler:

svn checkout https://svn.osgeo.org/grass/grass-addons/vector/v.strahler

Hamish

[mlennert]

Replying to hamish:

Replying to mlennert:

So this means that there is no way to avoid downloading the entire source tree ?

no, just the opposite. e.g. to download v.strahler: {{{ svn checkout  https://svn.osgeo.org/grass/grass-addons/vector/v.strahler }}}

I'm not talking about checking out the AddOn?, I'm talking about compiling it. You suggest:

make MODULE_TOPDIR=/path/to/grass_source_tree/

which indicates that I need the source tree for compilation...

I am looking for a way to get the equivalent of

make MODULE_TOPDIR=/usr/lib/grass

but working around the bug reported here...

This is not for me, but for my students who would like to install some addons, and so I would like to know if there is any way for them to do so currently, using the grass-dev package, or whether this bug makes this impossible.

Moritz

[mlennert]

Replying to neteler:

Replying to mlennert:

what is the recommended way of compiling addons without downloading the entire source tree ?

Won't the new g.extension script do the job?

Here are my attempts (using self-compiled 6.4svn debian package after having moved the original build tree to simulate use on a different machine):

Compiling 'r.terracost'...
Makefile:15: warning: overriding commands for target `/home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/bin/r.terracost'
/usr/lib/grass/include/Make/Module.make:15: warning: ignoring old commands for target `/home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/bin/r.terracost'
mkdir -p /home/mlennert/SRC/GRASS/grass64_release/bin.i486-pc-linux-gnu
mkdir -p /home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/include/grass
mkdir -p /home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/lib
mkdir -p /home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/bin
mkdir -p /home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/etc
mkdir -p /home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/driver
mkdir -p /home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/driver/db
mkdir -p /home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/fonts
test -d OBJ.i486-pc-linux-gnu || mkdir -p OBJ.i486-pc-linux-gnu
c++ -I/home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/include  -g -O2      -DPACKAGE=\""grassmods"\"  -I/home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/include -o OBJ.i486-pc-linux-gnu/boundary.o -c boundary.cc
In file included from boundary.cc:18:
boundary.h:22:32: error: grass/iostream/ami.h: No such file or directory
In file included from boundary.h:25,
                 from boundary.cc:18:
common.h:30:23: error: grass/gis.h: No such file or directory
In file included from boundary.cc:22:
pqueue.h:24:31: error: grass/iostream/mm.h: No such file or directory
[...]
make: *** [OBJ.i486-pc-linux-gnu/boundary.o] Error 1
ERROR: Compilation failed, sorry. Please check above error messages.

i.e.: it obviously cannot find the headers since it is looking for them in the original build directory.

If I go into /usr/lib/grass/include/Make/Platform.make and set GRASS_HOME to /usr/lib/grass, I get:

Makefile:15: warning: overriding commands for target `/usr/lib/grass/dist.i486-pc-linux-gnu/bin/r.terracost'
/usr/lib/grass/include/Make/Module.make:15: warning: ignoring old commands for target `/usr/lib/grass/dist.i486-pc-linux-gnu/bin/r.terracost'
mkdir -p /usr/lib/grass/bin.i486-pc-linux-gnu
mkdir: cannot create directory `/usr/lib/grass/bin.i486-pc-linux-gnu': Permission denied
make: *** [/usr/lib/grass/bin.i486-pc-linux-gnu] Error 1
ERROR: Compilation failed, sorry. Please check above error messages.

And even if I use the original grass-dev with GRASS_HOME pointing to the local build directory and keep this directory accessible, I get:

Installing 'r.terracost'...
Makefile:15: warning: overriding commands for target `/home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/bin/r.terracost'
/usr/lib/grass/include/Make/Module.make:15: warning: ignoring old commands for target `/home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/bin/r.terracost'
/usr/bin/install -c  /home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/bin/r.terracost /home/mlennert/.grass/addons/bin/
/usr/bin/install: target `/home/mlennert/.grass/addons/bin/' is not a directory: No such file or directory
make: *** [install] Error 1
WARNING: Installation failed, sorry. Please check above error messages.

If I create the .grass/addons/bin directory by hand, I get:

Installing 'r.terracost'...
Makefile:15: warning: overriding commands for target `/home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/bin/r.terracost'
/usr/lib/grass/include/Make/Module.make:15: warning: ignoring old commands for target `/home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/bin/r.terracost'
/usr/bin/install -c  /home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/bin/r.terracost /home/mlennert/.grass/addons/bin/
/usr/bin/install -c  -m 644 /home/mlennert/SRC/GRASS/grass64_release/dist.i486-pc-linux-gnu/docs/html/r.terracost.html /home/mlennert/.grass/addons/docs/html/
/usr/bin/install: target `/home/mlennert/.grass/addons/docs/html/' is not a directory: No such file or directory
make: *** [install] Error 1
WARNING: Installation failed, sorry. Please check above error messages.

I'll stop here before this becomes more of a bug report on g.extension than on the current issue, but it would be nice to get to a situation where by installing the grass binaries and libs and the grass headers, it is possible to compile extensions without needing the source tree.

Moritz

[glynn]

Replying to mlennert:

I'm not talking about checking out the AddOn?, I'm talking about compiling it. You suggest:

make MODULE_TOPDIR=/path/to/grass_source_tree/

which indicates that I need the source tree for compilation... I am looking for a way to get the equivalent of

make MODULE_TOPDIR=/usr/lib/grass

This should work in 7.0, although you also may also need to set GISBASE, ARCH_DISTDIR and/or RUN_GISBASE.

[neteler]

Same problem on Ubuntu apparently:

 http://lists.osgeo.org/pipermail/grass-user/2009-August/052073.html

 http://lists.osgeo.org/pipermail/grass-user/2009-August/052074.html

[neteler]

Same problem on OpenSuSe?:

 http://osgeo-org.1803224.n2.nabble.com/Installing-r-area-td5340139.html#a5363725

[hamish]

see also #854, #1178, and #1180

[hamish]

hopefully now fixed for 'make install' in 6.5svn. some calls to sed in the install target in the Makefile now replace the build dir with the install dir as part of the install process.

if distro package install scripts are not using 'make install' they'll have to recreate this step themselves.

combined patch for 6.4svn from r43737, r43748, r43749, r43811:

Index: Makefile
===================================================================
--- Makefile    (revision 43826)
+++ Makefile    (working copy)
@@ -95,6 +95,7 @@
        -cp -f $(FILES) ${ARCH_DISTDIR}/
        -cp -f ${ARCH_BINDIR}/grass${GRASS_VERSION_MAJOR}${GRASS_VERSION_MINOR} ${ARCH_DISTDIR}/grass${GRASS_VERSION_MAJOR}${GRASS_VERSION_MINOR}.tmp
        @test -d ${ARCH_DISTDIR}/tools/ || mkdir -p ${ARCH_DISTDIR}/tools/
+       -${INSTALL} tools/g.html2man/g.html2man ${ARCH_DISTDIR}/tools/ 2>/dev/null
        -${INSTALL} tools/mkhtml.sh ${ARCH_DISTDIR}/tools/ 2>/dev/null
        @(cd tools ; sh -c "./build_html_index.sh")
        @if [ `cat "$(ERRORLOG)" | wc -l` -gt 5 ] ; then \
@@ -288,6 +289,8 @@
        @ # default to be /usr/local
        @##### -cd ${GISBASE} ; tar cBf - man | (cd ${INST_DIR} ; tar xBf - ) 2>/dev/null
        -cd ${GISBASE} ; tar cBf - include | (cd ${INST_DIR} ; tar xBf - ) 2>/dev/null
+       -sed -i -e "s#^\(GRASS_HOME.[^=]*\).*#\1= ${INST_DIR}#" -e "s#^\(RUN_GISBASE.[^=]*\).*#\1= ${INST_DIR}#" ${INST_DIR}/include/Make/Platform.make
+       -sed -i -e 's#/tools/g.html2man/g.html2man#/tools/g.html2man#' ${INST_DIR}/include/Make/Man.make
        -cd ${GISBASE} ; tar cBf - lib | (cd ${INST_DIR} ; tar xBf - ) 2>/dev/null
        -sed 's#'${GISBASE}'#'${INST_DIR}'#g' ${GISBASE}/etc/monitorcap > ${INST_DIR}/etc/monitorcap
        -sed 's#'${GISBASE}'#'${INST_DIR}'#g' ${GISBASE}/etc/fontcap > ${INST_DIR}/etc/fontcap

Hamish

[neteler]

Please attach the patch as file, not inline, thanks.

[hamish]

latest g.extension from 6.5svn, built from source and 'make installed' to /usr/local/ on linux:

GRASS65> g.extension i.landsat.toar 
Fetching <i.landsat.toar> from GRASS-Addons SVN (be patient)...
A    i.landsat.toar/landsat_set.c
A    i.landsat.toar/local_proto.h
A    i.landsat.toar/main.c
A    i.landsat.toar/description.html
A    i.landsat.toar/landsat.c
A    i.landsat.toar/earth_sun.c
A    i.landsat.toar/landsat.h
A    i.landsat.toar/landsat_met.c
A    i.landsat.toar/Makefile
A    i.landsat.toar/earth_sun.h
Checked out revision 43842.
Compiling <i.landsat.toar>...
/usr/local/grass-6.5.svn/include/Make/Module.make:25: warning: overriding commands for target `install'
/usr/local/grass-6.5.svn/include/Make/Rules.make:90: warning: ignoring old commands for target `install'
mkdir -p /usr/local/grass-6.5.svn/bin.x86_64-unknown-linux-gnu
mkdir: cannot create directory `/usr/local/grass-6.5.svn/bin.x86_64-unknown-linux-gnu': Permission denied
make: *** [/usr/local/grass-6.5.svn/bin.x86_64-unknown-linux-gnu] Error 1
ERROR: Compilation failed, sorry. Please check above error messages.

the problem is that the end-user doesn't have write permission to /usr/local/grass-6.x/.

the goal is for the addon to compile in the .tmp/ dir where the source code was downloaded, then be installed to the ~user/$ADDON_DIR later on. but the current make request is trying to install it to (what we've tricked it into thinking is) the build dir.

I'm not sure how to proceed; hopefully something can be passed on the make command line to avoid this.

It works ok if grass was built from source (and that source tree still exists, user can write to it, etc).

Hamish

[martinl]

What is status of this ticket?

[hamish]

Replying to martinl:

What is status of this ticket?

Open.

see also #854, #1178, #1180, r45804, r45796.

Hamish

[neteler]

Please try fix in r46314.

[manu]

The fix worked for me (after compiling the grass-6.4.2svn from the releasebranch on Ubuntu 10.10

[manu]

Broken again on Ubuntu 11.10 and still "the Variable LOCATION_NAME not set" problem

[hamish]

Replying to manu:

Broken again on Ubuntu 11.10 and still "the Variable LOCATION_NAME not set" problem

if you can, please try the debian build rules from DebianGIS git, and the g.extension from devbr6, I just sync'd that to the latest behaviour of the g.extension script yesterday. I would not be surprised if earlier versions and combinations fail.

afair the "the Variable LOCATION_NAME not set" message does no harm and can be ignored. can you attach the full output?

how are you building? manually or in a .deb package? where are you running it from? directly from the build dir or after 'make install', or from the .deb?

I tested 6.4.2rc2 on ubuntu 11.10 yesterday (debian sid too) while helping Helmut, g.extension worked for me from the .deb, and from the build dir for 6.5svn, but I will have another try in case I missed something.

thanks, Hamish

[manu]

I'm on Ubuntu 11.10.

I tried to compile 6.4.2 (release_20111115_grass_6_4_2RC2) with the build rules from DebianGIS  http://anonscm.debian.org/gitweb/?p=pkg-grass/grass.git;a=blob_plain;f=debian/rules;hb=HEAD but as with the package Makefile, compilation yielded some errors although final error message was "No errors detected." .

Grass startup then crashes as discribed on other plateform  https://bugzilla.redhat.com/show_bug.cgi?id=694956, see below.

Any hints?

Cheers

I run ./configure with the -with-tcltk-includes=/usr/include/tcl8.4 option.

Starting GRASS ... Traceback (most recent call last):

File "/usr/local/grass-6.4.2RC2/etc/wxpython/gis_set.py", line 894, in <module>

GRASSStartUp = StartUp??(0)

File "/usr/lib/python2.7/dist-packages/wx-2.8-gtk2-unicode/wx/_core.py", line 7978, in init

self._BootstrapApp()

File "/usr/lib/python2.7/dist-packages/wx-2.8-gtk2-unicode/wx/_core.py", line 7552, in _BootstrapApp

return _core_.PyApp??BootstrapApp?(*args, **kwargs)

File "/usr/local/grass-6.4.2RC2/etc/wxpython/gis_set.py", line 865, in OnInit??

StartUp?? = GRASSStartup()

File "/usr/local/grass-6.4.2RC2/etc/wxpython/gis_set.py", line 166, in init

self._set_properties()

File "/usr/local/grass-6.4.2RC2/etc/wxpython/gis_set.py", line 213, in _set_properties

self.OnSetDatabase??(None)

File "/usr/local/grass-6.4.2RC2/etc/wxpython/gis_set.py", line 686, in OnSetDatabase??

self.UpdateLocations??(self.gisdbase)

File "/usr/local/grass-6.4.2RC2/etc/wxpython/gis_set.py", line 568, in UpdateLocations??

self.listOfLocations = utils.GetListOfLocations??(dbase)

File "/usr/local/grass-6.4.2RC2/etc/wxpython/gui_modules/utils.py", line 602, in GetListOfLocations??

for location in glob.glob(os.path.join(dbase, "*")):

File "/usr/lib/python2.7/glob.py", line 16, in glob

return list(iglob(pathname))

File "/usr/lib/python2.7/glob.py", line 43, in iglob

yield os.path.join(dirname, name)

File "/usr/lib/python2.7/posixpath.py", line 71, in join

path += '/' + b

UnicodeDecodeError??: 'ascii' codec can't decode byte 0xa3 in position 1: ordinal not in range(128) Error in GUI startup. If necessary, please report this error to the GRASS developers. Switching to text mode now.

[manu]

Replying to manu:

I'm on Ubuntu 11.10. I tried to compile 6.4.2 (release_20111115_grass_6_4_2RC2) with the build rules from DebianGIS  http://anonscm.debian.org/gitweb/?p=pkg-grass/grass.git;a=blob_plain;f=debian/rules;hb=HEAD but as with the package Makefile, compilation yielded some errors although final error message was "No errors detected." . Grass startup then crashes as discribed on other plateform  https://bugzilla.redhat.com/show_bug.cgi?id=694956, see below.

I just solved this problem by starting grass straight into a location and then starting the wxpython gui manually.

Regarding the g.extension problem, still some errors when compiling modules such as r.out.gmt2 but installation succeeds despite these, after a make MODULE_TOPDIR=/usr/local/grass-6.4.2RC2/ within the module directory

[hamish]

Replying to manu:

Regarding the g.extension problem, still some errors when compiling modules such as r.out.gmt2 but installation succeeds despite these,

ERROR: G_getenv(): Variable LOCATION_NAME not set now fixed in devbr6 and 6.4svn with r49341,2. It was harmless except that the --html-description usage part was not being added to the help pages because the dummy location was missing.

Hamish

[hamish]

I believe this has been fixed by a combination of recent changes to the grass build/install scripts and the debian packaging rules script.

Hamish