Opened 10 years ago

Closed 6 years ago

#718 closed defect (fixed)

r.li forgets mask/illegal filename

Reported by: kyngchaos Owned by: grass-dev@…
Priority: major Milestone: 6.4.4
Component: Raster Version: svn-releasebranch64
Keywords: r.li, smp Cc: pcav
CPU: All Platform: All

Description

The r.li daemon seems to be forgetting the mask name used for a moving window operation. In the terminal:

Illegal filename. Character < > not allowed.
Illegal filename. Character < > not allowed.
Illegal filename. Character < > not allowed.
Unable to open header file for raster map < @(null)>
CHILD[pid = 1486]: unable to open   mask ... continue without!!!
Segmentation fault [64bit only]
... many more illegal filename errors

It segfaults on OSX 64bit, but not on 32bit OSX, where it still completes but the output looks OK (but I don't know if it's right).

64bit crash log :

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000003036
Crashed Thread:  0

Thread 0 Crashed:
0   libSystem.B.dylib             	0x00007fff840d9d70 strcpy + 48
1   libgrass_rli.dylib            	0x000000010005d3ed next_Area + 111
2   libgrass_rli.dylib            	0x000000010005da8a calculateIndex + 859
3   r.li.patchdensity             	0x000000010000175c start + 52

Tests OK on Linux (thanks Markus).

Attachments (4)

Picture 1.png (5.4 KB) - added by kyngchaos 10 years ago.
rerun 32bit patchdensity: upper-left corner (rest of map is null), spearfish landuse
r.li.daemon-strtok.patch (5.8 KB) - added by glynn 10 years ago.
replace strtok with strtok_r
patchnoise.png (104.7 KB) - added by kyngchaos 9 years ago.
patchdensity noise bands 64bit (in different locations for 32bit)
landsat_test (49 bytes) - added by lucadelu 8 years ago.
configuration file for r.li.shannon test

Download all attachments as: .zip

Change History (35)

Changed 10 years ago by kyngchaos

Attachment: Picture 1.png added

rerun 32bit patchdensity: upper-left corner (rest of map is null), spearfish landuse

comment:1 Changed 10 years ago by hamish

Cc: pcav added
Keywords: r.li added

William wrote (on grass-user):

I did a little crude debugging yesterday (G_message) and
found it failed for me in daemon.c, line 626 / nextArea()

    else {
    return next(g, m);
    }

But all this tells me is that at this point the mask name
is missing, not where it was lost.

Paolo added:

I'm also having problems with r.li:

> r.li.shannon map=uso conf=latignano_5 output=provaln
WARNING: Unable to open header file for raster map <@(null)>
CHILD[pid = 1954]: unable to open mask ... continue without!!!
...
Segmentation fault

Debian ustable, 64 bit.

Could someone please post the series of steps needed to reproduce this using either the NC or Spearfish datasets?

thanks

Hamish

comment:2 Changed 10 years ago by pcav

CPU: Allx86-64

Here it is: === load landclass96

r.li.setup

SAMPLINGFRAME 0|0|1|1

SAMPLEAREA -11|0.0210970464135|0.0190114068441

MOVINGWINDOW

r.li.shannon map=landclass96@PERMANENT conf=/home/paolo/.r.li/history/landclass96_conf output=landclass96_shannon

Cannot make stat of /home/paolo/.r.li/historyhome/paolo/.r.li/history/landclass96_conf configuration file

Unable to open header file for raster map <@(null)> CHILD[pid = 26154]: unable to open mask ... continue without!!! Unable to open header file for raster map <@(null)>

CHILD[pid = 26153]: unable to open mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 26162]: unable to open mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 26161]: unable to open mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 26160]: unable to open mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 26159]: unable to open mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 26158]: unable to open mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 26157]: unable to open mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 26156]: unable to open mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 26155]: unable to open mask ... continue without!!! === Debian Sid amd64

This is not a recent bug: it has been there since at least one year. On i386 it should work fine

comment:3 in reply to:  2 ; Changed 10 years ago by hamish

Replying to pcav:

Here it is: ===

> load landclass96
> 
> r.li.setup
> 
> SAMPLINGFRAME 0|0|1|1
> SAMPLEAREA -1|-1|0.0210970464135|0.0190114068441
> MOVINGWINDOW

ok, I'm not familiar with the module at all so you'll have to explain it to me fully & slowly :)

so far I have:

# nc_spm_08 NC sample dataset
d.mon x0
g.region rast=landclass96
d.rast landclass96

r.li.setup

# GUI opens

now what?

hmmm.. (guessing)

$ cat << EOF > ~/.r.li/history/landclass96_conf
SAMPLINGFRAME 0|0|1|1
SAMPLEAREA -1|-1|0.0210970464135|0.0190114068441
MOVINGWINDOW
EOF

restart r.li.setup

# gui pops up
select landclass96_conf from list
[Load]

# gui pops up, red plot with blue sample-area cell in top left
# "Disposition Moving Windows" (aka current moving window setup??)

not sure what to do next, so [Close] r.li.setup.

# Debian/etch i386 32bit Pentium4
# full conf= path needed?
G65> r.li.shannon map=landclass96 conf=landclass96_conf \
   output=landclass96_shannon
[runs for a minute]
r.li.worker (pid 28631) terminated
r.li.worker (pid 28635) terminated
r.li.worker (pid 28626) terminated
r.li.worker (pid 28633) terminated
r.li.worker (pid 28634) terminated
r.li.worker (pid 28632) terminated
r.li.worker (pid 28627) terminated
r.li.worker (pid 28629) terminated
r.li.worker (pid 28628) terminated
r.li.worker (pid 28630) terminated

G65> echo $?
1

??

d.rast landclass96_shannon # looks good

If I do the exact same steps on Debian/Lenny? amd64 I get the exact same result.

?

I would note that gkrellm on both machines shows about 1/2 the time spent in kernel I/O, a sign of a bad inefficiency somewhere. (also gkrellm shows that it is using all 4-cores on the 64bit machine, which is nice; I guess that is what all those workers are busy doing)

real    0m4.895s
user    0m9.493s
sys     0m9.125s

?  (user + sys) / num_cores = real

from raster/r.li/r.li.daemon/daemon.c:

    /* TODO: check if this path is portable */
    sprintf(pathSetup, "%s/.r.li/history/%s", getenv("HOME"), file);
    parsed = parseSetup(pathSetup, l, g, raster);

So you should not pass the full path to r.li.shannon conf= as it adds it itself, just the config file-name. Note r.li.shannon's parser code is currently set up to use the file picker which fills in the full path automatically, so that would have to be removed or the above would need to test the pathname and try to chop away the $HOME/.r.li/ part if it was given twice. I think the r.colors rules= code in GRASS 6.5 has something like that for backwards compatibility - it checks in two places.

also in grass7 that should be moved to ~/.grass/r.li/

The pathSetup string is allocated with lots of space:

pathSetup[GPATH_MAX]

so I don't think the problem is you are overflowing that buffer.

int parseSetup(char *path, list l, g_areas g, char *raster)
{
    struct stat s;
    struct Cell_head cellhd;
    char *buf, *token, *mapset;
    int setup;
    int letti;
...
    int size;

    if (stat(path, &s) != 0)
        G_fatal_error(_("Cannot make stat of %s configuration file"), path);
    size = s.st_size * sizeof(char);
    buf = G_malloc(size);
    setup = open(path, O_RDONLY, 0755);
    if (setup == -1)
        G_fatal_error(_("Cannot read setup file"));
    letti = read(setup, buf, s.st_size);
    if (letti < s.st_size)
        G_fatal_error(_("Cannot read setup file"));

probably that stat error message could be better worded..

so the G_fatal_error() in daemon.c does not kill the process?

ah, if I give it a garbage conf file name: conf=landclass96_conf.blah it goes nuts:

G65> time r.li.shannon map=landclass96 conf=landclass96_conf.blah output=landclass96_shannon --o
ERROR: Cannot make stat of
       /home/hamish/.r.li/history/landclass96_conf.blah configuration file

[--- It ''should'' quit everything at this point ---]

Illegal filename. Character <ÿ> not allowed.
Illegal filename. Character <ÿ> not allowed.
Illegal filename. Character <ÿ> not allowed.
WARNING: Unable to open header file for raster map <ÿ@(null)>
CHILD[pid = 12064]: unable to open ÿ mask ... continue without!!!

real    0m0.072s
user    0m0.000s
sys     0m0.004s
G65> Illegal filename. Character <ÿ> not allowed.
Illegal filename. Character <ÿ> not allowed.
Illegal filename. Character <ÿ> not allowed.
WARNING: Unable to open header file for raster map <ÿ@(null)>
CHILD[pid = 12063]: unable to open ÿ mask ... continue without!!!
Illegal filename. Character <ÿ> not allowed.
Illegal filename. Character <ÿ> not allowed.
Illegal filename. Character <ÿ> not allowed.
WARNING: Unable to open header file for raster map <ÿ@(null)>
CHILD[pid = 12062]: unable to open ÿ mask ... continue without!!!
...
*** glibc detected *** r.li.shannon: corrupted double-linked list: 0x0000000001103830 ***

Hamish

comment:4 Changed 10 years ago by pcav

Correct. I kind of remember that the bus has something to do with paths (it does not work for certain paths, it does with others), especially (only?) on 64bit

comment:5 in reply to:  3 Changed 10 years ago by neteler

CPU: x86-64All
Platform: MacOSXAll

Replying to hamish: ...

also in grass7 that should be moved to ~/.grass/r.li/

Done in r38708 (but it is ~/.grass7/r.li/ to avoid confusion).

...

so the G_fatal_error() in daemon.c does not kill the process?

ah, if I give it a garbage conf file name: conf=landclass96_conf.blah it goes nuts:

...

[--- It should quit everything at this point ---]

That's the point - it should not start/continue to run the workers when the config file isn't found. I tried but I have no clue how to achieve that.

Markus

comment:6 in reply to:  2 Changed 10 years ago by glynn

Replying to pcav:

Cannot make stat of /home/paolo/.r.li/history//home/paolo/.r.li/history/landclass96_conf configuration file

It appears that it always prepends ~/.r.li/history/ to the configuration file name, so conf= has to be a filename, not a full pathname.

comment:7 Changed 10 years ago by pcav

Right, sorry for not checking this. However, setting the right path does not help:

Unable to open header file for raster map <@(null)>
CHILD[pid = 6809]: unable to open  mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 6808]: unable to open  mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 6807]: unable to open  mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 6806]: unable to open  mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 6805]: unable to open  mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 6804]: unable to open  mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 6803]: unable to open  mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 6802]: unable to open  mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 6801]: unable to open  mask ... continue without!!!

Unable to open header file for raster map <@(null)>

CHILD[pid = 6800]: unable to open  mask ... continue without!!!

comment:8 in reply to:  7 ; Changed 10 years ago by neteler

Replying to pcav:

Right, sorry for not checking this. However, setting the right path does not help:

As mentioned above, the path should NOT be set. conf= has to be a filename, not a full pathname. Please always post the command line to better illustrate the problem (also in GUI mode available)

@devs: can we simply strip off the directory stuff?

Markus

comment:9 Changed 10 years ago by pcav

r.li.shannon map=landclass96@PERMANENT conf=landclass96_conf output=landclass96_shannon

comment:10 in reply to:  9 Changed 10 years ago by hamish

Just to clarify some points,

  • the original command Paolo gave (copied in comment:1) is not to the full path and the
    WARNING: Unable to open header file for raster map <@(null)>
    

errors appear without the failure to stat config file. Apparently failure to kill spawned workers on a fatal error is a coincidental bug.

  • it is not just debian/sid 64 bit, William sees it too on Mac OSX 64bit.
  • I don't see it on amd64 debain/lenny with a self-compiled 6.5svn. I haven't tried the debian packages there.
  • is there any MASK set or is that message just coming from no where?

H

comment:11 Changed 10 years ago by hamish

One thing it may be, that I haven't tried is if GRASS is built without the gcc -g debugging flag. I always build with that and AFAIU that means all variables are initizalied to '\0' (null). If there is a test that some empty variable == NULL, such as r.li.daemon/worker.c's if (ad->mask_name == NULL) { , and that variable is not explicitly made empty upon creation and then used uninitialized.... well, it's a theory.

H

comment:12 in reply to:  8 Changed 10 years ago by glynn

Replying to neteler:

As mentioned above, the path should NOT be set. conf= has to be a filename, not a full pathname. Please always post the command line to better illustrate the problem (also in GUI mode available)

@devs: can we simply strip off the directory stuff?

It would be better to only prepend ~/.r.li/ if the filename isn't an absolute path (see G_is_absolute_path()), or possibly if it doesn't contain any directory separators.

AFAIK, the GUI will see "old_file" in the ->gisprompt field and provide an absolute pathname. Either the ->gisprompt field needs to be removed (so it's just a string option), or the code needs to be changed to allow an absolute pathname. Or it could enumerate ~/.r.li/ and add all files found therein to the ->options field.

Certainly, it shouldn't silently discard any directory components; that would leave the user trying to figure out why it isn't behaving as expected.

comment:13 in reply to:  11 ; Changed 10 years ago by glynn

Replying to hamish:

One thing it may be, that I haven't tried is if GRASS is built without the gcc -g debugging flag. I always build with that and AFAIU that means all variables are initizalied to '\0' (null).

No. Static variables (global variables and local variables with the "static" qualifier") without an explicit initialiser are implicitly initialised to zero regardless of the compiler options used. This behaviour is mandated by the C standards. A static variable's initial value is stored in the binary, so it isn't possible for a static variable to be uninitialised.

[However, variables which are initialised to zero, explicitly or implicitly, are all stored in the BSS segment. As the entire segment contains only zeros, it can be "compressed" to nothing. Nonetheless, everything within that segment is initialised to zero at startup.]

Automatic variables (local variables lacking the "static" qualifier) without an explicit initialiser are normally uninitialised; I have seen compilers which will initialise such variables (not necessarily to zero) in debug mode, but I've never seen gcc do it (gcc's -g switch controls whether or not debug info is added to the output file; it doesn't affect code generation in any way, unlike e.g. MSVC where "debug mode" disables optimisation by default).

Also, enabling or disabling optimisations may affect what happens to be stored in a particular section of uninitialised memory, even if it doesn't affect whether or not initialisation occurs.

However:

If there is a test that some empty variable == NULL, such as r.li.daemon/worker.c's if (ad->mask_name == NULL) { , and that variable is not explicitly made empty upon creation and then used uninitialized.... well, it's a theory.

ad is a pointer to a structure which is allocated with G_malloc(). This uses malloc, which doesn't normally initialise memory (although, again, some implementations will initialise malloc()d memory in debug mode, but not usually to zero).

comment:14 in reply to:  13 Changed 10 years ago by hamish

Replying to glynn:

Automatic variables (local variables lacking the "static" qualifier) without an explicit initialiser are normally uninitialised; I have seen compilers which will initialise such variables (not necessarily to zero) in debug mode, but I've never seen gcc do it (gcc's -g switch controls whether or not debug info is added to the output file; it doesn't affect code generation in any way, unlike e.g. MSVC where "debug mode" disables optimisation by default).

...

ad is a pointer to a structure which is allocated with G_malloc(). This uses malloc, which doesn't normally initialise memory (although, again, some implementations will initialise malloc()d memory in debug mode, but not usually to zero).

FWIW, my memory was from G_command_history() or somewhere where I used strncpy() (which unlike strcpy() doesn't always null terminate) and for me (on Debian) there was no problem as the memory space was all 0s to begin with & so I didn't notice it. On Markus's system this wasn't true and it took a while to figure out until I darkly remembered hearing about this before. It may well be a Debianism.

but anyway, back to the bug, it would seem that worker.c's mask_preprocessing() should return NULL for ad->mask_name if G_get_cellhd() fails, but I'm unable to reproduce the problem, so...

Hamish

comment:15 Changed 10 years ago by hamish

ok, I can reproduce this now. The conf file above doesn't trigger it because it sets up a moving window which covers the whole map. You need to use r.li.setup to make a subsampled window to have it happen. (???)

NC sample dataset / 'g.region rast=landclass96' ~/.r.li/history/landclass96_conf2

SAMPLINGFRAME 0.311787072243|0.18776371308|0.341772151899|0.376425855513
SAMPLEAREA 0.366920152091|0.259493670886|0.0443037974684|0.0361216730038
SAMPLEAREA 0.587452471483|0.42194092827|0.0443037974684|0.0437262357414
SAMPLEAREA 0.32319391635|0.42194092827|0.0886075949367|0.041825095057

causing

G65> r.li.shannon map=landclass96 conf=landclass96_conf2 output=landclass96_shannon --o --v
*** glibc detected *** malloc(): memory corruption: 0x09166940 ***
D0/0: daemon while loop: toReceive.type=-1208818294
Illegal filename. Character <   > not allowed.
Illegal filename. Character <   > not allowed.
Illegal filename. Character <   > not allowed.
WARNING: Unable to open header file for raster map < @(null)>
CHILD[pid = 20559]: unable to open < > mask ... continuing without!
Aborted

(I added that debug msg locally)

code updated in 6.5svn and 7svn to fail more gracefully if the type value is garbage. (in r38757 and r38758)

hmmph but now that I try gdb that memory has something else in it which doesn't trigger the breakage. I'll have to come back to it later.


I suspect there may be some bad inefficency in that worker.c receive() while loop, perhaps explaining the 50% of proc time spent in the kernel for at least the moving window conf file (found in comment:3).

perhaps the alloc()s within each iteration of the loop? especially for the moving window mode which will cycle through the loop many, many times. And no G_free()?


also I notice that the ~/.r.li/output/conf_filename file is created with the exe bit set for some reason?!:

daemon.c:138:   sprintf(out, "%s/.r.li/output/%s", getenv("HOME"), output);
daemon.c-139:   res = open(out, O_WRONLY | O_CREAT | O_TRUNC, 0755);

should that be 0644 or better yet just left to the system? (how, by just removing the 0755?)

Hamish

comment:16 Changed 10 years ago by hamish

Keywords: smp added

FWIW I'd note that I got this error on a 32bit Pentium4 running Debian/etch:

*** glibc detected *** malloc(): memory corruption: 0x09166940 ***
D0/0: daemon while loop: toReceive.type=-1208818294
Illegal filename. Character <   > not allowed.
...

ok, with the help of Nathan's backtrace in today's grass-user post I tracked it down to this:

source:grass/trunk/raster/r.li/r.li.daemon/daemon.c@38766#L397

i.e.:

	} while ((token = strtok(NULL, " ")) != NULL &&
	       strcmp(token, "SAMPLEAREA") == 0);

note the glibc man page for strtok() says:

BUGS
       Avoid using these functions.  If you do use them, note that:

              These functions modify their first argument.

              These functions cannot be used on constant strings.

              The identity of the delimiting character is lost.

              The strtok() function uses a static buffer while  parsing,
              so it's not thread safe. Use strtok_r() if this matters to
              you.

RETURN VALUE
       The strtok() and strtok_r() functions return  a  pointer  to  the
       next token, or NULL if there are no more tokens.

CONFORMING TO
       strtok()
              SVr4, POSIX.1-2001, 4.3BSD, C89.

       strtok_r()
              POSIX.1-2001

i.e. strtok() is not thread safe and it is being used in a thread. The result is breakage. And strtok_r() is not as portable as we require.

in light of that, could our G_tokenize() help? AFAICS this is just for simple plain text string parsing stuff, so no huge drama to work around it.

Maybe the landclass96_conf above doesn't break for me because it only has 1 line with a space in it?

Hamish

Changed 10 years ago by glynn

Attachment: r.li.daemon-strtok.patch added

replace strtok with strtok_r

comment:17 in reply to:  16 Changed 10 years ago by glynn

Replying to hamish:

ok, with the help of Nathan's backtrace in today's grass-user post I tracked it down to this:

source:grass/trunk/raster/r.li/r.li.daemon/daemon.c@38766#L397

i.e. strtok() is not thread safe and it is being used in a thread. The result is breakage. And strtok_r() is not as portable as we require.

strtok_r() is specified by POSIX. It doesn't exist on Windows (although Windows does have strtok_s(), which appears to be identical except for the name), but r.li doesn't work on Windows for more substantial reasons (e.g. fork()).

I've attached a patch (against current 7.0) to use strtok_r() instead of strtok().

in light of that, could our G_tokenize() help?

G_tokenize() uses a single set of delimiters, while the r.li code uses vertical bar, space and newline. It may be possible to first split the text into lines with G_tokenize(str, "\n"), then into fields with vertical bar and/or space; it depends upon the structure of the data.

comment:18 in reply to:  16 ; Changed 10 years ago by glynn

Replying to hamish:

i.e. strtok() is not thread safe and it is being used in a thread.

Huh? AFAICT, r.li doesn't use threads, it uses multiple processes.

comment:19 in reply to:  18 ; Changed 10 years ago by hamish

Replying to hamish:

i.e. strtok() is not thread safe and it is being used in a thread.

Replying to glynn:

Huh? AFAICT, r.li doesn't use threads, it uses multiple processes.

um, sorry about that. my bad... ok, so it's not a thread-safe issue and strtok() has been unjustly blamed; and the change to strtok_r() doesn't help. I still can't find any joy with gdb so resorting to sticking printf()s everywhere. With that I can see that it is falling over in r.li.daemon/list.c's insertNode() on the first malloc() on the second time that function is called.

source:grass/trunk/raster/r.li/r.li.daemon/list.c@38777#L32

void insertNode(list l, msg mess)
{
   node new;
   new = G_malloc(sizeof(node));
...

size(node) is 4 on the pass where it dies.

unfortunately I'm guessing that just means that the memory is already corrupt by the time it gets there. :(

which puts us back a step in the daemon.c SAMPLEAREA while loop:

source:grass/trunk/raster/r.li/r.li.daemon/daemon.c@38777#L367

If I pepper a bunch of test G_malloc()s here and there in that while loop it succeeds right up to the point -on the first pass- where insertNode() is called within the else{}. A G_malloc() just after that causes glibc to report the error.

Valgrind finds a whole pile of errors:

G65:nc_spm_08> CMD="r.li.shannon map=landclass96 conf=landclass96_conf2 output=landclass96_shannon"

G65:nc_spm_08> valgrind --tool=memcheck $CMD

==4276== Memcheck, a memory error detector.
==4276== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al.
==4276== Using LibVEX rev 1658, a library for dynamic binary translation.
==4276== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP.
==4276== Using valgrind-3.2.1-Debian, a dynamic binary instrumentation framework.
==4276== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al.
==4276== For more details, rerun with: -v
==4276== 
==4276== Invalid write of size 4
==4276==    at 0x40B445E: insertNode (list.c:32)
==4276==    by 0x40B2DCD: parseSetup (daemon.c:399)
==4276==    by 0x40B1DF3: calculateIndex (daemon.c:106)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276==  Address 0x4217840 is 4 bytes after a block of size 4 alloc'd
==4276==    at 0x401D38B: malloc (vg_replace_malloc.c:149)
==4276==    by 0x402EB8D: G__malloc (alloc.c:41)
==4276==    by 0x40B4437: insertNode (list.c:31)
==4276==    by 0x40B2DCD: parseSetup (daemon.c:399)
==4276==    by 0x40B1DF3: calculateIndex (daemon.c:106)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276== 
==4276== Invalid read of size 4
==4276==    at 0x40B446A: insertNode (list.c:35)
==4276==    by 0x40B2DCD: parseSetup (daemon.c:399)
==4276==    by 0x40B1DF3: calculateIndex (daemon.c:106)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276==  Address 0x4217840 is 4 bytes after a block of size 4 alloc'd
==4276==    at 0x401D38B: malloc (vg_replace_malloc.c:149)
==4276==    by 0x402EB8D: G__malloc (alloc.c:41)
==4276==    by 0x40B4437: insertNode (list.c:31)
==4276==    by 0x40B2DCD: parseSetup (daemon.c:399)
==4276==    by 0x40B1DF3: calculateIndex (daemon.c:106)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276== 
==4276== Invalid write of size 4
==4276==    at 0x40B4498: insertNode (list.c:36)
==4276==    by 0x40B2DCD: parseSetup (daemon.c:399)
==4276==    by 0x40B1DF3: calculateIndex (daemon.c:106)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276==  Address 0x421783C is 0 bytes after a block of size 4 alloc'd
==4276==    at 0x401D38B: malloc (vg_replace_malloc.c:149)
==4276==    by 0x402EB8D: G__malloc (alloc.c:41)
==4276==    by 0x40B4437: insertNode (list.c:31)
==4276==    by 0x40B2DCD: parseSetup (daemon.c:399)
==4276==    by 0x40B1DF3: calculateIndex (daemon.c:106)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276== 
==4276== Conditional jump or move depends on uninitialised value(s)
==4276==    at 0x40B44A2: insertNode (list.c:38)
==4276==    by 0x40B2DCD: parseSetup (daemon.c:399)
==4276==    by 0x40B1DF3: calculateIndex (daemon.c:106)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276== 
==4276== Invalid write of size 4
==4276==    at 0x40B44C3: insertNode (list.c:42)
==4276==    by 0x40B2DCD: parseSetup (daemon.c:399)
==4276==    by 0x40B1DF3: calculateIndex (daemon.c:106)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276==  Address 0x421783C is 0 bytes after a block of size 4 alloc'd
==4276==    at 0x401D38B: malloc (vg_replace_malloc.c:149)
==4276==    by 0x402EB8D: G__malloc (alloc.c:41)
==4276==    by 0x40B4437: insertNode (list.c:31)
==4276==    by 0x40B2DCD: parseSetup (daemon.c:399)
==4276==    by 0x40B1DF3: calculateIndex (daemon.c:106)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276== 
==4276== Invalid read of size 1
==4276==    at 0x4149FE7: strtok (in /lib/tls/i686/cmov/libc-2.3.6.so)
==4276==    by 0x40B1DF3: calculateIndex (daemon.c:106)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276==  Address 0x4216B40 is 0 bytes after a block of size 288 alloc'd
==4276==    at 0x401D38B: malloc (vg_replace_malloc.c:149)
==4276==    by 0x402EB8D: G__malloc (alloc.c:41)
==4276==    by 0x40B2734: parseSetup (daemon.c:319)
==4276==    by 0x40B1DF3: calculateIndex (daemon.c:106)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276== 
==4276== Invalid read of size 1
==4276==    at 0x414A00F: strtok (in /lib/tls/i686/cmov/libc-2.3.6.so)
==4276==    by 0x40B1DF3: calculateIndex (daemon.c:106)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276==  Address 0x4216B40 is 0 bytes after a block of size 288 alloc'd
==4276==    at 0x401D38B: malloc (vg_replace_malloc.c:149)
==4276==    by 0x402EB8D: G__malloc (alloc.c:41)
==4276==    by 0x40B2734: parseSetup (daemon.c:319)
==4276==    by 0x40B1DF3: calculateIndex (daemon.c:106)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276== 
==4276== Conditional jump or move depends on uninitialised value(s)
==4276==    at 0x40B3F6E: next_Area (daemon.c:640)
==4276==    by 0x40B1FD2: calculateIndex (daemon.c:147)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276== 
==4276== Invalid read of size 4
==4276==    at 0x40B3F81: next_Area (daemon.c:645)
==4276==    by 0x40B1FD2: calculateIndex (daemon.c:147)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276==  Address 0x4217840 is 4 bytes after a block of size 4 alloc'd
==4276==    at 0x401D38B: malloc (vg_replace_malloc.c:149)
==4276==    by 0x402EB8D: G__malloc (alloc.c:41)
==4276==    by 0x40B4437: insertNode (list.c:31)
==4276==    by 0x40B2DCD: parseSetup (daemon.c:399)
==4276==    by 0x40B1DF3: calculateIndex (daemon.c:106)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276== 
==4276== Invalid read of size 4
==4276==    at 0x40B451F: removeNode (list.c:63)
==4276==    by 0x40B3FCB: next_Area (daemon.c:647)
==4276==    by 0x40B1FD2: calculateIndex (daemon.c:147)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276==  Address 0x421783C is 0 bytes after a block of size 4 alloc'd
==4276==    at 0x401D38B: malloc (vg_replace_malloc.c:149)
==4276==    by 0x402EB8D: G__malloc (alloc.c:41)
==4276==    by 0x40B4437: insertNode (list.c:31)
==4276==    by 0x40B2DCD: parseSetup (daemon.c:399)
==4276==    by 0x40B1DF3: calculateIndex (daemon.c:106)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276== 
==4276== Invalid read of size 4
==4276==    at 0x40B4560: removeNode (list.c:73)
==4276==    by 0x40B3FCB: next_Area (daemon.c:647)
==4276==    by 0x40B1FD2: calculateIndex (daemon.c:147)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276==  Address 0x421783C is 0 bytes after a block of size 4 alloc'd
==4276==    at 0x401D38B: malloc (vg_replace_malloc.c:149)
==4276==    by 0x402EB8D: G__malloc (alloc.c:41)
==4276==    by 0x40B4437: insertNode (list.c:31)
==4276==    by 0x40B2DCD: parseSetup (daemon.c:399)
==4276==    by 0x40B1DF3: calculateIndex (daemon.c:106)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276== 
==4276== Syscall param write(buf) points to uninitialised byte(s)
==4276==    at 0x4000792: (within /lib/ld-2.3.6.so)
==4276==    by 0x40B1FA0: calculateIndex (daemon.c:148)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276==  Address 0xBED61FFC is on thread 1's stack
==4276== 
==4276== Conditional jump or move depends on uninitialised value(s)
==4276==    at 0x40B3F6E: next_Area (daemon.c:640)
==4276==    by 0x40B2198: calculateIndex (daemon.c:154)
==4276==    by 0x8048D12: main (shannon.c:58)
==4277== Conditional jump or move depends on uninitialised value(s)
==4277==    at 0x40B5381: RLI_get_cell_raster_row (worker.c:279)
==4277==    by 0x8048F51: calculate (shannon.c:167)
==4277==    by 0x8048DBF: shannon (shannon.c:84)
==4277==    by 0x40B500B: worker (worker.c:192)
==4277==    by 0x40B1D52: calculateIndex (daemon.c:91)
==4277==    by 0x8048D12: main (shannon.c:58)
====4279== Conditional jump or move depends on uninitialised value(s)
==4279==    at 0x40B5381: RLI_get_cell_raster_row (worker.c:279)
==4279==    by 0x8048F51: calculate (shannon.c:167)
==4279==    by 0x8048DBF: shannon (shannon.c:84)
==4279==    by 0x40B500B: worker (worker.c:192)
==4279==    by 0x40B1D52: calculateIndex (daemon.c:91)
==4279==    by 0x8048D12: main (shannon.c:58)
4278== Conditional jump or move depends on uninitialised value(s)
==4278==    at 0x40B5381: RLI_get_cell_raster_row (worker.c:279)
==4278==    by 0x8048F51: calculate (shannon.c:167)
==4278==    by 0x8048DBF: shannon (shannon.c:84)
==4278==    by 0x40B500B: worker (worker.c:192)
==4278==    by 0x40B1D52: calculateIndex (daemon.c:91)
==4278==    by 0x8048D12: main (shannon.c:58)
====4278== 
==4278== Syscall param write(buf) points to uninitialised byte(s)
==4278==    at 0x4000792: (within /lib/ld-2.3.6.so)
==4278==    by 0x40B506F: worker (worker.c:206)
==4278==    by 0x40B1D52: calculateIndex (daemon.c:91)
==4278==    by 0x8048D12: main (shannon.c:58)
==4278==  Address 0xBED61AF8 is on thread 1's stack
4277== 
==4277== Syscall param write(buf) points to uninitialised byte(s)
==4277==    at 0x4000792: (within /lib/ld-2.3.6.so)
==4277==    by 0x40B506F: worker (worker.c:206)
==4277==    by 0x40B1D52: calculateIndex (daemon.c:91)
==4277==    by 0x8048D12: main (shannon.c:58)
==4277==  Address 0xBED61AF8 is on thread 1's stack
==4276== 
==4276== Syscall param write(buf) points to uninitialised byte(s)
==4276==    at 0x4000792: (within /lib/ld-2.3.6.so)
==4276==    by 0x40B2354: calculateIndex (daemon.c:230)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276==  Address 0xBED61FFC is on thread 1's stack
==4279== 
==4279== Syscall param write(buf) points to uninitialised byte(s)
==4279==    at 0x4000792: (within /lib/ld-2.3.6.so)
==4279==    by 0x40B506F: worker (worker.c:206)
==4279==    by 0x40B1D52: calculateIndex (daemon.c:91)
==4279==    by 0x8048D12: main (shannon.c:58)
==4279==  Address 0xBED61AF8 is on thread 1's stack
==4278== 
==4278== ERROR SUMMARY: 21 errors from 2 contexts (suppressed: 23 from 1)
==4278== malloc/free: in use at exit: 83,775 bytes in 90 blocks.
==4278== malloc/free: 280 allocs, 190 frees, 101,718 bytes allocated.
==4278== For counts of detected errors, rerun with: -v
==4278== searching for pointers to 90 not-freed blocks.
==4278== checked 135,736 bytes.
==4278== 
==4278== LEAK SUMMARY:
==4278==    definitely lost: 60,587 bytes in 45 blocks.
==4278==      possibly lost: 0 bytes in 0 blocks.
==4278==    still reachable: 23,188 bytes in 45 blocks.
==4278==         suppressed: 0 bytes in 0 blocks.
==4278== Use --leak-check=full to see details of leaked memory.
==4277== 
==4277== ERROR SUMMARY: 21 errors from 2 contexts (suppressed: 23 from 1)
==4277== malloc/free: in use at exit: 83,165 bytes in 87 blocks.
==4277== malloc/free: 275 allocs, 188 frees, 101,034 bytes allocated.
==4277== For counts of detected errors, rerun with: -v
==4277== searching for pointers to 87 not-freed blocks.
==4277== checked 135,124 bytes.
==4277== 
==4277== LEAK SUMMARY:
==4277==    definitely lost: 60,543 bytes in 43 blocks.
==4277==      possibly lost: 0 bytes in 0 blocks.
==4277==    still reachable: 22,622 bytes in 44 blocks.
==4277==         suppressed: 0 bytes in 0 blocks.
==4277== Use --leak-check=full to see details of leaked memory.
==4279== 
==4279== ERROR SUMMARY: 42 errors from 2 contexts (suppressed: 23 from 1)
==4279== malloc/free: in use at exit: 128,639 bytes in 115 blocks.
==4279== malloc/free: 337 allocs, 222 frees, 148,422 bytes allocated.
==4279== For counts of detected errors, rerun with: -v
==4279== searching for pointers to 115 not-freed blocks.
==4279== checked 138,420 bytes.
==4279== 
==4279== LEAK SUMMARY:
==4279==    definitely lost: 102,777 bytes in 68 blocks.
==4279==      possibly lost: 2,108 bytes in 1 blocks.
==4279==    still reachable: 23,754 bytes in 46 blocks.
==4279==         suppressed: 0 bytes in 0 blocks.
==4279== Use --leak-check=full to see details of leaked memory.
==4276== 
==4276== Syscall param write(buf) points to uninitialised byte(s)
==4276==    at 0x4000792: (within /lib/ld-2.3.6.so)
==4276==    by 0x40B24A4: calculateIndex (daemon.c:255)
==4276==    by 0x8048D12: main (shannon.c:58)
==4276==  Address 0xBED61FFC is on thread 1's stack
==4280== 
==4280== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 23 from 1)
==4280== malloc/free: in use at exit: 40,433 bytes in 60 blocks.
==4280== malloc/free: 203 allocs, 143 frees, 55,540 bytes allocated.
==4280== For counts of detected errors, rerun with: -v
==4280== searching for pointers to 60 not-freed blocks.
==4280== checked 136,096 bytes.
==4280== 
==4280== LEAK SUMMARY:
==4280==    definitely lost: 16,113 bytes in 13 blocks.
==4280==      possibly lost: 0 bytes in 0 blocks.
==4280==    still reachable: 24,320 bytes in 47 blocks.
==4280==         suppressed: 0 bytes in 0 blocks.
==4280== Use --leak-check=full to see details of leaked memory.
==4282== 
==4282== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 23 from 1)
==4282== malloc/free: in use at exit: 40,999 bytes in 61 blocks.
==4282== malloc/free: 206 allocs, 145 frees, 56,164 bytes allocated.
==4282== For counts of detected errors, rerun with: -v
==4282== searching for pointers to 61 not-freed blocks.
==4282== checked 136,672 bytes.
==4282== 
==4282== LEAK SUMMARY:
==4282==    definitely lost: 16,113 bytes in 13 blocks.
==4282==      possibly lost: 0 bytes in 0 blocks.
==4282==    still reachable: 24,886 bytes in 48 blocks.
==4282==         suppressed: 0 bytes in 0 blocks.
==4282== Use --leak-check=full to see details of leaked memory.
==4284== 
==4284== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 23 from 1)
==4284== malloc/free: in use at exit: 41,565 bytes in 62 blocks.
==4284== malloc/free: 209 allocs, 147 frees, 56,788 bytes allocated.
==4284== For counts of detected errors, rerun with: -v
==4284== searching for pointers to 62 not-freed blocks.
==4284== checked 137,248 bytes.
==4284== 
==4284== LEAK SUMMARY:
==4284==    definitely lost: 16,113 bytes in 13 blocks.
==4284==      possibly lost: 0 bytes in 0 blocks.
==4284==    still reachable: 25,452 bytes in 49 blocks.
==4284==         suppressed: 0 bytes in 0 blocks.
==4284== Use --leak-check=full to see details of leaked memory.
==4285== 
==4285== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 23 from 1)
==4285== malloc/free: in use at exit: 42,131 bytes in 63 blocks.
==4285== malloc/free: 212 allocs, 149 frees, 57,412 bytes allocated.
==4285== For counts of detected errors, rerun with: -v
==4285== searching for pointers to 63 not-freed blocks.
==4285== checked 137,824 bytes.
==4285== 
==4285== LEAK SUMMARY:
==4285==    definitely lost: 16,113 bytes in 13 blocks.
==4285==      possibly lost: 0 bytes in 0 blocks.
==4285==    still reachable: 26,018 bytes in 50 blocks.
==4285==         suppressed: 0 bytes in 0 blocks.
==4285== Use --leak-check=full to see details of leaked memory.
==4286== 
==4286== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 23 from 1)
==4286== malloc/free: in use at exit: 42,697 bytes in 64 blocks.
==4286== malloc/free: 215 allocs, 151 frees, 58,036 bytes allocated.
==4286== For counts of detected errors, rerun with: -v
==4286== searching for pointers to 64 not-freed blocks.
==4286== checked 138,400 bytes.
==4286== 
==4286== LEAK SUMMARY:
==4286==    definitely lost: 16,113 bytes in 13 blocks.
==4286==      possibly lost: 0 bytes in 0 blocks.
==4286==    still reachable: 26,584 bytes in 51 blocks.
==4286==         suppressed: 0 bytes in 0 blocks.
==4286== Use --leak-check=full to see details of leaked memory.
==4287== 
==4287== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 23 from 1)
==4287== malloc/free: in use at exit: 43,263 bytes in 65 blocks.
==4287== malloc/free: 218 allocs, 153 frees, 58,660 bytes allocated.
==4287== For counts of detected errors, rerun with: -v
==4287== searching for pointers to 65 not-freed blocks.
==4287== checked 138,976 bytes.
==4287== 
==4287== LEAK SUMMARY:
==4287==    definitely lost: 16,113 bytes in 13 blocks.
==4287==      possibly lost: 0 bytes in 0 blocks.
==4287==    still reachable: 27,150 bytes in 52 blocks.
==4287==         suppressed: 0 bytes in 0 blocks.
==4287== Use --leak-check=full to see details of leaked memory.
==4288== 
==4288== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 23 from 1)
==4288== malloc/free: in use at exit: 43,830 bytes in 66 blocks.
==4288== malloc/free: 221 allocs, 155 frees, 59,285 bytes allocated.
==4288== For counts of detected errors, rerun with: -v
==4288== searching for pointers to 66 not-freed blocks.
==4288== checked 139,552 bytes.
==4288== 
==4288== LEAK SUMMARY:
==4288==    definitely lost: 16,113 bytes in 13 blocks.
==4288==      possibly lost: 0 bytes in 0 blocks.
==4288==    still reachable: 27,717 bytes in 53 blocks.
==4288==         suppressed: 0 bytes in 0 blocks.
==4288== Use --leak-check=full to see details of leaked memory.
==4276== 
==4276== ERROR SUMMARY: 40 errors from 15 contexts (suppressed: 23 from 1)
==4276== malloc/free: in use at exit: 2,489 bytes in 53 blocks.
==4276== malloc/free: 117 allocs, 64 frees, 5,458 bytes allocated.
==4276== For counts of detected errors, rerun with: -v
==4276== searching for pointers to 53 not-freed blocks.
==4276== checked 122,368 bytes.
==4276== 
==4276== LEAK SUMMARY:
==4276==    definitely lost: 1,816 bytes in 18 blocks.
==4276==      possibly lost: 0 bytes in 0 blocks.
==4276==    still reachable: 673 bytes in 35 blocks.
==4276==         suppressed: 0 bytes in 0 blocks.
==4276== Use --leak-check=full to see details of leaked memory.

(11 error summaries from 10 workers and 1 master process)

Hamish

comment:20 Changed 10 years ago by hamish

I added some code to strip away the "$HOME/r.li/history/" part of the config file name, if present.

I notice then if you give the full path to the config file on the command line it doesn't crash with the glibc error. I suppose giving the long path name slightly increases the memory allotment allowing the program to just trash its own memory space and not that of its neighbors? shrug

H

comment:21 in reply to:  19 Changed 10 years ago by glynn

Replying to hamish:

I still can't find any joy with gdb so resorting to sticking printf()s everywhere. With that I can see that it is falling over in r.li.daemon/list.c's insertNode() on the first malloc() on the second time that function is called.

source:grass/trunk/raster/r.li/r.li.daemon/list.c@38777#L32

unfortunately I'm guessing that just means that the memory is already corrupt by the time it gets there. :(

That's what it looks like.

Oh, look at insertNode() in list.c:

void insertNode(list l, msg mess)
{
    node new;

    new = malloc(sizeof(node));
    new->m = malloc(sizeof(msg));

Even without reading what follows, the above should smell fishy. Don't malloc() calls usually look like "TYPE *x = malloc(N * sizeof(TYPE))"? I.e. shouldn't the type of the return value be a pointer to whatever type appears in the sizeof?

Note: list.h says:

typedef struct nodoLista *node;

So "node" is a typedef for a pointer to the structure, so the malloc() is only allocating enough memory for the pointer, not the structure.

This is just one reason why sane programmers very rarely use typedefs, and almost never for pointer types.

[They may sometimes be used to hide implementation details in public interfaces, e.g. "typedef void *HANDLE", but the actual type would normally be used in the implementation.]

For the sake of anyone who has to clear up any other mistakes in r.li (given the overall quality, I doubt that this is the only one), I would ask whoever fixes this to finish the job and eliminate the typedefs altogether. If something is a pointer to a structure, it should look like "struct foo *p", not "foo p".

Oh, and bonus points for changing the type names to English, like the other 99.99% of the GRASS codebase.

comment:22 Changed 10 years ago by neteler

I guess I found it finally. Try r41560 (trunk), r415599 (6.5), r415598 (6.4).

Markus

comment:23 Changed 9 years ago by neteler

Is this still an issue on MacOSX 64bit? Can it be closed?

comment:24 Changed 9 years ago by kyngchaos

I had trouble remembering what I did for the r.li.setup part. I finally downloaded the NC data to try hamish's conf's.

The first one works fine, but hamish didn't have a problem with that either.

The next one gives an error (all jumbled from the multiple workers) and hangs (must ctl-c to exit):

> r.li.shannon map=landclass96 conf=test_conf2 out=test_shannon2
ERROR: ERROR: ProgramERROR:  error,Program worker()Program  toReceive.type=0 error,
error,  worker()worker()  toReceive.type=24184648toReceive.type=0

In 32bit mode, it finishes with no error, but there is also no output.

Patch density using the first conf seems to work in both 64 and 32bit modes, though there are bands of what looks like noise, in different places between 32 and 64bit. But I get the same error/hang on 64bit and no output on 32bit using the other conf.

Changed 9 years ago by kyngchaos

Attachment: patchnoise.png added

patchdensity noise bands 64bit (in different locations for 32bit)

comment:25 in reply to:  24 Changed 8 years ago by lucadelu

Milestone: 6.4.06.4.3
Priority: normalmajor
Version: 6.4.0 RCs6.4.2 RCs

Replying to kyngchaos:

I had trouble remembering what I did for the r.li.setup part. I finally downloaded the NC data to try hamish's conf's.

The first one works fine, but hamish didn't have a problem with that either.

The next one gives an error (all jumbled from the multiple workers) and hangs (must ctl-c to exit):

> r.li.shannon map=landclass96 conf=test_conf2 out=test_shannon2
ERROR: ERROR: ProgramERROR:  error,Program worker()Program  toReceive.type=0 error,
error,  worker()worker()  toReceive.type=24184648toReceive.type=0

I have the same problem with 6.4.2 to test it I used nc, with the attached landsat_test file, copy it into ~/.r.li/history and after launch

r.li.shannon map=lsat7_2002_40 conf=landsat_test out=landsat_shan
Segmentation fault
ERROR: Program error, worker() toReceive.type=0
ERROR: Program error, worker() toReceive.type=0
ERROR: Program error, worker() toReceive.type=0
ERROR: Program error, worker() toReceive.type=0
ERROR: Program error, worker() toReceive.type=0
ERROR: Program error, worker() toReceive.type=0
ERROR: Program error, worker() toReceive.type=0
ERROR: Program error, worker() toReceive.type=0
ERROR: Program error, worker() toReceive.type=0

Changed 8 years ago by lucadelu

Attachment: landsat_test added

configuration file for r.li.shannon test

comment:26 Changed 8 years ago by mmetz

At least the segfaults should be fixed in all branches as of r49886.

Markus M

comment:27 Changed 8 years ago by neteler

Thanks for the substantial changes.

On Fedora 16/64bit, I face the strange problem what the workers do not fork:

GRASS 6.4.2svn (nc_spm_08):~/grass64 > g.region rast=lsat7_2002_40 -p
projection: 99 (Lambert Conformal Conic)
zone:       0
datum:      nad83
ellipsoid:  a=6378137 es=0.006694380022900787
north:      228513
south:      214975.5
west:       629992.5
east:       645012
nsres:      28.5
ewres:      28.5
rows:       475
cols:       527
cells:      250325
GRASS 6.4.2svn (nc_spm_08):~/grass64 > time -p r.li.shannon map=lsat7_2002_40 conf=landsat_test out=landsat_shan
real 0.08
user 0.06
sys 0.01

In GRASS 7 the same:

GRASS 7.0.svn (nc_spm_08):~/grass70 > time -p r.li.shannon in=lsat7_2002_40 conf=landsat_test out=landsat_shan
real 0.04
user 0.00
sys 0.00

GRASS 7.0.svn (nc_spm_08):~/grass70 > uname -a
Linux north.localdomain 3.1.5-6.fc16.x86_64 #1 SMP Thu Dec 15 16:14:44 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

I tried with "ddd" and found in daemon.c, line 85, "parsed" is '\001', hence the rest fails.

comment:28 in reply to:  27 Changed 8 years ago by mmetz

Replying to neteler:

On Fedora 16/64bit, I face the strange problem what the workers do not fork:

[snip]

GRASS 7.0.svn (nc_spm_08):~/grass70 > time -p r.li.shannon in=lsat7_2002_40 conf=landsat_test out=landsat_shan
real 0.04
user 0.00
sys 0.00

GRASS 7.0.svn (nc_spm_08):~/grass70 > uname -a
Linux north.localdomain 3.1.5-6.fc16.x86_64 #1 SMP Thu Dec 15 16:14:44 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

I tried with "ddd" and found in daemon.c, line 85, "parsed" is '\001', hence the rest fails.

parsed is 1 and must be interpreted as a number, not as a char.

Can not reproduce, for both 6.4 and 7 (r49887), a result is produced. The content of the file landsat_shan is

RESULT 1|4.390990

Note the different locations of the output file for 6 and 7.

Markus M

comment:29 Changed 6 years ago by neteler

Milestone: 6.4.36.4.4
Version: 6.4.2 RCssvn-releasebranch64

I retested it with current 6.4.svn:

g.region rast=lsat7_2002_40 -p

mkdir -p $HOME/.r.li/history/
echo "SAMPLINGFRAME 0|0|1|1
SAMPLEAREA 0.0|0.0|1.0|1.0" > $HOME/.r.li/history/landsat_test

r.li.shannon map=lsat7_2002_40 conf=landsat_test out=landsat_shannon
r.univar landsat_shannon
ERROR: Raster map <landsat_shannon> not found

... no output is produced by r.li.shannon.

comment:30 in reply to:  29 ; Changed 6 years ago by mmetz

Replying to neteler:

I retested it with current 6.4.svn:

g.region rast=lsat7_2002_40 -p

mkdir -p $HOME/.r.li/history/
echo "SAMPLINGFRAME 0|0|1|1
SAMPLEAREA 0.0|0.0|1.0|1.0" > $HOME/.r.li/history/landsat_test

r.li.shannon map=lsat7_2002_40 conf=landsat_test out=landsat_shannon
r.univar landsat_shannon
ERROR: Raster map <landsat_shannon> not found

... no output is produced by r.li.shannon.

With this configuration, the output is a text file. The r.li daemon should have printed a corresponding message with the path to the output file.

comment:31 in reply to:  30 Changed 6 years ago by neteler

Resolution: fixed
Status: newclosed

Replying to mmetz:

Replying to neteler:

I retested it with current 6.4.svn:

g.region rast=lsat7_2002_40 -p

mkdir -p $HOME/.r.li/history/
echo "SAMPLINGFRAME 0|0|1|1
SAMPLEAREA 0.0|0.0|1.0|1.0" > $HOME/.r.li/history/landsat_test

r.li.shannon map=lsat7_2002_40 conf=landsat_test out=landsat_shannon
r.univar landsat_shannon
ERROR: Raster map <landsat_shannon> not found

... no output is produced by r.li.shannon.

With this configuration, the output is a text file. The r.li daemon should have printed a corresponding message with the path to the output file.

It now does in the backported, corrected version:

r.li.shannon map=lsat7_2002_40 conf=landsat_test out=landsat_shannon
Result written to ASCII file </home/neteler/.r.li/output/landsat_shannon>

The original issue reported here should also be fixed in the new version.

Closing, feel free top reopen if needed.

Note: See TracTickets for help on using tickets.