r.colors is lacking color bars for all predefined colortables

[milenan]

In wxGUI dialog of r.colors drop down list the color bars are missing under Windows

I discovered in ​https://wingrass.fsv.cvut.cz/grass72/x86_64/logs/log-r69771-56/package.log the following bug:

mingw32/lib:/usr/bin:/mingw64/bin/:/c/OSGeo4W64/bin:/c/msys64/usr/src/grass72_release/mswindows/osgeo4w/lib" LC_ALL=C  /c/msys64/usr/src/grass72_release/tools/thumbnails.py
WARNING: No data base element files found
Traceback (most recent call last):
  File "C:/msys64/usr/src/grass72_release/tools/thumbnails.py", line 206, in <module>
    main()
  File "C:/msys64/usr/src/grass72_release/tools/thumbnails.py", line 198, in main
    make_image(output_dir, table, grad)
  File "C:/msys64/usr/src/grass72_release/tools/thumbnails.py", line 157, in make_image
    convert_and_rotate(tmp_img, outfile, discrete)
  File "C:/msys64/usr/src/grass72_release/tools/thumbnails.py", line 105, in convert_and_rotate
    ppmtopng(dst, tmp_img)
  File "C:/msys64/usr/src/grass72_release/tools/thumbnails.py", line 89, in ppmtopng
    grass.run_command('g.ppmtopng', input = src, output = dst, quiet = True)
  File "C:\msys64\usr\src\grass72_release\dist.x86_64-w64-mingw32\etc\python/grass/script/core.py", line 410, in run_command
    return handle_errors(returncode, returncode, args, kwargs)
  File "C:\msys64\usr\src\grass72_release\dist.x86_64-w64-mingw32\etc\python/grass/script/core.py", line 329, in handle_errors
    returncode=returncode)
grass.exceptions.CalledModuleError: Module run None ['g.ppmtopng', '--q', 'input=C:/msys64/usr/src/grass72_release/dist.x86_64-w64-mingw32\\demolocation\\PERMANENT\\.tmp/unknown\\4072.0.ppm', 'output=C:/msys64/usr/src/grass72_release/dist.x86_64-w64-mingw32\\docs\\html\\colortables\\aspect.png'] ended with error
Process ended with non-zero return code -1073741819. See errors in the (error) output.
Makefile:16: recipe for target 'thumbnails' failed
make[4]: [thumbnails] Error 1 (ignored)

[mlennert]

A bit of a wild guess, but in thumbnails.py I see that the ppm files are created with:

def write_ppm(dst, data):
    w = height
    h = width
    fh = open(dst, "wb")
    fh.write("P6\n%d %d\n%d\n" % (w, h, 255))
    data.tofile(fh)
    fh.close()

Could it be that the '\n' are causing problems on Windows ?

[annakrat]

The ppm is created, the problem is in g.ppmtopng, it crashed on windows during writing the png.

[annakrat]

Replying to annakrat:

The ppm is created, the problem is in g.ppmtopng, it crashed on windows during writing the png.

The script generating thumbnails could be rewritten with PIL for example, instead of using g.ppmtopng. Or, ideally the functionality would be added to d.colortable to flip and rotate the table. Then the script would become quite short.

Related ticket: #3156

[mlennert]

Replying to annakrat:

The ppm is created, the problem is in g.ppmtopng, it crashed on windows during writing the png.

Yes, my question was whether the fact of writing the file with '\n' line endings could cause issues in reading it in g.ppmtopng. But as I said: wild guess. I suppose this worked before with these same line endings ?

[annakrat]

Replying to mlennert:

Replying to annakrat:

The ppm is created, the problem is in g.ppmtopng, it crashed on windows during writing the png.

Yes, my question was whether the fact of writing the file with '\n' line endings could cause issues in reading it in g.ppmtopng. But as I said: wild guess. I suppose this worked before with these same line endings ?

I think Python takes care of that automatically. The generated ppm looks fine, so I think it's really about writing the png.

[wenzeslaus]

In 70725:

replace g.ppmtopng and d.colortable by g.region, r.mapcalc and d.rast (see #3200)

PNG is now created directly correctly rotated. No flipping seems to be needed.
Same white-black border as from d.colortable done by d.graph but black just one pixel wide
resulting in images with lighter apparance.
No issue with missing black corner visible. All images have all borders (e.g. random, grey.log).
kelvin starts correctly with color (not white for nv), population correctly start with white,
gdd still starts with potentially too much white (0 white, 0.00001 grey) and is tricky to
satisfy by the same code as kelvin and grey1.0.
Alternative implementation could use frames and simpler r.mapcalc expressions.
Border colors can be easilly changed, but widths would need some refactoring of constants.
Leaving the PPM manipulation code there.

[wenzeslaus]

After testing r70725 can be potentially backported to 7.2 branch before 7.2.1. This is the difference between 7.2.0 and current trunk:

In trunk, the code can be further improved by separating or removing the currently unused PPM manipulation code and by creating a more general module for visualizing (raw) color tables from the thumbnails.py script. Another possible improvement is revising the border (now black inner, white outer) to make it simpler or as in d.legend.

[martinl]

Something to backport, set milestone to 7.2.3 or just time to close the ticket?

[wenzeslaus]

In 71463:

replace g.ppmtopng and d.colortable by g.region, r.mapcalc and d.rast (see #3200, backport of r70725)

[wenzeslaus]

No complains about r70725, backported in r70725 to 7.2, but I didn't realized I should have waited. We can revert if needed.

[martinl]

Lets check next 7.2 WinGRASS daily build and re-open if needed.

[martinl]

I quickly (start GRASS, launch G72:r.colors from Command prompt, open widget for selecting color tables) tested ​https://wingrass.fsv.cvut.cz/grass72/x86_64/WinGRASS-7.2.2svn-r71463-353-Setup-x86_64.exe. First test ended with frozen GRASS. The second attempt was successful. Could someone else tested on Windows too? I think that my frozen GRASS was unrelated to this ticket.