Opened 8 years ago

Last modified 5 years ago

#3156 new defect

Cannot export GRASS raster map using r.out.png (crashes on windows)

Reported by: spawley Owned by: grass-dev@…
Priority: normal Milestone: 7.6.2
Component: Raster Version: svn-releasebranch76
Keywords: r.out.png, wingrass, g.ppmtopng Cc:
CPU: x86-64 Platform: MSWindows 7

Description

I'm having a problem with the r.out.png module on windows. I have tried using the standalone grass 7.0.4 (64), as well as OSGeo4W 7.04 and 7.3 64 bit versions. Any attempt to export a GRASS raster using r.out.png causes a 'GRASS 7 has stopped working' error.This also affects the Pygrass ability to display GRASS rasterrow objects, for instance as part of jupyter notebook. The output is in writable directory with a correct path. The module creates an output png file, but then crashes, leaving the png file empty.

Change History (22)

comment:1 by martinl, 8 years ago

Keywords: r.out.png wingrass added

comment:2 by neteler, 7 years ago

Milestone: 7.0.57.0.6

in reply to:  2 ; comment:3 by hellik, 7 years ago

Replying to neteler:

tested by

System Info                                                                     
GRASS version: 7.2.svn                                                          
GRASS SVN revision: r69794                                                      
Build date: 2016-11-13                                                          
Build platform: x86_64-w64-mingw32                                              
GDAL: 2.1.2                                                                     
PROJ.4: 4.9.3                                                                   
GEOS: 3.5.0                                                                     
SQLite: 3.14.1                                                                  
Python: 2.7.5                                                                   
wxPython: 2.8.12.1                                                              
Platform: Windows-8-6.2.9200        

within the NC sample dataset

r.out.png input=elevation@PERMANENT output=D:\tmp\test.png  

it also crashes in the winGRASS7.2.line.

in reply to:  3 comment:4 by hellik, 7 years ago

Replying to hellik:

Replying to neteler:

tested by

System Info                                                                     
GRASS version: 7.2.svn                                                          
GRASS SVN revision: r69794                                                      
Build date: 2016-11-13                                                          
Build platform: x86_64-w64-mingw32                                              
GDAL: 2.1.2                                                                     
PROJ.4: 4.9.3                                                                   
GEOS: 3.5.0                                                                     
SQLite: 3.14.1                                                                  
Python: 2.7.5                                                                   
wxPython: 2.8.12.1                                                              
Platform: Windows-8-6.2.9200        

DEBUG=3 on

r.out.png --verbose input=elevation@PERMANENT output=D:\dl\lz_10m\test.png      
D1/3: G_set_program_name(): r.out.png
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\user1
D3/3: G_str_to_color(): str = 'white'
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\user1\WIND
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\user1\WIND
D2/3:   file open: read (mode = r)
D2/3: G__read_Cell_head
D2/3: G__read_Cell_head_array
D3/3: region item: proj:       99
D3/3: region item: zone:       0
D3/3: region item: north:      228500
D3/3: region item: south:      215000
D3/3: region item: east:       645000
D3/3: region item: west:       630000
D3/3: region item: cols:       1500
D3/3: region item: rows:       1350
D3/3: region item: e-w resol:  10
D3/3: region item: n-s resol:  10
D3/3: region item: top:        1.000000000000000
D3/3: region item: bottom:     0.000000000000000
D3/3: region item: cols3:      1500
D3/3: region item: rows3:      1350
D3/3: region item: depths:     1
D3/3: region item: e-w resol3: 10
D3/3: region item: n-s resol3: 10
D3/3: region item: t-b resol:  1
D1/3: rows = 1350, cols = 1500
D1/3: G_find_raster2(): name=elevation mapset=PERMANENT
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\cell\elevation
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\cellhd\elevation
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\cellhd\elevation
D2/3:   file open: read (mode = r)
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\cellhd\elevation
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\cellhd\elevation
D2/3:   file open: read (mode = r)
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\cellhd\elevation
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\cellhd\elevation
D2/3:   file open: read (mode = r)
D2/3: G__read_Cell_head
D2/3: G__read_Cell_head_array
D3/3: region item: proj:       99
D3/3: region item: zone:       0
D3/3: region item: north:      228500
D3/3: region item: south:      215000
D3/3: region item: east:       645000
D3/3: region item: west:       630000
D3/3: region item: cols:       1500
D3/3: region item: rows:       1350
D3/3: region item: e-w resol:  10
D3/3: region item: n-s resol:  10
D3/3: region item: format:     -1
D3/3: region item: compressed: 1
D1/3: G_find_raster2(): name=elevation mapset=PERMANENT
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\cell\elevation
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\fcell\elevation
D1/3: G_find_raster2(): name=elevation mapset=PERMANENT
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\cell\elevation
D2/3: G_file_name(): path = D:\grassdata\nc_spm_08_grass7\PE
RMANENT\cell_misc\elevation\f_format
D1/3: G_find_raster2(): name=elevation mapset=PERMANENT
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\cell\elevation
D1/3: G_find_raster2(): name=elevation mapset=PERMANENT
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\cell\elevation
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\fcell\elevation
D1/3: G_find_raster2(): name=elevation mapset=PERMANENT
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\cell\elevation
D2/3: G_file_name(): path = D:\grassdata\nc_spm_08_grass7\PE
RMANENT\cell_misc\elevation\f_format
D2/3: G_file_name(): path = D:\grassdata\nc_spm_08_grass7\PE
RMANENT\cell_misc\elevation\gdal
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\fcell\elevation
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\fcell\elevation
D3/3: create window mapping (1500 columns)
D1/3: G_find_raster2(): name=elevation mapset=PERMANENT
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\cell\elevation
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\fcell\elevation
D1/3: G_find_raster2(): name=elevation mapset=PERMANENT
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\cell\elevation
D2/3: G_file_name(): path = D:\grassdata\nc_spm_08_grass7\PE
RMANENT\cell_misc\elevation\f_format
D2/3: G_file_name(): path = D:\grassdata\nc_spm_08_grass7\us
er1\quant2/PERMANENT\elevation
D2/3: G_file_name(): path = D:\grassdata\nc_spm_08_grass7\PE
RMANENT\cell_misc\elevation\f_quant
D2/3: G_file_name(): path = D:\grassdata\nc_spm_08_grass7\PE
RMANENT\cell_misc\elevation\f_quant
D2/3: G_file_name(): path = D:\grassdata\nc_spm_08_grass7\PE
RMANENT\cell_misc\elevation\null
D2/3: G_file_name(): path = D:\grassdata\nc_spm_08_grass7\PE
RMANENT\cell_misc\elevation\null
D1/3: G_find_raster(): name=MASK mapset=user1
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\user1\cell\MASK
Converting <elevation@PERMANENT>...
D1/3: G_find_raster2(): name=elevation@PERMANENT mapset=
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\cell\elevation
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\fcell\elevation
D1/3: G_find_raster(): name=elevation@PERMANENT mapset=
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\cell\elevation
D2/3: G_file_name(): path = D:\grassdata\nc_spm_08_grass7\us
er1\colr2/PERMANENT\elevation
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\colr\elevation
D2/3: G_file_name(): path =
D:\grassdata\nc_spm_08_grass7\PERMANENT\colr\elevation
D2/3:   file open: read (mode = r)
D3/3: adding rule 55=55.58 0 191 191  75=75.73 0 255 0
D3/3: adding rule 75=75.73 0 255 0  95=95.88 255 255 0
D3/3: adding rule 95=95.88 255 255 0  116=116.03 255 127 0
D3/3: adding rule 116=116.03 255 127 0  136=136.18 191 127
63
D3/3: adding rule 136=136.18 191 127 63  156=156.33 20 20 20

comment:5 by annakrat, 7 years ago

This is probably related to #3200 (generating r.colors thumbnails on windows) where g.ppmtopng fails to write the png. So some problem with PNG library on Windows? Not sure, what to do about it...

comment:6 by marisn, 7 years ago

Still an issue:

GRASS versija: 7.3.svn                                                          
GRASS SVN revīzija: r70508                                                      
Būvējuma datums: 2017-01-09                                                     
Būvēšanas platforma: x86_64-w64-mingw32                                         
GDAL: 2.1.2                                                                     
PROJ.4: 4.9.3                                                                   
GEOS: 3.5.0                                                                     
SQLite: 3.14.1                                                                  
Python: 2.7.5                                                                   
wxPython: 2.8.12.1                                                              
Platforma: Windows-8-6.2.9200 (OSGeo4W)

comment:7 by neteler, 6 years ago

Milestone: 7.0.67.0.7

comment:8 by wenzeslaus, 6 years ago

Can somebody debug this on MS Windows (gdb, valgrind, ...)? Does it give return code -1073741819 (which based on internet search seems to be either Access Violation - dereferencing an invalid memory location - or File System Error) as in #3200? What about DEBUG=5? Does g.ppmtopng? Does switching cairo for png works in GUI or with d.mon or direct rendering?

I didn't get any issues reported with valgrind on Linux and program finished successfully.

in reply to:  8 comment:9 by hellik, 6 years ago

Replying to wenzeslaus:

Can somebody debug this on MS Windows (gdb, valgrind, ...)? Does it give return code -1073741819 (which based on internet search seems to be either Access Violation - dereferencing an invalid memory location - or File System Error) as in #3200? What about DEBUG=5? Does g.ppmtopng? Does switching cairo for png works in GUI or with d.mon or direct rendering?

I didn't get any issues reported with valgrind on Linux and program finished successfully.

tested with

System Info                                                                     
GRASS version: 7.4.1                                                            
GRASS SVN revision: r72807                                                      
Build date: 2018-06-13                                                          
Build platform: x86_64-w64-mingw32                                              
GDAL: 2.2.4                                                                     
PROJ.4: 4.9.3                                                                   
GEOS: 3.5.0                                                                     
SQLite: 3.17.0                                                                  
Python: 2.7.14                                                                  
wxPython: 2.8.12.1                                                              
Platform: Windows-10-10.0.17134 (OSGeo4W) 
r.out.png input=elevation@PERMANENT output=D:\wd\test\test_01.png               
(Wed Sep 12 19:32:06 2018) Command finished (2 sec)                             
                                                
r.out.png -w input=elevation@PERMANENT output=D:\wd\test\test_02.png compression=0
(Wed Sep 12 19:32:28 2018) Command finished (2 sec)                             
                                               
r.out.png -t -w input=elevation@PERMANENT output=D:\wd\test\test_03.png compression=4
(Wed Sep 12 19:32:46 2018) Command finished (2 sec)               

no crash, but empty png files (header unkown).

in reply to:  8 ; comment:10 by hellik, 6 years ago

Replying to wenzeslaus:

Does g.ppmtopng?

(Wed Sep 12 19:39:08 2018)                                                      
g.ppmtopng --verbose input=C:\Users\hkmyr\AppData\Local\Temp\grass7-hkmyr-8260\tmp3lx1lm.ppm output=D:\wd\test\test_05.png
(Wed Sep 12 19:39:11 2018) Command finished (2 sec)   

the file is empty with an unknown header

in reply to:  10 comment:11 by hellik, 6 years ago

Replying to hellik:

Replying to wenzeslaus:

Does g.ppmtopng?

(Wed Sep 12 19:39:08 2018)                                                      
g.ppmtopng --verbose input=C:\Users\hkmyr\AppData\Local\Temp\grass7-hkmyr-8260\tmp3lx1lm.ppm output=D:\wd\test\test_05.png
(Wed Sep 12 19:39:11 2018) Command finished (2 sec)   

the file is empty with an unknown header

C:\>g.gisenv set=DEBUG=3

C:\>g.ppmtopng --verbose input=C:\Users\hkmyr\AppData\Local\Temp\grass7-hkmyr-8260\tmp3lx1lm.ppm output=D:\wd\test\test_06.png
D1/3: G_set_program_name(): g.ppmtopng
D2/3: G_file_name(): path = D:\grassdata\nc_spm_08_grass7\user1

comment:12 by neteler, 6 years ago

How about making it "simply" a Python wrapper around r.out.gdal with PNG etc hardcoded?

in reply to:  12 ; comment:13 by wenzeslaus, 6 years ago

Keywords: g.ppmtopng added

Replying to neteler:

How about making it "simply" a Python wrapper around r.out.gdal with PNG etc hardcoded?

That would solve only r.out.png not g.ppmtopng or PNG driver (if it is broken (please test)).

Additionally, r.out.gdal and r.out.png outputs are not identical and r.out.png always gave me a more expected output, for example transparency works.

However, your suggestion makes sense. An alternative I used was d.rast. That again gives different result - the pixels are resampled so that when ns and ew resolutions don't match you get the right shape unlike with r.out.gdal or r.out.png where you get the result stretched when viewed without georeferencing.

The code for d.rast is below. AFAIU r.out.gdal would require setting transparency; code for a simple but limited way to do that is attached too. I'm thinking about adding some of these functions to the Python library.

def set_rendering_environment(width, height, filename, transparent,
                              backgroud_color='ffffff', driver='cairo',
                              compression=None, env=None):
    # if parameter not provided (but allow for empty dictionary)
    if env is None:
        env = os.environ
    env['GRASS_RENDER_WIDTH'] = str(width)
    env['GRASS_RENDER_HEIGHT'] = str(height)
    env['GRASS_RENDER_IMMEDIATE'] = driver
    env['GRASS_RENDER_BACKGROUNDCOLOR'] = backgroud_color
    env['GRASS_RENDER_TRUECOLOR'] = "TRUE"
    if transparent:
        env['GRASS_RENDER_TRANSPARENT'] = "TRUE"
    else:
        env['GRASS_RENDER_TRANSPARENT'] = "FALSE"
    if compression:
        env['GRASS_RENDER_FILE_COMPRESSION'] = str(compression)
    env['GRASS_RENDER_FILE'] = str(filename)


def get_region():
    gregion_out = gs.read_command('g.region', flags='pg')
    region = gs.parse_key_val(gregion_out, sep='=')
    return {'east': float(region['e']), 'north': float(region['n']),
            'west': float(region['w']), 'south': float(region['s']),
            'rows': int(region['rows']), 'cols': int(region['cols']),
            'nsres': float(region['nsres']),
            'ewres': float(region['ewres'])}


region = get_region()
if region['nsres'] > region['ewres']:
    # oversample in rows, do not loose columns
    width = region['cols']
    height = region['rows'] * (region['nsres'] / region['ewres'])
else:
    # oversample in columns, do not loose rows
    width = region['cols'] * (region['ewres'] / region['nsres'])
    height = region['rows']
if 't' in routpng_flags:
    transparent = True
else:
    transparent = False
set_rendering_environment(width=width, height=height,
                          filename=output_file,
                          transparent=True, driver='cairo',
                          compression=compression)
        try:
                from PIL import Image
        except ImportError:
                gcore.warning(_("Cannot save transparency because"
                                " PIL library is missing."))
                return
        if rgb == 'white':
                rgb = (255, 255, 255)
        elif rgb == 'black':
                rgb = (0, 0, 0)
        img = Image.open(filename)
        img = img.convert("RGBA")
        old_data = img.getdata()
        new_data = []
        for item in old_data:
                if item[0] == rgb[0] and item[1] == rgb[1] and item[2] == rgb[2]:
                        new_data.append((255, 255, 255, 0))
                else:
                        new_data.append(item)
        img.putdata(new_data)
        img.save(filename, "PNG")

A step further would be a r.export.png module which would, with whatever backend, create a georeferenced PNG but in a SRS provided by the user.

in reply to:  13 ; comment:14 by wenzeslaus, 6 years ago

Replying to wenzeslaus:

Replying to neteler:

How about making it "simply" a Python wrapper around r.out.gdal with PNG etc hardcoded?

... Additionally, r.out.gdal and r.out.png outputs are not identical and r.out.png always gave me a more expected output, for example transparency works.

That might have been case of r.out.tiff too. This goes outside of this ticket, but now I wonder what is the relation of r.out.png to r62524. Why was r.out.tiff moved to addons and r.out.png left in core? Maybe dependency considerations?

In any case, the point of both r.out.png and r.out.tiff is an easy interface at this point (since GDAL will be always present), assuming that the differences in outputs are addressed. So than r.export.png makes more sense for 8.0 than keeping r.out.png around.

in reply to:  14 ; comment:15 by neteler, 6 years ago

Replying to wenzeslaus:

That might have been case of r.out.tiff too. This goes outside of this ticket, but now I wonder what is the relation of r.out.png to r62524. Why was r.out.tiff moved to addons and r.out.png left in core? Maybe dependency considerations?

It was your suggestion to move r.out.tiff to addons :-)

https://lists.osgeo.org/pipermail/grass-dev/2014-November/071498.html

in reply to:  15 ; comment:16 by wenzeslaus, 6 years ago

Replying to neteler:

Replying to wenzeslaus:

That might have been case of r.out.tiff too. This goes outside of this ticket, but now I wonder what is the relation of r.out.png to r62524. Why was r.out.tiff moved to addons and r.out.png left in core? Maybe dependency considerations?

It was your suggestion to move r.out.tiff to addons :-)

https://lists.osgeo.org/pipermail/grass-dev/2014-November/071498.html

As far as I understand the discussion there, I suggested addons as an alternative to removal. I don't know how it got to the original "dead code" list. Perhaps it was just considered to be superseded by r.out.gdal?

in reply to:  16 ; comment:17 by mmetz, 6 years ago

Replying to wenzeslaus:

[...] Perhaps it was just considered to be superseded by r.out.gdal?

r.out.png and r.out.tiff create something like a screenshot using the current region settings: the output contains only coloring information. This can be changed with r.colors.

r.out.gdal exports cell values using the current region settings: the output contains the original cell values.

r.out.png and r.out.tiff have been superseeded by the button to "Save display to file" in the map display.

in reply to:  17 ; comment:18 by mmetz, 6 years ago

Replying to mmetz:

Replying to wenzeslaus:

[...] Perhaps it was just considered to be superseded by r.out.gdal?

r.out.png and r.out.tiff create something like a screenshot using the current region settings: the output contains only coloring information. This can be changed with r.colors.

r.out.gdal exports cell values using the current region settings: the output contains the original cell values.

r.out.png and r.out.tiff have been superseeded by the button to "Save display to file" in the map display.

Not exactly, "Save display to file" does not produce a georeferenced screenshot, while r.out.tiff and r.out.png do produce a georeferenced screenshot.

in reply to:  18 comment:19 by wenzeslaus, 6 years ago

Replying to mmetz:

Replying to mmetz:

Replying to wenzeslaus:

[...] Perhaps it was just considered to be superseded by r.out.gdal?

r.out.png and r.out.tiff create something like a screenshot using the current region settings: the output contains only coloring information. This can be changed with r.colors.

r.out.gdal exports cell values using the current region settings: the output contains the original cell values.

r.out.png and r.out.tiff have been superseeded by the button to "Save display to file" in the map display.

Not exactly, "Save display to file" does not produce a georeferenced screenshot, while r.out.tiff and r.out.png do produce a georeferenced screenshot.

Thank you for clarification. That's exactly how I used r.out.png, to create something like a "georeferenced screenshot".

Than it is clear that "Save display to file" is not a good replacement because it is not georeferenced and it is not a module. The georeferencing could be added. I think there is even a ticket for that. However, using it as module is not possible, although d.out.file is pretty close.

in reply to:  13 comment:20 by wenzeslaus, 6 years ago

Replying to wenzeslaus:

...would require setting transparency; code for a simple but limited way to do that is attached too. I'm thinking about adding some of these functions to the Python library.

Transparency function added to trunk (7.7) in a new module grass.imaging.operations in r73329 (together with crop, invert colors and create thumbnail).

comment:21 by martinl, 5 years ago

Still relevant?

in reply to:  21 comment:22 by hellik, 5 years ago

Milestone: 7.0.77.6.2
Version: 7.0.4svn-releasebranch76

Replying to martinl:

Still relevant?

no crash, but empty png files (header unkown).

upgrade milestone

Note: See TracTickets for help on using tickets.