wxGUI Cartographic Composer: preview broken on Windows

[martinl]

Recently python-pil package has been introduced into OSGeo4W framework. When testing "preview" functionality in cartographic composer I am getting error bellow

Traceback (most recent call last):
  File "c:\osgeo4w\usr\src\grass64_release\dist.i686-pc-mingw32\etc\wxpython\gui_modules\psmap.py", line 317, in OnCmdDone
    im.save(self.imgName, format = 'png')
  File "C:\OSGeo4w\apps\Python27\lib\site-packages\PIL\Image.py", line 1406, in save
    self.load()
  File "C:\OSGeo4w\apps\Python27\lib\site-packages\PIL\EpsImagePlugin.py", line 283, in load
    self.im = Ghostscript(self.tile, self.size, self.fp)
  File "C:\OSGeo4w\apps\Python27\lib\site-packages\PIL\EpsImagePlugin.py", line 72, in Ghostscript
    gs.write(s)
IOError: [Errno 32] Broken pipe
c:\osgeo4w\usr\src\grass64_release\dist.i686-pc-mingw32\etc\wxpython\gui_modules\psmap.py:325: RuntimeWarning: tp_compare didn't return -1 or -2 for exception
  rect = self.previewCanvas.ImageRect()
Traceback (most recent call last):
  File "c:\osgeo4w\usr\src\grass64_release\dist.i686-pc-mingw32\etc\wxpython\gui_modules\psmap.py", line 325, in OnCmdDone
    rect = self.previewCanvas.ImageRect()
  File "c:\osgeo4w\usr\src\grass64_release\dist.i686-pc-mingw32\etc\wxpython\gui_modules\psmap.py", line 1626, in ImageRect
    iW, iH = img.GetWidth(), img.GetHeight()
  File "C:\OSGeo4w\apps\Python27\lib\site-packages\wx-2.8-msw-unicode\wx\_core.py", line 3031, in GetWidth
    return _core_.Image_GetWidth(*args, **kwargs)
wx._core.PyAssertionError: C++ assertion "Ok()" failed at ..\..\src\common\image.cpp(1454) in wxImage::GetWidth(): invalid image

[annakrat]

I have no idea how to fix this, I've done small changes in r50520 but I don't think it will help. I found out, that someone had similar problem but there is no answer: ​http://stackoverflow.com/questions/6281744/python-pil-errno-32-broken-pipe-when-saving-png

[martinl]

Replying to annakrat:

I have no idea how to fix this, I've done small changes in r50520 but I don't think it will help. I found out, that someone had similar problem but there is no answer: ​http://stackoverflow.com/questions/6281744/python-pil-errno-32-broken-pipe-when-saving-png

yeap, PIL using Ghostscript for converting ps to image. That would probably require to add ghostscript as new package into OSGeo4W framework.

[martinl]

Replying to martinl:

Replying to annakrat:

I have no idea how to fix this, I've done small changes in r50520 but I don't think it will

anyway the candidate for backport...

[annakrat]

Replying to martinl:

anyway the candidate for backport...

it's done

[annakrat]

There are a few problems:

• Ghostscript is missing (not in OSGeo4W framework)

• if it is installed it is not on the PATH

• from the PIL source code (C:\OSGeo4W\apps\Python27\Lib\site-packages\PIL\EpsImagePlugin.py):

def Ghostscript(tile, size, fp):
    """Render an image using Ghostscript (Unix only)"""

...

    command = ["gs",
               "-q",                    # quite mode
               "-g%dx%d" % size,        # set output geometry (pixels)
               "-dNOPAUSE -dSAFER",     # don't pause between pages, safe mode
               "-sDEVICE=ppmraw",       # ppm driver
               "-sOutputFile=%s" % file,# output file
               "- >/dev/null 2>/dev/null"]

[martinl]

Replying to annakrat:

• Ghostscript is missing (not in OSGeo4W framework)

just a quick test, I have added Ghostscript [1] then I have tried to produced PS using wxGUI Cartographic Composer, the file has been created. But if I try to open it using e.g. Gimp, it fails.

Another issue is missing ps2pdf. But first we should fix generation of PostScript output.

Martin

[1] ​http://trac.osgeo.org/grass/wiki/CompileOnWindows#Ghostscript

[annakrat]

Replying to martinl:

Replying to annakrat:

• Ghostscript is missing (not in OSGeo4W framework)

just a quick test, I have added Ghostscript [1] then I have tried to produced PS using wxGUI Cartographic Composer, the file has been created. But if I try to open it using e.g. Gimp, it fails.

I don't understand much. You tried the preview or exporting PS file? The creation of the ps file is not related to this ticket, the reading of the ps file by PIL is the problem. When GIMP on Windows is not able to read it, it's a problem of ps.map module.

The 'unix only' and '>/dev/null 2>/dev/null' in the ghostscript command must be a serious problem for Windows or not?

[hellik]

Replying to martinl:

Replying to annakrat:

• Ghostscript is missing (not in OSGeo4W framework)

just a quick test, I have added Ghostscript [1] then I have tried to produced PS using wxGUI Cartographic Composer, the file has been created. But if I try to open it using e.g. Gimp, it fails.

I've followed your example and exported a simple elevation.ps and converted it with freepdf [1], a ghostscript based pdf converter, successfully to a pdf. Gimp not tested.

Helmut

[2] ​http://freepdfxp.de/

[annakrat]

There is a ​patch for Windows but nobody seems to react on it. To make it work on windows I could copy the code from EpsImagePlugin.py, change it (according to the patch) and force it to use the new code at runtime. This a ugly hack but Python allows it.

[annakrat]

After some changes, preview seems to work. Tested with winGRASS 7. Please could anyone test the tomorrow snapshot?

Anna

[martinl]

Replying to annakrat:

After some changes, preview seems to work. Tested with winGRASS 7. Please could anyone test the tomorrow snapshot?

after putting gswin32c.exe to PATH, it works. At least partially. I have added raster and line vector map. In the preview only vector map is rendered. Previewing raster seems to be broken (the maps have same extent). See attachment:wxpsmap1.png​

I noticed also a small bug, labels are not centered and they are in one line (draft mode, see attachment:wxpsmap0.png​).

OSGeo4W doesn't provide gs package. We have two options - to add new package for gs or provide it within grass package.

[martinl]

wxpsmap, labels

[martinl]

wxpsmap, preview, vector only

[martinl]

Replying to martinl:

after putting gswin32c.exe to PATH, it works. At least partially. I have added raster and line vector map. In the preview only vector map is rendered. Previewing raster seems to be broken (the maps have same extent). See attachment:wxpsmap1.png​

update: final ps file is fine (raster is included)

one note: I would expect that when switching to 'preview' mode, the preview is automatically generated. At this moment it's generated only when the button is pushed.

[hamish]

Replying to martinl:

one note: I would expect that when switching to 'preview' mode, the preview is automatically generated. At this moment it's generated only when the button is pushed.

is it possible in wxPy to give the button a 3px wide yellow border, or otherwise make it blink?

Hamish

[annakrat]

Replying to hamish:

Replying to martinl:

one note: I would expect that when switching to 'preview' mode, the preview is automatically generated. At this moment it's generated only when the button is pushed.

I am not sure about that. Switching tabs usually does nothing in most applications. Also to generate the picture is quite time-consuming and I think users don't mind waiting after pressing button 'run' but they might be surprised by waiting after switching tabs.

is it possible in wxPy to give the button a 3px wide yellow border, or otherwise make it blink?

There is a way to distinguish between active and nonactive tabs it (text color) but I think it's not possible to highlight individual tabs. What exactly you would like it for?

[annakrat]

Replying to martinl:

Replying to martinl:

after putting gswin32c.exe to PATH, it works. At least partially. I have added raster and line vector map. In the preview only vector map is rendered. Previewing raster seems to be broken (the maps have same extent). See attachment:wxpsmap1.png​

update: final ps file is fine (raster is included)

Is there anything we can do about that? It seems to be the problem of PIL (which uses ghostscript), or maybe something is wrong with the PS file generated by ps.map?

[wenzeslaus]

Replying to annakrat:

Replying to hamish:

Replying to martinl:

one note: I would expect that when switching to 'preview' mode, the preview is automatically generated. At this moment it's generated only when the button is pushed.

I am not sure about that. Switching tabs usually does nothing in most applications. Also to generate the picture is quite time-consuming and I think users don't mind waiting after pressing button 'run' but they might be surprised by waiting after switching tabs.

Maybe there is one (for programmer) complicated and (for user) unclear way to deal with generation. Start a timer after switching a tab and after a while if the tab is still active start with generation.

[hamish]

Replying to annakrat:

There is a way to distinguish between active and nonactive tabs it (text color) but I think it's not possible to highlight individual tabs. What exactly you would like it for?

I didn't mean to highlight the tab, but rather to give the green triangle run button a backlit yellow sun glow when the Preview tab was switched to -- trade out the icon, then trade it back once the rendering was done.

Replying to wenzeslaus:

Maybe there is one (for programmer) complicated and (for user) unclear way to deal with generation. Start a timer after switching a tab and after a while if the tab is still active start with generation.

If the computation were relatively cheap that could be ok, but since it can be expensive I'd vote to stick with the user-initiated approach. (just make it more obvious)

Along the line to make it more obvious: how about if the initial preview document loaded a dummy graphic which had a picture of the green triangle button circled and an arrow pointing NW saying "click up there to generate preview". ?

Hamish

[wenzeslaus]

Replying to hamish:

Along the line to make it more obvious: how about if the initial preview document loaded a dummy graphic which had a picture of the green triangle button circled and an arrow pointing NW saying "click up there to generate preview". ?

Good idea. Generally, we should try to add some hints this and there in the GUI to guide user what to do, what does this and that mean. Example of this is a recently added gray text in command console with hint to use Ctrl+Space and Tab keys.

The text would be enough for me. The graphics seems to be little bit too much.

[annakrat]

Replying to wenzeslaus:

Replying to hamish:

Along the line to make it more obvious: how about if the initial preview document loaded a dummy graphic which had a picture of the green triangle button circled and an arrow pointing NW saying "click up there to generate preview". ?

Good idea. Generally, we should try to add some hints this and there in the GUI to guide user what to do, what does this and that mean.

I added a message (r55118) which is shown in the statusbar when switching to preview tab. It might be enough? Feel free to improve the message if it sounds strange.

Anna

[hamish]

Replying to annakrat:

I added a message (r55118) which is shown in the statusbar when switching to preview tab. It might be enough? Feel free to improve the message if it sounds strange.

yes, that works very nicely, thank you. the text is right next to the tab you click so you're already looking at it.

The message doesn't sound strange, just concise. I think it's fine.

cheers, Hamish

[annakrat]

Replying to martinl:

I noticed also a small bug, labels are not centered and they are in one line (draft mode, see attachment:wxpsmap0.png​).

Fixed in r55141 (and r55182).

Anna

[wenzeslaus]

Summary

Preview works now; info about generating preview is in the status bar, so solved too; as well as not centered labels.

Problem highly related to or part of this ticked:

OSGeo4W doesn't provide gs package. We have two options - to add new package for gs or provide it within grass package.

Another problem:

I have added raster and line vector map. In the preview only vector map is rendered. Previewing raster seems to be broken (the maps have same extent). See attachment:wxpsmap1.png​

[neteler]

Replying to wenzeslaus: ...

OSGeo4W doesn't provide gs package. We have two options - to add new package for gs or provide it within grass package.

A package outside of GRASS would be likely cleaner.

Related; ​http://ghostscript.com/pipermail/gs-devel/2013-January/009382.html

[neteler]

Rather than crashing with "... gs failed", how about trapping this and opening a message with the hint to visit ​http://trac.osgeo.org/grass/wiki/CompileOnWindows#Ghostscript

?

[annakrat]

Replying to neteler:

Rather than crashing with "... gs failed", how about trapping this and opening a message with the hint to visit ​http://trac.osgeo.org/grass/wiki/CompileOnWindows#Ghostscript ?

Sure, done in r55713-5. Can anyone confirm that the instructions work?

[hellik]

screenshot

[hellik]

Replying to annakrat:

Can anyone confirm that the instructions work?

instructions followd I get error message (see attached mc_12042013_150414.png)

[annakrat]

Replying to hellik:

Replying to annakrat:

Can anyone confirm that the instructions work?

instructions followd I get error message (see attached mc_12042013_150414.png)

oh, sorry, I was sure that the previous fixes were already applied to 64 but obviously not. Please try r55716 tomorrow.

[hellik]

composer

[hellik]

preview

[hellik]

pdf

[hellik]

Replying to annakrat:

oh, sorry, I was sure that the previous fixes were already applied to 64 but obviously not. Please try r55716 tomorrow.

tested with the nightly build.

it fails gracefully now, if gs isn't in path. thanks.

the instructions to put gs in gswin32c.exe and gsdll32.dll to c:\OSGeo4W\bin\ works also.

just another little thing, see attached screenshots composer, preview, pdf.

just added elevation and lakes from the nc-sample data set (composer.png). in the preview only the vector is displayed (preview.png). after exporting a ps and converting it to a pdf, raster and vector are there correctly.

[martinl]

Replying to neteler:

Replying to wenzeslaus: ...

OSGeo4W doesn't provide gs package. We have two options - to add new package for gs or provide it within grass package.

A package outside of GRASS would be likely cleaner.

Related; ​http://ghostscript.com/pipermail/gs-devel/2013-January/009382.html

New package added to OSGeo4W framework (1). GRASS dependences updated. Testing welcomed.

(1) ​http://trac.osgeo.org/osgeo4w/wiki/pkg-gs

[martinl]

Replying to hellik:

just added elevation and lakes from the nc-sample data set (composer.png). in the preview only the vector is displayed (preview.png). after exporting a ps and converting it to a pdf, raster and vector are there correctly.

I can confirm this issue. Could be that pkg-gs (1) is incomplete?

(1) ​http://trac.osgeo.org/osgeo4w/wiki/pkg-gs

[hamish]

Hi,

just tested in a wingrass nightly build, preview fails with unable to find the ps2pdf program.

(see also earlier comment:6)

Hamish

[annakrat]

Replying to hamish:

Hi,

just tested in a wingrass nightly build, preview fails with unable to find the ps2pdf program.

Really preview? It fails when creating pdf file (Export to PDF). ps2pdf seems to be missing, we could add it there or maybe call gs directly?

[hamish]

Replying to annakrat:

Replying to hamish:

just tested in a wingrass nightly build, preview fails with unable to find the ps2pdf program.

Really preview? It fails when creating pdf file (Export to PDF).

right. at creating the pdf, sorry. in the preview tab pressing the green '>' button makes a page, but it is all white except for the map box frame in the preview window. Saving to PostScript makes a nice file on the disk, all's correct there, just the preview image is missing.

pressing the PDF button comes back with a pop-up window "ps2pdf exited with return code 1".

ps2pdf seems to be missing, we could add it there or maybe call gs directly?

right, no ps2pdf in \extrabin.

On Debian, ps2pdf comes with the ghostscript package, it's just a shell script which just calls:

ps2pdfwr -dCompatibilityLevel=1.4

And ps2pdfwr is another shell/wrapper script which calls gs. It boils down to:

gs -P- -dSAFER -dCompatibilityLevel=1.4 -q -P- -dNOPAUSE -dBATCH \
   -sDEVICE=pdfwrite -sstdout=%stderr -sOutputFile=elevation.pdf \
   -P- -dSAFER -dCompatibilityLevel=1.4 -c .setpdfwrite -f \
   elevation.ps

just replace elevation.pdf and elevation.ps with %s, and gs for gswin32c.

I was a little worried about the %stderr option, but I just tried the above on the wingrass dosbox command line and it worked! I've a nice pdf now.

Perhaps hardcode the above long command into an if(wingrass) part of the composer code, or maybe cleaner to write a little ps2pdf.bat or ps2pdf.pyw (.pyw to avoid a dosbox popping up; we know python is there..) script to do it?

minor cosmetics:

• "Generating PostScript..." and "Generating PDF..." never go away after it's done. Can we clear the status bar after the file is closed?

• I also notice when you add a mapframe with a raster in wingrass the text in the green rectangle on the draft mode page shows as "MAP FRAME[]raster: mapname"; the newline in the string is rendered as an unknown char instead of a newline.

thanks, Hamish

[hamish]

oh, and we might want to offer draft/print quality option for ps2pdf, see ​http://grasswiki.osgeo.org/wiki/Ps.map_scripts#Converting_PostScript_to_PDF

the default for gs is to reduce to 72dpi which is fine for on the computer monitor and presentations, but not so good for printing hardcopies. it might be worth considering making the high quality version the default (the file size is bigger, but it doesn't scale as hugely you might expect, more like 75% qual JPEG -> 90% qual JPEG).

-dPDFSETTINGS=/prepress -r1200

that would need to be added twice, near the -dCompatibilityLevel= options, from ps2pdfwr:

# We have to include the options twice because -I only takes effect if it
# appears before other options.

(but we are not using -I, so perhaps it doesn't matter and the early -P- -dSAFER -dCompatibilityLevel=1.4 can be removed)

Hamish

[martinl]

Replying to hamish:

pressing the PDF button comes back with a pop-up window "ps2pdf exited with return code 1".

ps2pdf seems to be missing, we could add it there or maybe call gs directly?

right, no ps2pdf in \extrabin.

On Debian, ps2pdf comes with the ghostscript package, it's just a shell script which just calls:

I have added ps2pdf script to osgeo4w gs package (1). Testing welcomed.

(1) ​http://trac.osgeo.org/osgeo4w/wiki/pkg-gs

[martinl]

Replying to martinl:

I have added ps2pdf script to osgeo4w gs package (1). Testing welcomed.

(1) ​http://trac.osgeo.org/osgeo4w/wiki/pkg-gs

Quick test: ps2pdf works from cmd, cartographic composer fails with Cannot open this file: _.at

[annakrat]

Replying to martinl:

Replying to martinl:

I have added ps2pdf script to osgeo4w gs package (1). Testing welcomed.

(1) ​http://trac.osgeo.org/osgeo4w/wiki/pkg-gs

Quick test: ps2pdf works from cmd, cartographic composer fails with Cannot open this file: _.at

and it complains about missing gssetgs.bat (in terminal)

[annakrat]

Replying to hamish:

minor cosmetics:

• "Generating PostScript..." and "Generating PDF..." never go away after it's done. Can we clear the status bar after the file is closed?

I added statusbar messages after exporting PS and PDF files.

• I also notice when you add a mapframe with a raster in wingrass the text in the green rectangle on the draft mode page shows as "MAP FRAME[]raster: mapname"; the newline in the string is rendered as an unknown char instead of a newline.

this should be fixed already (see comment:22)

[martinl]

Replying to annakrat:

Quick test: ps2pdf works from cmd, cartographic composer fails with Cannot open this file: _.at

and it complains about missing gssetgs.bat (in terminal)

added to gs package (9.07-3)

[annakrat]

Replying to martinl:

Replying to annakrat:

Quick test: ps2pdf works from cmd, cartographic composer fails with Cannot open this file: _.at

and it complains about missing gssetgs.bat (in terminal)

added to gs package (9.07-3)

still not working, some 'unrecoverable error' shows in the terminal

[annakrat]

Hamish, please test as soon as possible r56077 (in grass6.5) so I can backport it if working. PDF is created now directly with gs instead of ps2pdf. I used the command from comment:34.

[annakrat]

Replying to annakrat:

Hamish, please test as soon as possible r56077 (in grass6.5) so I can backport it if working. PDF is created now directly with gs instead of ps2pdf. I used the command from comment:34.

Since we don't have much time before release, I committed it directly to 6.4, it cannot break anything.

[hamish]

Replying to annakrat:

Hamish, please test as soon as possible r56077 (in grass6.5) so I can backport it if working. PDF is created now directly with gs instead of ps2pdf. I used the command from comment:34.

thanks, fingers crossed :) too late here tonight so I'll test in the morning once Martin's build is complete.

it worked from the C:\> prompt, so with luck & no typos..

Since we don't have much time before release, I committed it directly to 6.4, it cannot break anything.

Once I thought the same; you'll see a 6.3.0.1 version in the svn tags because I was wrong about it. :)

thanks, Hamish

[annakrat]

Issue with the statusbar messages is fixed now (r56085), PDF generating is working finally too. Preview is working except for raster maps which are not displayed (but the final PS, PDF is fine). Since this is not likely to be solved in the near future and this ticket is getting unreadable I am closing this ticket as solved and creating a new one for this issue (#1956).

[hamish]

Anna wrote:

PDF generating is working finally too.

tested in wingrass nightly build, works great, thanks! wonderful to have that working.

Hamish

(argh.. trac logouts continue..)