Color problems when using raster symbols and 8 bits PNG

[dmorissette]

We have noticed a problem with MapServer V4.0 when drawing raster symbols on a
map and using 8 bits PNG output.  Sometimes the color of the raster symbol will
change.

In our specific case, we were using a png symbol that contained red color.  
- If the rest of the map also contained some red elsewhere (in the roads layer
for instance) then the symbol was displayed in red correctly
- If there was no red color anywhere else in the map then the symbol was drawn
in black instead of red.

The above was happening when using 8 bits PNG output.  Switching to 24 bits PNG
or JPEG output resolves the issue.  Sounds very much like a color table issue.

Please keep in mind that this happens with raster symbols, not vector symbols.

Unfortunately we don't have a simple reproduceable test case for this.  If we
can build one then we'll attach it.

[dmorissette]

There was a report of a similar issue on the mapserver-users list.  I'm not sure
that it's directly related but I'll copy it here just in case.  Shannon, can you
check what happens if you use PNG24 output and report to this bug?

On 2003-08-05, Shannon Scott wrote:
> 
> Hello,
> I created an interstate route shield PNG image.  I have found that the 
> color of the image changes sometimes.
> The issue seems to be related to the percentage of the image covered by 
> another color ( green in this example ).
> Has anyone ever seen this before?
> Does anyone know what could cause this?
> I have provided a couple links to help illustrate the issue.
>  
> http://216.220.241.10/images/shield_good.png
>  
> http://216.220.241.10/images/shield_notasgood.png
>  
> It is an issue in 3.6.x and 4.0.
> Thank You for any help or info.
> Shannon
>  
> Apache CGI
> Linux RedHat 8.0
> MN Mapserver 4.0
>  
>  

[fwarmerdam]

Is there some reason to believe the problem is anything other than having
run out of colors in the 8bit palette?

If by the time we get to rendering raster symbols we don't have any colors
left in the palette then there isn't much we can do other than pick the nearest
color is there?  

By the way, is there a way of declaring an exact color in the map file that we
would like to be added to the palette right off the bat?  This might be helpful
for situations where particular colors need to be reproduced quite exactly
even if it's use comes late in the rendering process.

[dmorissette]

I would be surprised if it was a color table overflow since we don't use
rasters, the mapfile contains only vector data and probably uses less than 20
colors total.  Say 50 to be real safe.  Also this same mapfile works fine with
mapserv CGI in 3.6.

Julien will try to produce a new image with this problem and report on the
number of colors used.  We can't attach the image to the bug but could send it
to you directly if you need it.

[sscott@…]

When I change the mapfile to include:

IMAGETYPE png24

OUTPUTFORMAT
  NAME png24
  DRIVER "GD/PNG"
  IMAGEMODE RGBA
  TRANSPARENT ON
END

I cannot reproduce the problem.
I will check to see if I have a GD or libpng version/multicopy problem.
Let me know if I can help further.
Shannon

[pspencer@…]

same problem (red symbols appear gray only at some scales) with imagetype gif 
and png
and vector symbols (e.g. circle)
Symbol
Name 'circle'
Type ELLIPSE
  Points
   2 2
  END
#Filled true
END

Works with jpeg and png24.

[sdlime]

I really think this is a color table issue. It would really be nice to get some
test data and mapfiles. Tough to debug otherwise. 

I suspect what's happening, at least with shields is a result of antialiasing.
With a shield the layer is most often of type annotation so the shield itself
doesn't get drawn until the label itself, well after other colors have been
allocated. With antialiasing the color table could fill up quickly so that by
the time the shield label is drawn there are no colors left. On solution might
be to allocate colors in the main image for annotation layers using a marker
when a feature is placed in the cache. Then the antialiasing process can fight
over remaining colors, usually not as noticeable.

I'm curious about Paul's last message. It may indicate a different problem but
there's not enough detail to know under what circumstance the circle symbol is
being used.

As for reserving colors (Frank's idea). What about adding to the image format
section? Perhaps something like:

COLORS
  255 0 0
  0 255 0
END

So that when the image is opened the reserved colors could quickly be allocated
(after the background color).

Steve

[pspencer@…]

I'll try to put together an example.  I would say from my point of view that it
is not directly a colour table problem.  In one case, there was a raster (wms)
layer, but in the other case, there was no raster.  It did seem to be
scale-dependent in the non-raster case.  Very strange :)

[dmorissette]

In our case the problem seemed to be scale dependent too, but it was not
magic... it turned out that as we zoomed in then the set of colors used in the
image were different, and the problem started happening (in our case) when there
were no more red roads in the view and we were displaying a red symbol.

Also, since in Paul's case and our case this didn't happen in 3.6 which also had
the same 8 bits limitation that kind of rules out the possibility of a color
table becoming full.  (But we should still verify this possibility before ruling
it out completely)

[sdlime]

3.6 handled colors differently though. It made a pass through the mapfile and
allocated colors for all layers (except for raster layers) before any processing
started. This was necessary to assign colors index values. In 4.O we store RGB
values and allocate colors as necessary. The palettes in 3.6 would be more
consistent between maps. This approach lead to more colors being allocated than
necessary.

[dmartin@…]

I'm getting the same problem with the color of antialiased fonts on text 
lables.  Text labels are not the correct color unless the color is in the image 
anyway (from a same-colored symbol).  It can be fixed by turning off 
antialiasing (yuk) or using IMAGETYPE PNG24 (but then you end up with huge 
files).  The problem is also there when using GIF, but not when using JPEG.  
The problem did not exist with the same map under 3.6.

This is on the Windows binary 4.0 build from the UMN website.

If this should be filed as another bug, please let me know and I will do so.  
If someone would like screenshots or map definitions, I can provide those as 
well.

[sdlime]

I just fixed bug 306 which reserves colors for layers that are ON/DEFAULT for
8-bit output. The fix only considers colors defined in the mapfile and not
colors in a symbol image. Pre-4.0 did the same thing and also didn't
pre-allocate symbol image colors. You can now define colors ahead of time:

e.g.

SYMBOL 'redmaker.png'
COLOR 255 0 0 # reserve the color even though COLOR is ignored

May be an ok work around. The color issues with vector symbols have been
resolved. There was a problem with the symbol image cache.

I'm tempted to mark this bug as WONTFIX since it is such a hodgepodge. Dan, what
do you think?

Steve

[dmorissette]

We could possibly fix this by scanning the color tables of the pixmap symbols
but that would be expensive to do on each request. You can mark as WONTFIX if
you like, unless someone else has objections. With the vector color fix from bug
306 we should be fine.

[tbonfort]

this issue is obsolete for 6.0.