setting LAYER TRANSPARENCY causes entire map image to become transparent

[keon@…]

If I set the LAYER TRANSPARENCY option in 3.6.3, just that layer becomes
transparent, not the entire map image.  However, in 3.7 cvs, setting the LAYER
TRANSPARENCY option for a visible layer results in any other visible polygon
layers or raster background images also becoming transparent across the entire
map image.  This happens regardless of whether the OUTPUTFORMAT TRANSPARENT
option is set to ON or OFF, or not declared at all.  I'm using standard PC256
PNG format, with the GD/PNG driver.

Note that if TRANSPARENCY is set on a polygon layer, visible point and line
layers do NOT seem to be affected, but if TRANSPARENCY is set on a visible point
or line layer, the entire map image IS affected.

Not sure if this is a GDAL thing or not...

[fwarmerdam]

Dylan,

Sorry for not responding to this sooner.  Could you provide a simple example
that demonstrates this?  Ideally a map file, and supporting data files that
I can run shp2img against to demonstrate the problem?

Thanks,

[keon@…]

Frank,

I'll attach a tar.gz file containing a test case with a simple mapfile and
sample data.  This is reproducible with shp2img.  I'm using 19 Dec 2002 cvs
versions of MS 3.7 and GDAL. 

After further digging, it appears that the rest of the layers are given the
inverse of the value assigned to the layer with transparency.  For example,
there are two layers in the example I'm providing...a polygon layer and a single
DRG tiff.  If the polygon layer is given a value of TRANSPARENCY 80, the DRG
then seems to have a transparency of roughly 20.  Polygon TRANSPARENCY 40 = DRG
transparency 60, and so on.

However, this ONLY happens to layers that are "underneath" the layer with
TRANSPARENCY.  For example, when I have the polygon layer on top of the DRG, I
get the problem I described above.  If I switch the layers in the mapfile so
that the polygon layer is below the DRG, everything works fine.

Thanks,
Dylan

[keon@…]

The attachment is too large, due to the raster data included.  You can download
the tar.gz file here:  http://www.nacse.org/~keon/temp/example.tar.gz

[fwarmerdam]

The immediate bug is that gdImageColorTransparent() wasn't getting called
in msDrawLayer() because the test was "if( image_draw->format ==
MS_IMAGEMODE_PC256 )" instead of "if( image_draw->format->imagemode ==
MS_IMAGEMODE_PC256 )".  I am still a bit shocked this didn't produce a warning.

However, I have found that RGB/RGBA mode still doesn't work properly.  Mainly
this is because gdImageCopyMerge() (which we use to blend in the overlay layer
with transparency) does not have any logic to check the alpha in RGBA gdImage's. 
We will likely need to implement our own logic for this case. 

I would further add that if we work with an RGB image there will be no way of
knowing the areas of the overlayimage that should be completely transparent.
I think the local msApplyOutputFormat() logic will need a way of ensuring the
overlay image is RGBA if the main map imageFormat is RGB.  Perhaps the 
transparent flag should force RGB images into RGBA mode?  

So, three outstanding issues:

1) gdImageCopyMerge() ignores alpha in RGBA images.
2) RGB overlay images will not be able to have transparent areas. 
3) We need to add tests to the msautotest for various combinations of this.

[fwarmerdam]

I have implemented an msImageCopyMerge() that takes into account the alpha
components of true color images, and also modified msApplyOutputFormat() to
switch from RGB to RGBA if the transparency is requested to be force on. 

The msautotest/gdal/trlay1.map and trlay2.map autotests have been added for
the cases where the final output is RGB and RGBA respectively and things seem to
be working.  I also added an msOutputFormatValidate() function and modified the
map file parsing so that output formats with the transparency flag conflicting
with the image mode would trigger a parse time error. 

I believe this is now fully resolved.

[keon@…]

I have tested with the latest CVS version, and it works great.  Thanks Frank.