Opened 4 months ago

Closed 4 months ago

Last modified 4 months ago

#6836 closed defect (fixed)

Making mbtiles with alpha band depends on resolution

Reported by: mattwilkins Owned by: warmerdam
Priority: normal Milestone:
Component: default Version: unspecified
Severity: normal Keywords:
Cc:

Description (last modified by Jukka Rahkonen)

Hi,

I can't find a general mailing list to ask questions, so sorry, I have to pose this as a bug. But I'm probably making a mistake somewhere.

I have a geotiff that I want to make into a mbtiles with an alpha band. This sequence of commands makes a mbtile with an alpha band:

 gdalwarp  -tr 0.02 0.02   -r average 11.tif tmp.tif
 gdal_translate tmp.tif foo.mbtiles  -of MBTILES

but this sequence of commands (NB: just the resolution different) results in an mbtiles without an alpha band:

 gdalwarp  -tr 0.0002 0.0002   -r average 11.tif tmp.tif
 gdal_translate tmp.tif foo.mbtiles  -of MBTILES

The input file 11.tif is kinda large so not sure I want to submit it here. But this is its gdalinfo

Driver: GTiff/GeoTIFF
Files: 11.tif
Size is 4863, 2999
Coordinate System is:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433],
    AUTHORITY["EPSG","4326"]]
Origin = (173.727848284398078,-37.696798582530377)
Pixel Size = (0.000224206890208,-0.000224206890208)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  ( 173.7278483, -37.6967986) (173d43'40.25"E, 37d41'48.47"S)
Lower Left  ( 173.7278483, -38.3691950) (173d43'40.25"E, 38d22' 9.10"S)
Upper Right ( 174.8181664, -37.6967986) (174d49' 5.40"E, 37d41'48.47"S)
Lower Right ( 174.8181664, -38.3691950) (174d49' 5.40"E, 38d22' 9.10"S)
Center      ( 174.2730073, -38.0329968) (174d16'22.83"E, 38d 1'58.79"S)
Band 1 Block=4863x1 Type=Byte, ColorInterp=Red
  Mask Flags: PER_DATASET ALPHA
Band 2 Block=4863x1 Type=Byte, ColorInterp=Green
  Mask Flags: PER_DATASET ALPHA
Band 3 Block=4863x1 Type=Byte, ColorInterp=Blue
  Mask Flags: PER_DATASET ALPHA
Band 4 Block=4863x1 Type=Byte, ColorInterp=Alpha

Note that I tried doing a

gdalwarp -t_srs EPSG:3857 -r average 11.tif  tmp2.tif

first and I got similar results: this didn't work,

 gdalwarp  -tr 40 40   -r average tmp2.tif tmp.tif
 gdal_translate tmp.tif foo.mbtiles  -of MBTILES

but this did:

 gdalwarp  -tr 4000 4000   -r average tmp2.tif tmp.tif
 gdal_translate tmp.tif foo.mbtiles  -of MBTILES

and the gdalinfo of tmp2.tif is

Driver: GTiff/GeoTIFF
Files: tmp2.tif
Size is 11277, 16132
Coordinate System is:
PROJCS["WGS 84 / Pseudo-Mercator",
    GEOGCS["WGS 84",
        DATUM["WGS_1984",
            SPHEROID["WGS 84",6378137,298.257223563,
                AUTHORITY["EPSG","7030"]],
            AUTHORITY["EPSG","6326"]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]],
        AUTHORITY["EPSG","4326"]],
    PROJECTION["Mercator_1SP"],
    PARAMETER["central_meridian",0],
    PARAMETER["scale_factor",1],
    PARAMETER["false_easting",0],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AXIS["X",EAST],
    AXIS["Y",NORTH],
    EXTENSION["PROJ4","+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs"],
    AUTHORITY["EPSG","3857"]]
Origin = (19339295.607630282640457,-4533291.924883016385138)
Pixel Size = (20.000000000000000,-20.000000000000000)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  (19339295.608,-4533291.925) (173d43'40.25"E, 37d40'21.72"S)
Lower Left  (19339295.608,-4855931.925) (173d43'40.25"E, 39d55'51.88"S)
Upper Right (19564835.608,-4533291.925) (175d45'14.07"E, 37d40'21.72"S)
Lower Right (19564835.608,-4855931.925) (175d45'14.07"E, 39d55'51.88"S)
Center      (19452065.608,-4694611.925) (174d44'27.16"E, 38d48'39.02"S)
Band 1 Block=11277x1 Type=Byte, ColorInterp=Red
  Mask Flags: PER_DATASET ALPHA
Band 2 Block=11277x1 Type=Byte, ColorInterp=Green
  Mask Flags: PER_DATASET ALPHA
Band 3 Block=11277x1 Type=Byte, ColorInterp=Blue
  Mask Flags: PER_DATASET ALPHA
Band 4 Block=11277x1 Type=Byte, ColorInterp=Alpha

Thanks for any help.

Matt

Attachments (1)

Screenshot_2017-03-16-22-18-24.png (288.2 KB) - added by mattwilkins 4 months ago.
Black border around map

Download all attachments as: .zip

Change History (13)

comment:1 Changed 4 months ago by Jukka Rahkonen

Gdal-dev list is also for users https://lists.osgeo.org/mailman/listinfo/gdal-dev.

I guess that the reason for your experience is that alpha is added to the tiles only when needed. It is needed when the source raster data does not fill completely the tile that is created. For raster data alpha is rather useless if the whole tile is filled with true data pixels because there is no no-data at all and thus nothing to make transparent.

comment:2 Changed 4 months ago by mattwilkins

That makes sense, but all my images need alpha around the edge because the maps aren't orientated perfectly north south, east west. Is there anyway to force gdal_translate to always use alpha? Thanks!

comment:3 Changed 4 months ago by Jukka Rahkonen

But all your tiles do not need alpha, only those tiles which are at the borders and contain either pure nodata or partially data and partially nodata. The situation is actually the same even for north-sound oriented images because usually the image borders do not fit exactly with the tile borders.

Make a small downsampled version of your input file 11.tif, ensure that your issue can be reproduced with it and add the image as an attachment. If it is still too large put it somewhere for downloading.

comment:4 Changed 4 months ago by mattwilkins

You are welcome to give it a go with 11.tif. It is at http://mcw.massey.ac.nz/11.tif Thanks! Matt

comment:5 Changed 4 months ago by Jukka Rahkonen

Remember that alpha is set per tile and only when needed.

I re-used your commands with GDAL 2.2dev.

gdalwarp -tr 0.0002 0.0002 -r average 11.tif tmp.tif
gdal_translate tmp.tif foo.mbtiles -of MBTILES

Then I renamed the database into foo.sqlite so I could browse the data with spatialite-gui. I selected these two tiles, first one shows data from the border of your map and the second one from the beginning. The theory is that tile number one has alpha but tile number two does not.

Tile with nodata, exported as image_alpha.png with spatialite-gui

SELECT * from "tiles"
where "zoom_level"=13 and "tile_column"=8073 and "tile_row"=3168;

Tile without nodata, exported as image.png with spatialite-gui

SELECT * from "tiles"
where "zoom_level"=13 and "tile_column"=8051 and "tile_row"=3165;

Gdalinfo:

Supposed to have alpha:

gdalinfo image_alpha.png
Driver: PNG/Portable Network Graphics
Files: image_alpha.png
Size is 256, 256
Coordinate System is `'
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,  256.0)
Upper Right (  256.0,    0.0)
Lower Right (  256.0,  256.0)
Center      (  128.0,  128.0)
Band 1 Block=256x1 Type=Byte, ColorInterp=Red
  Mask Flags: PER_DATASET ALPHA
Band 2 Block=256x1 Type=Byte, ColorInterp=Green
  Mask Flags: PER_DATASET ALPHA
Band 3 Block=256x1 Type=Byte, ColorInterp=Blue
  Mask Flags: PER_DATASET ALPHA
Band 4 Block=256x1 Type=Byte, ColorInterp=Alpha

Supposed to be without alpha

gdalinfo image.png
Driver: PNG/Portable Network Graphics
Files: image.png
Size is 256, 256
Coordinate System is `'
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,  256.0)
Upper Right (  256.0,    0.0)
Lower Right (  256.0,  256.0)
Center      (  128.0,  128.0)
Band 1 Block=256x1 Type=Byte, ColorInterp=Red
Band 2 Block=256x1 Type=Byte, ColorInterp=Green
Band 3 Block=256x1 Type=Byte, ColorInterp=Blue

Result seems to be correct at least with my GDAL version.

comment:6 Changed 4 months ago by Jukka Rahkonen

Description: modified (diff)

Changed 4 months ago by mattwilkins

Black border around map

comment:7 Changed 4 months ago by mattwilkins

Thanks for helping me out with this, that is interesting, I didn't know you could have tiles some with alpha and some without. That got me a bit mixed up, I was just using gdalinfo (on the entire file, not tile) to determine if the alpha was there or not.

OK, so I have tracked down a bit more of the problem. It appears when I add the extra levels with gdaladdo I get the black border. So

gdalwarp -r average 11.tif tmp.tif

gdal_translate tmp.tif foo.mbtiles -of MBTILES

produces a nice foo.mbtiles with alpha at the edges, but when I run

gdaladdo foo.mbtiles 2 4 8

I get a mbtiles that is fine (transparent along the crooked edge), but when I zoom out I see the non transparent border. So, assuming I am understanding this correctly (you are probably right in thinking not at this stage!), how can I get that alpha band on the edges at the other zoom levels?

I've attached a screenshot

Thank you so much for your help!

Matt

comment:8 Changed 4 months ago by mattwilkins

Oh, and here are the gdalinfo's on foo.mbtiles BEFORE doing the gdaladdo and then AFTER

Driver: MBTiles/MBTiles Files: foo.mbtiles Size is 6352, 4973 Coordinate System is: PROJCS["WGS 84 / Pseudo-Mercator",

GEOGCS["WGS 84",

DATUM["WGS_1984",

SPHEROID["WGS 84",6378137,298.257223563,

AUTHORITY["EPSG","7030"]],

AUTHORITY["EPSG","6326"]],

PRIMEM["Greenwich",0,

AUTHORITY["EPSG","8901"]],

UNIT["degree",0.0174532925199433,

AUTHORITY["EPSG","9122"]],

AUTHORITY["EPSG","4326"]],

PROJECTIONMercator_1SP?, PARAMETER["central_meridian",0], PARAMETER["scale_factor",1], PARAMETER["false_easting",0], PARAMETER["false_northing",0], UNIT["metre",1,

AUTHORITY["EPSG","9001"]],

AXIS["X",EAST], AXIS["Y",NORTH], EXTENSION["PROJ4","+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs"], AUTHORITY["EPSG","3857"]]

Origin = (19339295.607630282640457,-4536681.723232603631914) Pixel Size = (19.109257071294063,-19.109257071294063) Metadata:

ZOOM_LEVEL=13 name=foo type=overlay description=foo version=1.1 format=png bounds=173.7278482843981,-38.36918997274248,174.8182413508043,-37.69679858253038 minzoom=13 maxzoom=13

Image Structure Metadata:

INTERLEAVE=PIXEL

Corner Coordinates: Upper Left (19339295.608,-4536681.723) (173d43'40.25"E, 37d41'48.47"S) Lower Left (19339295.608,-4631712.059) (173d43'40.25"E, 38d22' 9.08"S) Upper Right (19460677.609,-4536681.723) (174d49' 5.67"E, 37d41'48.47"S) Lower Right (19460677.609,-4631712.059) (174d49' 5.67"E, 38d22' 9.08"S) Center (19399986.608,-4584196.891) (174d16'22.96"E, 38d 2' 1.56"S) Band 1 Block=256x256 Type=Byte, ColorInterp?=Red Band 2 Block=256x256 Type=Byte, ColorInterp?=Green Band 3 Block=256x256 Type=Byte, ColorInterp?=Blue

Files: foo.mbtiles Size is 6352, 4973 Coordinate System is: PROJCS["WGS 84 / Pseudo-Mercator",

GEOGCS["WGS 84",

DATUM["WGS_1984",

SPHEROID["WGS 84",6378137,298.257223563,

AUTHORITY["EPSG","7030"]],

AUTHORITY["EPSG","6326"]],

PRIMEM["Greenwich",0,

AUTHORITY["EPSG","8901"]],

UNIT["degree",0.0174532925199433,

AUTHORITY["EPSG","9122"]],

AUTHORITY["EPSG","4326"]],

PROJECTIONMercator_1SP?, PARAMETER["central_meridian",0], PARAMETER["scale_factor",1], PARAMETER["false_easting",0], PARAMETER["false_northing",0], UNIT["metre",1,

AUTHORITY["EPSG","9001"]],

AXIS["X",EAST], AXIS["Y",NORTH], EXTENSION["PROJ4","+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs"], AUTHORITY["EPSG","3857"]]

Origin = (19339295.607630282640457,-4536681.723232603631914) Pixel Size = (19.109257071294063,-19.109257071294063) Metadata:

ZOOM_LEVEL=13 name=foo type=overlay description=foo version=1.1 format=png bounds=173.7278482843981,-38.36918997274248,174.8182413508043,-37.69679858253038 maxzoom=13 minzoom=10

Image Structure Metadata:

INTERLEAVE=PIXEL

Corner Coordinates: Upper Left (19339295.608,-4536681.723) (173d43'40.25"E, 37d41'48.47"S) Lower Left (19339295.608,-4631712.059) (173d43'40.25"E, 38d22' 9.08"S) Upper Right (19460677.609,-4536681.723) (174d49' 5.67"E, 37d41'48.47"S) Lower Right (19460677.609,-4631712.059) (174d49' 5.67"E, 38d22' 9.08"S) Center (19399986.608,-4584196.891) (174d16'22.96"E, 38d 2' 1.56"S) Band 1 Block=256x256 Type=Byte, ColorInterp?=Red

Overviews: 3176x2487, 1588x1243, 794x622

Band 2 Block=256x256 Type=Byte, ColorInterp?=Green

Overviews: 3176x2487, 1588x1243, 794x622

Band 3 Block=256x256 Type=Byte, ColorInterp?=Blue

Overviews: 3176x2487, 1588x1243, 794x622

comment:9 Changed 4 months ago by Even Rouault

Resolution: fixed
Status: newclosed

In 37720:

MBTiles: on opening if detecting 3 bands, expose 4 bands since there might be transparent border tiles (fixes #6836)

comment:10 Changed 4 months ago by Even Rouault

In 37721:

MBTiles: on opening if detecting 3 bands, expose 4 bands since there might be transparent border tiles (fixes #6836)

comment:11 Changed 4 months ago by mattwilkins

Thanks for fixing it. I patched the file and it works great now!

To make my mbtiles file smaller I tried:

gdal_translate tmp.tif foo.mbtiles -of MBTILES -co TILE_FORMAT=JPEG

and indeed it was smaller, but then I get the black border again. I'm wondering if there is another bug somewhere, or it doesn't make sense to have JPEG tile format and expect an alpha band?

Thanks

comment:12 Changed 4 months ago by Even Rouault

it doesn't make sense to have JPEG tile format and expect an alpha band?

Yes, this is documented in http://gdal.org/frmt_mbtiles.htm "When using JPEG, the alpha channel will not be stored.". This is a limitation of the JPEG format. Unfortunately the mbtiles specification doesn't seem to allow a mix of JPEG and PNG tiles, as the GeoPackage tiled raster allows to do.

Note: See TracTickets for help on using tickets.