gdal_translate (2.0.0dev) performance degrades on wide VRTs (VRT to PNG only, TIFF OK)

[riveryeti]

I am using the commandline gdal_translate -of png -co "ZLEVEL=1" [filename].vrt [filename].png to translate two rasters from VRT to PNG (also converting to tiled tiff, but that works fine).

Given two vrt "rasters" with sizes as follows:

191488,46080 (wide raster) 59392,201728 (tall raster)

performance degrades to about 20bytes/sec on the wide raster when I hit about 30% processing (still at 30% + 1 dot 3 hours later). On the other hand, performance does not noticeably degrade when processing the tall raster, and the entire translation takes about 2-3 minutes.

both vrts are built from 1024x1024 uncompressed TIFF tiles generated using otherwise the same methods/settings/projection/coordinate system/etc.

gdalinfo from "good" vrt below (excluding file-list) vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv

Size is 59392, 201728
Coordinate System is:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433],
    AUTHORITY["EPSG","4326"]]
Origin = (-123.610326674747300,48.157496099385916)
Pixel Size = (0.000001641647305,-0.000001100758214)
Corner Coordinates:
Upper Left  (-123.6103267,  48.1574961) (123d36'37.18"W, 48d 9'26.99"N)
Lower Left  (-123.6103267,  47.9354423) (123d36'37.18"W, 47d56' 7.59"N)
Upper Right (-123.5128260,  48.1574961) (123d30'46.17"W, 48d 9'26.99"N)
Lower Right (-123.5128260,  47.9354423) (123d30'46.17"W, 47d56' 7.59"N)
Center      (-123.5615763,  48.0464692) (123d33'41.67"W, 48d 2'47.29"N)
Band 1 Block=128x128 Type=Byte, ColorInterp=Red
  NoData Value=0
Band 2 Block=128x128 Type=Byte, ColorInterp=Green
  NoData Value=0
Band 3 Block=128x128 Type=Byte, ColorInterp=Blue
  NoData Value=0
Band 4 Block=128x128 Type=Byte, ColorInterp=Alpha
  NoData Value=0

gdalinfo from "bad" vrt below (excluding file-list) vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv

Size is 191488, 46080
Coordinate System is:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433],
    AUTHORITY["EPSG","4326"]]
Origin = (-123.657297355231180,48.158143203488351)
Pixel Size = (0.000001374831964,-0.000000920181878)
Corner Coordinates:
Upper Left  (-123.6572974,  48.1581432) (123d39'26.27"W, 48d 9'29.32"N)
Lower Left  (-123.6572974,  48.1157412) (123d39'26.27"W, 48d 6'56.67"N)
Upper Right (-123.3940335,  48.1581432) (123d23'38.52"W, 48d 9'29.32"N)
Lower Right (-123.3940335,  48.1157412) (123d23'38.52"W, 48d 6'56.67"N)
Center      (-123.5256654,  48.1369422) (123d31'32.40"W, 48d 8'12.99"N)
Band 1 Block=128x128 Type=Byte, ColorInterp=Red
  NoData Value=0
Band 2 Block=128x128 Type=Byte, ColorInterp=Green
  NoData Value=0
Band 3 Block=128x128 Type=Byte, ColorInterp=Blue
  NoData Value=0
Band 4 Block=128x128 Type=Byte, ColorInterp=Alpha
  NoData Value=0

end.

[Even Rouault]

• Is it specific to gdal 2.0dev ? (That would be surprising)
• I think you're confusing wide and tall rasters. Size is 59392, 201728 means "width = 59392 and height = 201712", so this is a tall raster.

It is not completely surprising that wide rasters are slower than tall rasters as copying operates by chunks of lines. You could try increasing the GDAL block cache size to 200 MB for example by setting GDAL_CACHEMAX=200 as environment variable.

[riveryeti]

Whoops, thanks rouault for catching that. I mixed up the wide and tall labels. Corrected now. Haven't tested other versions. Did try changing cache size to 256 MB then 2048 MB. Could try larger value too (I have 192 GB RAM). Also tried converting single TIF --> PNG, and never got past 0 pct (only ran for 10 minutes).