Opened 12 years ago

Closed 8 years ago

#3898 closed defect (fixed)

gdalwarp doesn't preserve dataset and band-level metadata

Reported by: jrheard Owned by: etourigny
Priority: normal Milestone:
Component: Utilities Version:
Severity: normal Keywords: gdalwarp, metadata
Cc: warmerdam

Description

When warping GRIB files to GeoTIFF, I discovered that all my band level metadata was being erased in the warp. This does not happen for other command line utilities that read GRIBs and write GeoTIFFs, only gdalwarp and gdal_transform.

Attached is a sample GRIB file from the NDFD. This bug can easily be reproduced with the following command line:

$ gdalwarp -of GTiff -t_srs EPSG:900913 ds.wx.bin

Attachments (8)

ds.wx.bin (116.7 KB ) - added by jrheard 12 years ago.
gdalwarp-metadata.patch (3.2 KB ) - added by etourigny 11 years ago.
patch for copying metadata from first source
netcdf-4d.nc (4.3 KB ) - added by etourigny 11 years ago.
file with dataset and band-level metadata
tmp1.nc (2.8 KB ) - added by etourigny 11 years ago.
tmp2.nc (2.8 KB ) - added by etourigny 11 years ago.
tmp1.tif (265.0 KB ) - added by etourigny 11 years ago.
tmp2.tif (337.3 KB ) - added by etourigny 11 years ago.
tmp3.tif (539.7 KB ) - added by etourigny 11 years ago.

Download all attachments as: .zip

Change History (19)

by jrheard, 12 years ago

Attachment: ds.wx.bin added

comment:1 by etourigny, 11 years ago

Summary: gdalwarp doesn't preserve band-level metadatagdalwarp doesn't preserve dataset and band-level metadata
Version: 1.5.4

This has also been discussed on the mailing list at http://osgeo-org.1560.n6.nabble.com/gdal-dev-lost-GeoTiff-file-metadata-with-gdalwarp-td3742896.html

Attaching a patch to gdalwarp (against trunk) which copies dataset and band-level metadata.

The metadata is only copied from the first source (if there is only one source), and band-level metadata is copied if source and destination band count are equal.

New option -nomd prevents metadata from being copied.

A complete implementation could check for identical items in all sources, but this should be sufficient for many cases IMHO.

by etourigny, 11 years ago

Attachment: gdalwarp-metadata.patch added

patch for copying metadata from first source

by etourigny, 11 years ago

Attachment: netcdf-4d.nc added

file with dataset and band-level metadata

comment:2 by warmerdam, 11 years ago

While I'm a bit nervous about it, I think this will be on balance a positive change (in trunk). I think it is too potentially disruptive to back port.

comment:3 by etourigny, 11 years ago

Agreed.

Would it be worth it or necessary to check for identical MD in all sources? If so it would be nice to have some test datasets (with slightly different metadata) to test with.

comment:4 by warmerdam, 11 years ago

Comparing all the datasets and reporting something useful if they differ seems involved. I'd just stress in the documentation that metadata (and stuff like color tables presumably) are copied from the first input image when multiple images are mosaiced.

comment:5 by jrheard, 11 years ago

Thanks, warmerdam, for fixing this. It was really quite painful.

comment:6 by warmerdam, 11 years ago

Etienne is fixing it, I'm just providing moral support.

comment:7 by warmerdam, 11 years ago

Cc: warmerdam added
Owner: changed from warmerdam to etourigny

comment:8 by etourigny, 11 years ago

ok I'll implement my patch and update gdalwarp documentation.

jrheard - can you test this patch or the svn trunk?

BTW - is the dev version to be 2.0 like I have seen in come of Even's commits to driver docs) or 1.10?

comment:9 by etourigny, 11 years ago

Fixed in trunk (r24089).

I have added conflict detection, default is to set conflicting values to "*", use -cfmd option to set to other value (or "") to remove them.

See documentations for more info.

Also added some band information like Description and Unit Type, there should probably should be other elements as well.

Attaching test datasets which work well with current implementation.

by etourigny, 11 years ago

Attachment: tmp1.nc added

by etourigny, 11 years ago

Attachment: tmp2.nc added

by etourigny, 11 years ago

Attachment: tmp1.tif added

by etourigny, 11 years ago

Attachment: tmp2.tif added

by etourigny, 11 years ago

Attachment: tmp3.tif added

comment:10 by Even Rouault, 11 years ago

I don't think it is yet decided if the next major version will be 1.10 or 2.0. It will depend on whether someone wants to put forward a breaking change or not... It will not be complicated to "s/2.0/1.10/" in the docs in case the next version is 1.10 finally...

comment:11 by Jukka Rahkonen, 8 years ago

Resolution: fixed
Status: newclosed

Seems to be implemented in GDAL 1.10. From http://www.gdal.org/gdalwarp.html:

-nomd:
    (GDAL >= 1.10.0) Do not copy metadata. Without this option, dataset and band metadata (as well as some band information) will be copied from the first source dataset. Items that differ between source datasets will be set to * (see -cvmd option). 
-cvmd meta_conflict_value:
    (GDAL >= 1.10.0) Value to set metadata items that conflict between source datasets (default is "*"). Use "" to remove conflicting items.

Note: See TracTickets for help on using tickets.