Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#5119 closed defect (fixed)

gdal_translate -sds uses incorrect subdataset filenames, simply appends a number to the end of target filename

Reported by: etourigny Owned by: etourigny
Priority: normal Milestone:
Component: Utilities Version: unspecified
Severity: normal Keywords: translate, subdatsets
Cc: Even Rouault, warmerdam

Description

This was raised in the mailing list http://lists.osgeo.org/pipermail/gdal-dev/2013-May/036373.html

when I run:

gdal_translate -sds -of "GTiff" [some.hdf] "myoutput"

The output filenames do not have the .tif extension.  If I set the output
file to "myoutput.tif", it appends the "layer numbers" to the end of the
file, e.g. myoutput.tif1.  Any tricks to getting the naming working
properly (myoutput1.tif would be what I'd expect)?

example with a netcdf file with 2 subdatasets:

tourigny@supernova: /data/src/gdal/svn/trunk/autotest/gdrivers/data/tmp $ gdal_translate -sds netcdf_2vars.nc tmp2.tif
Input file size is 10, 10
0...10...20...30...40...50...60...70...80...90...100 - done.
Input file size is 10, 10
0...10...20...30...40...50...60...70...80...90...100 - done.
tourigny@supernova: /data/src/gdal/svn/trunk/autotest/gdrivers/data/tmp $ ls
netcdf_2vars.nc  tmp2.tif1  tmp2.tif2

resulting files should be tmp21.tif tmp22.tif

preparing a fix which would add an underscore (for easier reading) and also zero-pad if there are more than 9 subdatasets.

e.g. tmp_01.tif [...] tmp_10.tif

as this would change behavior of gdal_translate, I am not sure if this should go into 1.9 and 1.10. Any thoughts?

Attachments (4)

netcdf_2vars.nc (16.2 KB) - added by etourigny 4 years ago.
sample dataset with 2 subdatasets
netcdf_10vars.nc (3.1 KB) - added by etourigny 4 years ago.
patch_translate_sds1.txt (1.9 KB) - added by etourigny 4 years ago.
patch
patch_translate_sds2.txt (1.5 KB) - added by etourigny 4 years ago.

Download all attachments as: .zip

Change History (17)

Changed 4 years ago by etourigny

Attachment: netcdf_2vars.nc added

sample dataset with 2 subdatasets

Changed 4 years ago by etourigny

Attachment: netcdf_10vars.nc added

Changed 4 years ago by etourigny

Attachment: patch_translate_sds1.txt added

patch

comment:1 Changed 4 years ago by Even Rouault

Etienne,

char papszTokens = CSLTokenizeString2(pszDest,".",0); doesn't look like a relieable way of detecting the extension in case the output file is something like "odd.directory.name/odd.output.filename.tif". I'd encourage you to use CPLGetPath(), CPLGetBasename() and CPLGetExtension() to split the components, and then CPLFormFilename() to build the new filename.

As this is a change of behaviour (some people might depend on the current behaviour, even if it is admittedly odd), I'd rather push that change just in trunk. Initiating a "behavioral changes" section in the NEWS file might also be appropriate.

Changed 4 years ago by etourigny

Attachment: patch_translate_sds2.txt added

comment:2 Changed 4 years ago by stuartehamilton

This is quite a major issue when processing many hdf files which is often the case when working with hdf format and subdataset in gdal_translate (MODIS for example). We have a work around appending another .tif to the end of the filename (tif1.tif, tif2.tif). It is also referenced as a problem by others at http://gis.stackexchange.com/questions/69451/issue-with-sds-naming-convention-in-gdal-translate. Im not sure why anyone would be reliant on the current method of processing of .tif1, .tif2 etc.

comment:3 Changed 4 years ago by etourigny

can you test the last patch attached here?

comment:4 in reply to:  3 Changed 4 years ago by stuartehamilton

Replying to etourigny:

can you test the last patch attached here?

sure can. The link is missing though.

comment:5 Changed 4 years ago by etourigny

I meant the patch attached in this ticket (see link above), patch_translate_sds2.txt

comment:6 in reply to:  5 Changed 4 years ago by stuartehamilton

Replying to etourigny:

I meant the patch attached in this ticket (see link above), patch_translate_sds2.txt

Got it

comment:7 Changed 4 years ago by stuartehamilton

Perfect. We ran on modis data and got the correct return (out_01.tif out_02.tif out_03.tif out_04.tif out_05.tif out_06.tif out_07.tif out_08.tif out_09.tif out_10.tif out_11.tif out_12.tif)

Thanks

I will look out for the update will it be in 1.9 or 1.10.

comment:8 Changed 4 years ago by etourigny

actually this won't make it into 1.9 or 1.10, see Even's comment above. If you say it is satisfactory then I will add it to trunk, which will make it into 1.11 or 2.0

comment:9 Changed 4 years ago by stuartehamilton

It is satisfactory.

comment:10 Changed 4 years ago by etourigny

Resolution: fixed
Status: newclosed

commited to trunk (r26400). I have updated the docs but not the NEWS file.

comment:11 Changed 4 years ago by Kyle Shannon

Resolution: fixed
Status: closedreopened

Etienne, test_gdal_translate_15 in autotest/utilities/test_gdal_translate.py looks like it needs to be updated. I think r26400 broke the test.

comment:12 Changed 4 years ago by Even Rouault

Resolution: fixed
Status: reopenedclosed

I was tired of the travis bots spamming me about failed builds, so ...r26416 "Fix failed tests after changes done in #5232 and #5119"

comment:13 Changed 4 years ago by etourigny

my sincere apologies for not looking into this sooner, I was away.

Note: See TracTickets for help on using tickets.