Opened 13 years ago

Closed 3 years ago

#2581 closed enhancement (wontfix)

netcdf: add metadata record for original array dimension when reading from multidimensional files

Reported by: mdsumner Owned by: etourigny
Priority: normal Milestone: closed_because_of_github_migration
Component: GDAL_Raster Version: unspecified
Severity: normal Keywords: netcdf
Cc: Markus Neteler, dnadeau, Kyle Shannon

Description

When accessing a file with 3D or 4D subdatasets such as NetCDF it's possible to discover the original array dimension via the "DESC=[TxZxYxX]" tag.

This information is not available for single dataset files - there is no analogous "DESC=[dim]" tag for multi-dim arrays from single dataset files.

The higher dimensions get unrolled into bands but there's no record of the original shape - having this metadata available as per SDS files would be useful.

An example file from the Unidata samples: http://www.unidata.ucar.edu/software/netcdf/examples/rhum.2003.nc

This file contains variable "rhum"(time=365, level=8, lat=73, lon=144) that is unrolled to a 2920 band raster.

Attachments (2)

foo.nc (4.0 KB) - added by mdsumner 13 years ago.
single 3D variable netCDF file
netcdf-4d.nc (4.4 KB) - added by etourigny 10 years ago.

Download all attachments as: .zip

Change History (14)

Changed 13 years ago by mdsumner

Attachment: foo.nc added

single 3D variable netCDF file

comment:1 Changed 13 years ago by mdsumner

Attached a more convenient file "foo.nc":

This contains a single 5x2x2 raster ([2x2x5] by NC convention) that unrolls to 2 bands:

Warning 1: No UNIDATA NC_GLOBAL:Conventions attribute Driver: netCDF/Network Common Data Format Files: foo.nc Size is 5, 2 Coordinate System is `' Corner Coordinates: Upper Left ( 0.0, 0.0) Lower Left ( 0.0, 2.0) Upper Right ( 5.0, 0.0) Lower Right ( 5.0, 2.0) Center ( 2.5, 1.0) Band 1 Block=5x1 Type=Float64, ColorInterp?=Undefined

NoData? Value=9.969209968386869e+036 Metadata:

NETCDF_VARNAME=temperature NETCDF_DIMENSION_x=1

Band 2 Block=5x1 Type=Float64, ColorInterp?=Undefined

NoData? Value=9.969209968386869e+036 Metadata:

NETCDF_VARNAME=temperature NETCDF_DIMENSION_x=2

comment:2 Changed 13 years ago by Markus Neteler

Cc: Markus Neteler added

comment:3 Changed 13 years ago by warmerdam

Cc: dnadeau added

comment:4 Changed 13 years ago by Even Rouault

Component: defaultGDAL_Raster
Keywords: netcdf added
Summary: add metadata record for original array dimension when reading from multidimensional filesnetcdf: add metadata record for original array dimension when reading from multidimensional files

comment:5 Changed 11 years ago by Kyle Shannon

Cc: Kyle Shannon added

comment:6 Changed 11 years ago by Kyle Shannon

Would a metadata item such as:

someVariable#original_dimensions="time=365, level=8, lat=73, lon=144"

or something along that line? As in a file level metadata item identifying variables and the associated dimensions. Sorry, I just found this ticket a couple of weeks ago.

comment:7 Changed 11 years ago by Kyle Shannon

Owner: changed from warmerdam to Kyle Shannon
Status: newassigned

comment:8 Changed 10 years ago by etourigny

in trunk (r24073) : write multi-dimensional variables to a single variable (not one for each unrolled band) in CreateCopy??() - see #2581; removed verbose output (define NCDF_DEBUG to print)

Unfortunately, it does not work for the test file attached here, because the order of dimensions does not adhere to the CF-1 standard. The fundamental problem is that there are no lat/lon or x/y dimensions, which the driver assumes.

The process is quite involved, will see if it can be adapted for general cases.

Attaching a test file (included in autotest/gdrivers/data) that was added in r24074.

Changed 10 years ago by etourigny

Attachment: netcdf-4d.nc added

comment:9 Changed 10 years ago by etourigny

Owner: changed from Kyle Shannon to etourigny
Status: assignednew

added autotest in trunk (r24090) to test writing a multidimensional file after using gdalwarp. One cannot directly call gdalwarp and expect the result file to have multidimensional variables, instead on must first create a vrt file using gdalwarp and then create the result (warped and multidimensional netcdf) using gdal_translate.

This workaround is necessary due to the way Create() works and also due to limitations in the netcdf driver, which needs the multidimensional metadata (which is not available after Create()) before defining the bands and also does not save metadata to file when calling SetMetadata?() and SetMetadataItem?(). The later could be solved using PAM (which has been disable except for stats for various reasons).

I'm afraid the core problem cannot be solved without changing the way Create() works - i.e. by allowing to copy metadata BEFORE band creation, perhaps as an optional argument (either a source dataset or a char for the dataset metatada and a char* for the band metadata).

Another solution would be to find another way to store multi-dimensional information, but I don't see anything other than gdal metadata.

comment:10 Changed 10 years ago by etourigny

Status: newassigned

comment:11 Changed 10 years ago by etourigny

fixed regressions in band selection (r24215), when nBands!=computed.

1) segfault because of deleted papszExtraDimNames 2) fix band names when nBands>1 and nBands!=computed

comment:12 Changed 3 years ago by Even Rouault

Milestone: closed_because_of_github_migration
Resolution: wontfix
Status: assignedclosed

This ticket has been automatically closed because Trac is no longer used for GDAL bug tracking, since the project has migrated to GitHub?. If you believe this ticket is still valid, you may file it to https://github.com/OSGeo/gdal/issues if it is not already reported there.

Note: See TracTickets for help on using tickets.