#6856 closed defect (invalid)
fix support for non-classic netCDF4
Reported by: | jacobst | Owned by: | warmerdam |
---|---|---|---|
Priority: | normal | Milestone: | |
Component: | default | Version: | 2.1.1 |
Severity: | normal | Keywords: | |
Cc: |
Description
While netCDF driver has seen significant improvements, its documented to support non-classic netCDF4, but faces problems with unsigned integer variables in the netCDF (one of the differences between classic and non-classic models of netCDF4).
Tried with gdal v2.1.1, shipped with QGIS Las Palmas: When gdal_translate is performed from input netCDF4 to output VRT, unsigned data types are reported as 'unsupported' and changed to Float32, which can only mess up the data read.
As work-around, the compagnon HDF5 driver can be used to actually access the unsigned integer data.
Proof of concept:
- take netCDF4 with unsigned byte or unsigned short data layer
- gdal_translate to VRT format, which issues a warning on data type
- edit the VRT, then change 'Float32' back to the correct data type
- and change the VRT's source file reference from NETCDF:some_file:some_layer into HDF5:some_file://some_layer
The data can then be read through the VRT file.
Attachments (2)
Change History (14)
comment:1 by , 7 years ago
comment:2 by , 7 years ago
Sample files I have are much too big to be attached to the ticket (1MB size limit). If you email me, than I can send it to you via eg dropbox link.
comment:3 by , 7 years ago
I've just attached several small text files:
- outputs of h5dump -p -A, ncdump -c (to illustrate the structure of the file)
- outputs of gdalinfo, using both HDF5 and NetCDF drivers, at both level of the entire file and at level of the NDVI subdataset (or band) in the file
- VRT files produced with HDF5 and NETCDF drivers => note the difference in data type for the band that is described (Byte vs Float32)
Warning text that is displayed: Warning 1: Unsupported netCDF datatype (7), treat as Float32.
The priority is to fix the data read. But concerning the warning text itself, I can further note that:
- The (unsupported?) data type is Byte, which is presumably value 2, not 7, according to the enum definition on http://www.gdal.org/gdal_8h.html#a22e22ce0a55036a96f652765793fb7a4).
- Such change of data type would disrupt any subsequent data read via the VRT, so I wonder why this is only a warning, not an error.
comment:6 by , 7 years ago
Ah, can you send instead to even.rouault at gmail.com because of Dropbox identification...
comment:8 by , 7 years ago
It is working fine for me on Ubuntu 16.04 with libnetcdf 4.4.0-2. Are you sure your GDAL build is made against libnetcdf V4 , and not an older version ? This really looks like if that was the case.
#ifdef NETCDF_HAS_NC4 /* NC_UBYTE (unsigned byte) is only available for NC4 */ else if( nc_datatype == NC_UBYTE ) eDataType = GDT_Byte; else if( nc_datatype == NC_USHORT ) eDataType = GDT_UInt16; else if( nc_datatype == NC_UINT ) eDataType = GDT_UInt32; #endif else { if( nBand == 1 ) CPLError( CE_Warning, CPLE_AppDefined, "Unsupported netCDF datatype (%d), treat as Float32.", (int) nc_datatype ); eDataType = GDT_Float32; nc_datatype = NC_FLOAT; }
comment:9 by , 7 years ago
How would I found out this version for a pre-compiled NETCDF driver that was shipped with QGIS Las Palmas?
ncdump reports libnetcdf 4.3.0, but it is from a different install, so I'm not sure QGIS' copy of GDAL would use the same lib under its hood.
comment:10 by , 7 years ago
Well, this depends on the operating system: Linux, Windows (OSGeo4W ?)... ?
If you can access to a shell with the GDAL version used by your QGIS, you can type
$ gdalinfo --format netcdf
If you see NC4 mentionned in the output, like in
<CreationOptionList> <Option name="FORMAT" type="string-select" default="NC"> <Value>NC</Value> <Value>NC2</Value> <Value>NC4</Value> <Value>NC4C</Value> </Option>
then this should work like in my setup. But from what you reported and the source code, I'm pretty sure the issue is that the GDAL build you use doesn't enable NC4.
comment:11 by , 7 years ago
Resolution: | → invalid |
---|---|
Status: | new → closed |
OK the output of gdalinfo confirms that your GDAL build is not netCDF v4 enabled. Something to report to the binary distribution that provides GDAL to you
Closing as not a GDAL bug
comment:12 by , 7 years ago
I've attached the output of the gdalinfo --format netcdf command. You seem to be on the right track. It lists just NC and NC2, not NC4/NC4C formats.
OS is Windows 7, 64bit with Service Pack 1 GDAL was included in stand-alone QGIS installation file QGIS-OSGeo4W-2.18.0-1-Setup-x86_64.exe downloaded from qgis.org.
Can you attach a file or point to a file that demonstrates the issue ? From a quick skim and testing of the code, uint32 should be handled.