Opened 8 years ago

Last modified 7 years ago

#4554 closed defect

HDF5-DIAG Error detected when processing netcdf file with multiple datasets — at Version 5

Reported by: hsumanto Owned by: warmerdam
Priority: normal Milestone:
Component: Utilities Version: unspecified
Severity: normal Keywords: netcdf
Cc: etourigny, jluis, Even Rouault, warmerdam

Description (last modified by jluis)

Etienne,

On Win7 64 with HDF5 1.8.8 and netCDF 4.1.3 I get this

c:\>gdalwarp -of netCDF multipledataset1.nc dataset1.nc
Warning 1: No UNIDATA NC_GLOBAL:Conventions attribute
Input file multipledataset1.nc has no raster bands.

c:\>gdalwarp -of netCDF multipledataset2.nc dataset1.nc
Input file multipledataset2.nc has no raster bands.

and with the -debug mode

c:\>gdalwarp --debug on -of netCDF multipledataset1.nc dataset1.nc
GDAL_netCDF:
=====
calling nc_open( multipledataset1.nc )

GDAL_netCDF: driver detected file type=3, libnetcdf detected type=4
GDAL_netCDF: seting file type to 4, was 3
GDAL_netCDF: dim_count = 2
Warning 1: No UNIDATA NC_GLOBAL:Conventions attribute
GDAL_netCDF: var_count = 4
GDAL: GDALOpen(multipledataset1.nc, this=00000000005B6660) succeeds as netCDF.
Input file multipledataset1.nc has no raster bands.
GDAL: force close of multipledataset1.nc in GDALDriverManager cleanup.
GDAL: GDALClose(multipledataset1.nc, this=00000000005B6660)

Change History (6)

Changed 8 years ago by hsumanto

Attachment: dataset.zip added

sample dataset used

comment:1 Changed 8 years ago by Even Rouault

Cc: etourigny added

comment:2 Changed 8 years ago by etourigny

GDAL treats both files as having subdatasets (with no bands), therefore gdalwarp cannot operate on them, you have to work with the subdatasets.

The error for the first file, as well as the debugging output (GDAL: force close of multipledataset2.nc in GDALDriverManager cleanup.) is cause for minor concern, I will see what is wrong. It seems the input file is not closed properly, and is unrelated to the lack of support for gdalwarp with multiple subdatasets.

One way or another, don't count on having the fundamental problem fixed soon though (see my email to the list today in answer to Hendy's post).

$ gdalwarp --debug on -of netCDF multipledataset1.nc dataset1.nc
GDAL_netCDF: 
=====
calling nc_open( multipledataset1.nc )

GDAL_netCDF: driver detected file type=3, libnetcdf detected type=4
GDAL_netCDF: seting file type to 4, was 3
GDAL_netCDF: dim_count = 2
Warning 1: No UNIDATA NC_GLOBAL:Conventions attribute
GDAL_netCDF: var_count = 4
GDAL: GDALOpen(multipledataset1.nc, this=0x17656a0) succeeds as netCDF.
Input file multipledataset1.nc has no raster bands.
GDAL: In GDALDestroy - unloading GDAL shared library.
GDAL: force close of multipledataset1.nc in GDALDriverManager cleanup.
HDF5-DIAG: Error detected in HDF5 (1.8.8) thread 0:
  #000: H5T.c line 1721 in H5Tclose(): not a datatype
    major: Invalid arguments to routine
    minor: Inappropriate type
GDAL: GDALClose(multipledataset1.nc, this=0x17656a0)

 $ gdalwarp --debug on -of netCDF multipledataset2.nc dataset2.nc2
GDAL_netCDF: 
=====
calling nc_open( multipledataset2.nc )

GDAL_netCDF: driver detected file type=1, libnetcdf detected type=1
GDAL_netCDF: dim_count = 2
GDAL_netCDF: var_count = 9
GDAL_netCDF: variable #7 [lat] was ignored
GDAL_netCDF: variable #8 [lon] was ignored
GDAL: GDALOpen(multipledataset2.nc, this=0x12666a0) succeeds as netCDF.
Input file multipledataset2.nc has no raster bands.
GDAL: In GDALDestroy - unloading GDAL shared library.
GDAL: force close of multipledataset2.nc in GDALDriverManager cleanup.
GDAL: GDALClose(multipledataset2.nc, this=0x12666a0)

comment:3 Changed 8 years ago by hsumanto

Thanks for having a look at the cause of the HDF5-Diag error, Etienne.

comment:4 Changed 8 years ago by etourigny

Cc: jluis added

With much help from Even and Frank, the source of the error message has been found.

It seems that the hdf5 library uses atextit() exit handlers (specifically the function H5_term_library()) which get called before gdal unloads. The call to nc_close (ehich calls H5close()) raises an error because the hdf5 library has been unloaded.

Disabling most of gdaldllmain.cpp seems to solve it temporarily, but a better solution should be found.

//#ifdef __GNUC__
#ifdef notdef

Not sure if this happens in Windows - adding Joaquim in cc... Can you see if you get the same error?

comment:5 in reply to:  4 Changed 8 years ago by jluis

Description: modified (diff)

Replying to etourigny:

With much help from Even and Frank, the source of the error message has been found.

It seems that the hdf5 library uses atextit() exit handlers (specifically the function H5_term_library()) which get called before gdal unloads. The call to nc_close (ehich calls H5close()) raises an error because the hdf5 library has been unloaded.

Disabling most of gdaldllmain.cpp seems to solve it temporarily, but a better solution should be found.

//#ifdef __GNUC__
#ifdef notdef

Not sure if this happens in Windows - adding Joaquim in cc... Can you see if you get the same error?

Note: See TracTickets for help on using tickets.