Opened 12 years ago
Closed 12 years ago
#4437 closed defect (invalid)
lack of closing read-only dataset dataset or calling GDALDestroyDriverManager() on NC4C (netCDF-4 classic model) file producing HDF5-DIAG error messages
Reported by: | hsumanto | Owned by: | warmerdam |
---|---|---|---|
Priority: | normal | Milestone: | |
Component: | default | Version: | unspecified |
Severity: | normal | Keywords: | netcdf, java |
Cc: | etourigny |
Description
Just trying to gdal.Open NC4C (netCDF-4 classic model) file and keep getting the below:
HDF5-DIAG: Error detected in HDF5 (1.8.8) thread 0: #000: H5T.c line 1721 in H5Tclose(): not a datatype major: Invalid arguments to routine minor: Inappropriate type
I attached the sample program getinfo and the sample dataset.
gdal.Open with ERROR
ncdump -k tile-nc4c.nc netCDF-4 classic model getinfo tile-nc4c.nc Driver: netCDF/Network Common Data Format Size is 2000x2000x1 HDF5-DIAG: Error detected in HDF5 (1.8.8) thread 0: #000: H5T.c line 1721 in H5Tclose(): not a datatype major: Invalid arguments to routine minor: Inappropriate type
gdal.Open OK
ncdump -k tile-nc.nc classic getinfo tile-nc.nc Driver: netCDF/Network Common Data Format Size is 333x333x1
Attachments (3)
Change History (12)
by , 12 years ago
Attachment: | GetInfo.zip added |
---|
comment:1 by , 12 years ago
Keywords: | netcdf added |
---|
comment:3 by , 12 years ago
Using gdal.Open in java (See GdalTest.zip for example) to open NC4C file also produces the same error.
comment:4 by , 12 years ago
Cc: | added |
---|
Intuitively, I don't believe it is related to the Java bindings (I guess you get the same issue with gdalinfo ?). Not sure if it is in the netCDF driver or HDF5 driver either. CC'ing Etienne in case he's interested in that.
comment:5 by , 12 years ago
Keywords: | java added |
---|
gdalinfo can open both files fine, so the problem is probably with the java bindings.
Perhaps the hdf5 driver is being used instead of the netcdf driver? Try debug output in your test program and see if it really is the netcdf driver that opens the file (there is a lot of debug output from the netcdf driver).
Is there a java port of gdalinfo to test this file with?
Sorry I don't have any idea how to work with the java bindings nor the time to learn.
comment:6 by , 12 years ago
gdalinfo can open both files fine, so the problem is probably with the java bindings.
Like Even, I don't believe it is related to the java bindings too as the getinfo program (a simple program written in C++ which just uses gdal.Open) can produce the same message below:
HDF5-DIAG: Error detected in HDF5 (1.8.8) thread 0: #000: H5T.c line 1721 in H5Tclose(): not a datatype major: Invalid arguments to routine minor: Inappropriate type
Perhaps the hdf5 driver is being used instead of the netcdf driver? Try debug output in your test program and see if it really is the netcdf driver that opens the file (there is a lot of debug output from the netcdf driver).
The statement below:
printf( "Driver: %s/%s\n", GDALGetDriverShortName( hDriver ), GDALGetDriverLongName( hDriver ) );
printed out Driver: netCDF/Network Common Data Format, so it must be using netcdf driver.
The is the full content of the getinfo program:
#include <iostream> #include "gdal_priv.h" #include "exceptions.h" namespace VPAC { /** * Get info from raster. */ class GetInfoTest { protected: GDALDataset *input; public: std::string inputFileName; std::string outputFileName; public: GetInfoTest() : input(NULL) { } ~GetInfoTest() throw() { if (input != NULL) { //GDALClose(input); //input = NULL; } } void run() { openDatasets(); } protected: void openDatasets() throw (MyException) { if (input != NULL) { throw MyException("Datasets are already open!"); } input = (GDALDataset *) GDALOpen(inputFileName.c_str(), GA_ReadOnly); if (input == NULL) { throw RequestException("Could not open source file."); } GDALDriverH hDriver; hDriver = GDALGetDatasetDriver( input ); printf( "Driver: %s/%s\n", GDALGetDriverShortName( hDriver ), GDALGetDriverLongName( hDriver ) ); printf( "Size is %dx%dx%d\n", GDALGetRasterXSize( input ), GDALGetRasterYSize( input ), GDALGetRasterCount( input ) ); } }; // class GetInfoTest } // namespace VPAC using namespace VPAC; int main(int argc, char *argv[]) { try { GetInfoTest getinfo; GDALAllRegister(); if (argc != 2) { throw RequestException("Invalid arguments."); } getinfo.inputFileName = argv[1]; getinfo.run(); } catch (RequestException e) { std::cerr << "Error: " << e.what() << std::endl; std::cerr << "Usage: getinfo <input file>" << std::endl; } catch (MyException e) { std::cerr << "Error: " << e.what() << std::endl; } return 0; }
comment:7 by , 12 years ago
Please test against gdalinfo (java version), as gdalinfo (c version) does not report the error.
The errors you report are with programs written by your organisation, not by the official gdalinfo in c or java.
The lack of GDALClose( ) in gdalinfo.cpp causes the error, if you uncomment the line GDALClose(input);
the error does not appear.
I suspect it is the lack hDataset.delete() in GDALTest.java that is causing the error.
comment:8 by , 12 years ago
Yeah I noticed that during testing as well. Having GDALClose( ) in gdalinfo.cpp and having hDataset.delete() in GDALTest.java fix the error messages which appear when opening NC4C (netCDF-4 classic model).
The only thing which still left me puzzled at that time:
When opening NC file (tile-nc.nc in sample_dataset.zip), not having GDALClose( ) in gdalinfo.cpp and not having hDataset.delete() in GDALTest.java -> No error messages are encountered.
When opening NC4C file (tile-nc4c.nc in sample_dataset.zip), not having GDALClose( ) in gdalinfo.cpp and not having hDataset.delete() in GDALTest.java -> HDF5-DIAG error messages are shown.
In both cases (gdalinfo.cpp and GDALTest.java), gdal is opening the dataset in readonly mode (not update mode), thus I was assuming that we won't have to explicitly close a dataset opened in readonly mode. Therefore, I was just wondering whether I should report this inconsistency as a ticket.
Thanks for the quick reply Etienne, really appreciate it :)
public static Dataset Open(String name, int eAccess) Open a raster file as a Dataset object. This function will try to open the passed file, or virtual dataset name by invoking the Open method of each registered Driver in turn. The first successful open will result in a returned dataset. If all drivers fail then null is returned. It is required that you explicitly close a dataset opened in update mode with the Dataset.delete() method. Otherwise the data might not be flushed to the disk. Don't rely only on Java garbage collection. Parameters: name - the name of the file to access. In the case of exotic drivers this may not refer to a physical file, but instead contain information for the driver on how to access a dataset. eAccess - the desired access, either gdalconst.GA_Update or gdalconst.GA_ReadOnly. Many drivers support only read only access. Returns: A Dataset object or null on failure.
comment:9 by , 12 years ago
Resolution: | → invalid |
---|---|
Status: | new → closed |
Summary: | gdal.Open on NC4C (netCDF-4 classic model) file producing HDF5-DIAG error messages → lack of closing read-only dataset dataset or calling GDALDestroyDriverManager() on NC4C (netCDF-4 classic model) file producing HDF5-DIAG error messages |
Further investigation reveals that the lack of calling GDALDestroyDriverManager() and GDALClose( hDataset ) cause the hdf5 errors. By using one or the other the error does not appear.
The same is probably true for your java program, which is lacking gdal.GDALDestroyDriverManager() as well hDataset.delete();
I don't see how to fix the netcdf driver for this case as the internals of netcdf-4/hdf5 are rather complex.
Closing this bug as invalid, as the recommendation is to close the dataset after usage, or using GDALDestroyDriverManager() - which in 1.9 properly closes all open datasets.
getinfo program