Enhancement in HDF4/HDF5 Drivers for GDAL

[yjiang7]

Most GIS software (both commercial and open source) use GDAL to access Earth Observation Datasets. But many EO data products cannot be correctly opened using GDAL, therefore, cannot be used for GIS analysis and visualization. The are different reasons, such as missing georeference, image rotation, and not being able to access multi-dimensional data.

Although we can identify the problem through exploration, it will take very long for a satellite mission to change their data production workflow. Therefore, our project aims to address the problems at the GDAL level. We modified GDAL source code to make GDAL compatible and adaptive with those problems. Since most EOS data products are stored in the format of HDF4/HDF5, the modification is in HDF4 and HDF5 data drivers.

Our development is based on GDAL 2.0.0 and the enhanced version has been tested in major GIS software (ArcGIS and QGIS) and been able to access some EOS products (e.g. MOPITT, TES, TerraAuqura) correctly. NASA data centers are adopting the enhancement. Esri, the largest GIS software vendor is also planning to take our modified GDAL library. When used by them, this project will benefits hundreds of thousand scientists who use GDAL to access EOS data.

(The current development cannot ensure solving problems in all EOS data yet. We are on the way to test more EOS data and enhance the HDF4/HDF5 data drivers.) YOU CAN TAKE THIS OUT.

Considering the potential broad impact, we would like to ask if GDAL community is interest in the enhanced drivers and integrate our development into the official HDF drivers.

[Even Rouault]

A few remarks :

• it would be easier to review if the patches were generated with 'svn diff' or 'diff -u'
• you should update the file headers with the appropriate copyright holders since the changes are not trivial and definitely copyrightable material.
• your changes seem to involve a XML sidecar file. Is this format somewhat standardized ? Is there documentation for it ?
• test products would be more than welcome. Public ones preferably. And also possibly small ones to be able to integrate them in the regression test suite.

[Kurt Schwehr]

In addition to what rouault already said, here are some quick comments based on hdf4imagedataset.2.patch that I think would make your could easier to follow in the context of GDAL:

• Use // for comments (not ///). Start comments with capital letters and use punctuation
• Do not put your name on each chunk of code
• Use 4 spaces for indenting - match the code that you are going into. Do not use tabs
• Drop the empty line that you added between the last method of the class and the class closing };
• Provide a description of each function, arguments, and return
• Please use C++ casts rather than C casts
• Use const when possible
• Follow the the spacing convention within lines. e.g. if( expr )
• Don't have commented out code (e.g. your printfs). Either use #ifdef DEBUG_VERBOSE or delete the code.
• Convert printfs to CPLDebug calls

If you have any questions on C++ casts, feel free to ask. I think I've got it figured out. I realized in the last couple days that I should have been using static_cast in many places where I used the more loose reinterpret_cast (e.g. with CPLMalloc and other things that return void pointers).

[yjiang7]

Replying to goatbar:

In addition to what rouault already said, here are some quick comments based on hdf4imagedataset.2.patch that I think would make your could easier to follow in the context of GDAL:

• Use // for comments (not ///). Start comments with capital letters and use punctuation
• Do not put your name on each chunk of code
• Use 4 spaces for indenting - match the code that you are going into. Do not use tabs
• Drop the empty line that you added between the last method of the class and the class closing };
• Provide a description of each function, arguments, and return
• Please use C++ casts rather than C casts
• Use const when possible
• Follow the the spacing convention within lines. e.g. if( expr )
• Don't have commented out code (e.g. your printfs). Either use #ifdef DEBUG_VERBOSE or delete the code.
• Convert printfs to CPLDebug calls

If you have any questions on C++ casts, feel free to ask. I think I've got it figured out. I realized in the last couple days that I should have been using static_cast in many places where I used the more loose reinterpret_cast (e.g. with CPLMalloc and other things that return void pointers).

Thank you very much for your suggestions, goatbar. I will update it asap.

[yjiang7]

Attached please find an xml example.

[yjiang7]

Attached please find an xml example.

[Even Rouault]

Unmilestoning

[Even Rouault]

This ticket has been automatically closed because Trac is no longer used for GDAL bug tracking, since the project has migrated to GitHub. If you believe this ticket is still valid, you may file it to ​https://github.com/OSGeo/gdal/issues if it is not already reported there.