GDALDestroyDriverManager() - dataset cleanup

[Even Rouault]

Quoting gdal-dev

Folks,

I have modified GDALDriverManager::~GDALDriverManager() so that when
it is destroyed it will first close all still-open GDAL datasets.  A debug
message will be issued for each dataset force-closed.

This is part of an effort to ensure a clean shutdown and to make it easier
to detect memory leaks in some situations.

This is not a completely safe operation.  In particular, if there are VRT
files referencing other GDALDataset's the referenced dataset may be cleaned
up before the referencing dataset which would be bad.

I need this primarily for cleanup in MapServer where GDAL datasets are now
normally left open (CLOSE_CONNECTION=DEFER).

If this automatic dataset description causes problems we can disable or
remove it before the GDAL 1.9 release.  But it seems relatively harmless
to incorporate in trunk for the time being.

   http://trac.osgeo.org/gdal/changeset/21662

Best regards,

This is indeed unsafe (a trivial code that opens a VRT and doesn't close it shows it under Valgrind), not only in VRT, but also for NITF (with JPEG compression), TIL, RS2, ERS and BIGGIF drivers (I established the list by finding the drivers that call GDALClose())

I have implemented a safer way by adding a new virtual method, GDALDataset::CloseDependantDatasets(), which is called by GDALDriverManager::~GDALDriverManager() on the list of still opened datasets while there are datasets that still declare dependant datasets. Once all remaining datasets are "stand-alone" they can safely be destroyed.

[Even Rouault]

r21664 /trunk/gdal/ (13 files in 7 dirs): Add a GDALDataset::CloseDependantDatasets() that can be used by GDALDriverManager::~GDALDriverManager() to safely close remaining opened datasets; Use it in VRT, NITF, TIL, RS2, ERS and GIF drivers (#3954)

[Even Rouault]

r21665 /trunk/gdal/frmts/ (dimap/dimapdataset.cpp pds/pdsdataset.cpp): PDS and DIMAP drivers also need to implement CloseDependentDatasets(); by the way fix the DIMAP driver so that it doesn't directly attach the raster bands of its dependant images to itself but use a ProxyRasterBand itself (#3954)

[Even Rouault]

r21668 /trunk/gdal/frmts/vrt/vrtdataset.cpp: VRT: call FlushCache() in CloseDependentDatasets() to avoid the sources to be removed from the serizalized VRT (#3955)

r21669 /trunk/gdal/gcore/ (gdaldrivermanager.cpp gdalproxypool.cpp): Fix crash that can occur sometimes when ~GDALDriverManager() is called on a VRT referencing a VRT (#3954)

[Even Rouault]

There were also issues with internal and external overviews/mask bands...

r21674 /trunk/gdal/ (13 files in 10 dirs): Implement CloseDependentDatasets() for GDALDefaultOverviews and GTiffDataset; In VRT, NITF, TIL, RS2, ERS, GIF, PDS and DIMAP, call superclass CloseDependentDatasets() implementation (#3954)

r21677 /trunk/autotest/cpp/ (Makefile testclosedondestroydm.c): Test most corner cases of the CloseDependentDatasets() mechanism (#3954)

Hopefully the issue is now completely addressed...

[Even Rouault]

r21688 /trunk/ (2 files in 2 dirs): GDALDestroyDriverManager(): force closing of datasets with reference count > 1 (#3954)

[Even Rouault]

r21721 /trunk/ (2 files in 2 dirs): Implement CloseDependentDatasets() on MrSID (#3954)

[Even Rouault]

r21723 /trunk/ (3 files in 2 dirs): Implement CloseDependentDatasets() on Rasterlite (#3954)