Opened 7 years ago

Closed 7 years ago

#5467 closed defect (fixed)

ENVI driver .hdr corruption

Reported by: wilsonwaters Owned by: warmerdam
Priority: normal Milestone: 1.11.1
Component: GDAL_Raster Version: 1.11.0
Severity: major Keywords: envi, hdr
Cc: antonio

Description (last modified by wilsonwaters)

We found our ENVI .hdr files were being corrupted whenever opened in write mode with GDAL. No changes to the data or metadata are required to corrupt the file, simply opening the file in write mode was enough. I've attached example before and after .hdr files. These files have map info projection set to "Arbitrary" (we're not storing map data. It's long skinny images of underground drill core).

This is the process causing the corruption in ENVIDataset:

  1. open envi file in read/write mode uses OPEN_EXISTING flag on .hdr
  2. ENVIDataset::Open() doesn't understand "Arbitrary" projection so generates a "LOCAL_CS" projection string
  3. close dataset causing ENVIDataset::FlushCache?() to be called
  4. FlushCache?() re-writes hdr file as the bHeaderDirty was set in the Open() function. map info field is omitted for some reason?
  5. FlushCache?() overwrites the old hdr file without clearing first causing trailing data in the file

As far as I can tell there are four bugs in the ENVIDataset related to this:

  1. ENVIDataset::ProcessMapinfo?() Doesn't understand the "Arbitrary" projection and defaults to LOCAL_CS - which is probably ok, but;
  2. ENVIDataset::WriteProjectionInfo? fails to write map info field when projection is LOCAL_CS
  3. ENVIDataset::Open() causes bHeaderDirty to be set (should only happen when something external updates the image?)
  4. ENVIDataset::FlushCache?() should truncate existing file before overwriting

We're using GDAL 1.11 on windows, though I think the behaviour would be the same on any OS.

The attached patch fixes the issues for me, though my experience of GDAL is limited so I would suggest it would need to be reviewed before being used.

Attachments (3)

envidataset.cpp.diff (2.0 KB) - added by wilsonwaters 7 years ago.
patch for envidataset.cpp
good.hdr (284 bytes) - added by wilsonwaters 7 years ago.
example of an original, un-corrupted hdr file
bad.hdr (287 bytes) - added by wilsonwaters 7 years ago.
example of a bad, corrupted hdr file

Download all attachments as: .zip

Change History (8)

Changed 7 years ago by wilsonwaters

Attachment: envidataset.cpp.diff added

patch for envidataset.cpp

Changed 7 years ago by wilsonwaters

Attachment: good.hdr added

example of an original, un-corrupted hdr file

Changed 7 years ago by wilsonwaters

Attachment: bad.hdr added

example of a bad, corrupted hdr file

comment:1 Changed 7 years ago by wilsonwaters

Component: JavaBindingsGDAL_Raster
Description: modified (diff)
Owner: changed from Even Rouault to warmerdam

comment:2 Changed 7 years ago by antonio

Cc: antonio added

comment:3 Changed 7 years ago by Even Rouault

I've applied everything except the "
(strlen(pszProjection) >= 8 && strncmp(pszProjection, "LOCAL_CS", 8) != 0 )" part. I think the last part of the test is wrong. It should be == 0. Otherwise any PROJCS or GEOGCS SRS would be considered as an underfined one.

trunk r27353 "ENVI: avoid generating potentially corrupted .hdr files when opening in update mode; Write 'Arbitrary' instead of 'Unknown' as the projection name for an undefined SRS (#5467, patch by wilsonwaters)"

comment:4 Changed 7 years ago by wilsonwaters

Hi Even, Yes you are correct. It should be == 0.

Since fixing the "write on open" bug I haven't tested this with our data as we always manually add our own projection info before writing (rather than LOCAL_CS which the Open function adds).

Thanks for adding the patch and fixing my mistake!

comment:5 Changed 7 years ago by Even Rouault

Milestone: 1.11.1
Resolution: fixed
Status: newclosed

trunk r27365 "ENVI: when writing, consider that LOCAL_CS SRS is like ungeoreferenced (#5467)"

branches/1.11 r27366 "ENVI: avoid generating potentially corrupted .hdr files when opening in update mode; Write 'Arbitrary' instead of 'Unknown' as the projection name for an undefined SRS; when writing, consider that LOCAL_CS SRS is like ungeoreferenced (#5467, patch by wilsonwaters)"

Note: See TracTickets for help on using tickets.