aigrid file format broken after upgrade?

[mark@…]

I'm developing on debian based systems.  Up until this past week, I've been working on debian stable, with gdal version 1.2.6 and everything worked great.  In the process of testing my build system for the upcoming etch release, which currently includes gdal version 1.3.2, I ran into major problems.  I use the gdal to extract elevation data from the National Elevation Dataset (NED), which according to gdal 1.2.6 uses the AIG/Arc/Info Binary Grid Driver; aigrid I believe.  Unfortunately, when I try to use gdal version 1.3.2, even simple commands like gdalinfo fail with a message:

ERROR 4: `/var/www/data/maps/ned02/DEM7442' not recognised as a supported file format.

GDALOpen failed - 4
`/var/www/data/maps/ned02/DEM7442' not recognised as a supported file format.

The exact same command, on the same data, works just fine.  I've looked at both debian packages and with the exception of turning on mysql support in the 1.3.2 version of the package, there are little differences between the two versions.  Regardless, thining this may have been a debian packaging problem, I downloaded the 1.3.2 sourccode and build it from scratch.  This too experienced the same failure.  I also downloaded the version from the anonymous CVS server, built it and, likewise, it failed in the same way.

Am I missing some fundamental difference between 1.2.6 and 1.3.2?  I couldn't find anything on the devel mailing list nor the bugzilla site.  There was one potentially related ticket I found #1198, which appears to describe my underlying data; I have W001001.ADF and W001001X.ADF files.  Although I don't get read errors as gdal never even attempts a read.

I'd be happy to provide you with some example data, it's about 56M in size.

Any help would be most appreciated.

Thanks!

Mark

[mark@…]

I should have said:

"The exact same command, on the same data, works just fine" under gdal 1.2.6.

Mark

[warmerdam]

Mark,

Yes, I will need the data.  I am not aware of any intentional change in
the aigrid driver to explain this, but there have been a number of bug fixes,
some of which could have broken something else.

[mark@…]

I uploaded an example of the NED data to http://dtrick.org/tmp/DEM7442.tgz.  Let me know if there is anything else I can help with.

Thanks!

Mark

[warmerdam]

the problem is this relatively new logic in AIGDataset::Open()
which does not leave room for files in upper case.  

/* -------------------------------------------------------------------- */
/*      Verify that a few of the "standard" files are available.        */
/* -------------------------------------------------------------------- */
    VSIStatBufL sStatBuf;
    CPLString osTestName;
    
    osTestName.Printf( "%s/hdr.adf", osCoverName.c_str() );
    if( VSIStatL( osTestName, &sStatBuf ) != 0 )
        return NULL;

    osTestName.Printf( "%s/w001001x.adf", osCoverName.c_str() );
    if( VSIStatL( osTestName, &sStatBuf ) != 0 )
        return NULL;

    osTestName.Printf( "%s/w001001.adf", osCoverName.c_str() );
    if( VSIStatL( osTestName, &sStatBuf ) != 0 )
        return NULL;
    
A work around is to rename your files into lower case.  But I will 
correct the code before the next release.

For the record, the files look like this:

./DEM7442:
total 54220
-rw-r--r--  1 warmerda warmerda       32 2000-03-22 10:27 DBLBND.ADF
-rw-r--r--  1 warmerda warmerda      308 2000-03-22 10:27 HDR.ADF
-rw-r--r--  1 warmerda warmerda      286 2000-03-22 10:26 LOG
-rw-r--r--  1 warmerda warmerda      648 2000-03-22 10:27 PRJ.ADF
-rw-r--r--  1 warmerda warmerda       32 2000-03-22 10:26 STA.ADF
-rw-r--r--  1 warmerda warmerda 55323100 2000-03-22 10:27 W001001.ADF
-rw-r--r--  1 warmerda warmerda   108100 2000-03-22 10:27 W001001X.ADF

[warmerdam]

Swapnil,

Can you take this problem?  You can reproduce the problem by renaming
a coverage directory and all the interior files into upper case. 

The AIGOpen() function apparently already handles the upper case files on
Unix, it is just the AIGDataset::Open() that does not.  Make sure that
.clr files are also handled properly.

[mark@…]

Frank,

Thanks for looking into this.  I can confirm that renaming files with lowercase names allows me to work just fine with the NED data using the 1.3.2 version of gdal.

Mark

[dreamil@…]

I am working on this bug right now. I have a doubt, do we want to be able to read files with names in all uppercase only (e.g. HDR.ADF) or should the AIGDataset::Open() be able to read files with files with any combination of lowercase and uppercase (e.g. Hdr.Adf)? 
Checking for all lowercase and all uppercase will be easy to implement.But checking for mixed case names doesnt seem trivial.

[dreamil@…]

I have fixed this. The AIGDataset::Open() now also looks for all uppercase version of coverage if all lowercase coverage is not found. I have made necessary changes in frmts/aigrid/aigdataset.cpp and committed it to CVS. This also takes care of .clr file which may be in coverage dir itself or in the parent directory and it may be all lowercase or all uppercase.