Opened 15 years ago
Closed 15 years ago
Last modified 15 years ago
#2369 closed defect (fixed)
short integer overflow with AAGRID when input grid contains some 32bit integers
|Reported by:||dylan||Owned by:||Even Rouault|
There appears to be a problem with GDAL utilities not recognizing the possibility that an AAGRID file may contain a mixture of 16 and 32 bit integers. This is not surprising, as the AAGRID file format does not contain any hints about its contents. A ticket was started on the GRASS tracker, but the issue appears to be a purely GDAL bug.
One idea for a fix includes an environmental flag which would cause the AAGRID driver to scan the input file in order to properly determine an appropriate data type. Another approach would be to manually set the data type for any input file from the command line.
Example using the attached file:
gdal_translate big_int_arc_file.asc test.tiff
the resulting file is produced:
gdalinfo -stats test.tiff Driver: GTiff/GeoTIFF Files: test.tiff Size is 7, 7 Coordinate System is `' Origin = (592000.000000000000000,4926000.000000000000000) Pixel Size = (1000.000000000000000,-1000.000000000000000) Image Structure Metadata: INTERLEAVE=BAND Corner Coordinates: Upper Left ( 592000.000, 4926000.000) Lower Left ( 592000.000, 4919000.000) Upper Right ( 599000.000, 4926000.000) Lower Right ( 599000.000, 4919000.000) Center ( 595500.000, 4922500.000) Band 1 Block=7x7 Type=Int16, ColorInterp=Gray Minimum=-31293.000, Maximum=32420.000, Mean=216.837, StdDev=18087.589 NoData Value=-9999 Metadata: STATISTICS_MINIMUM=-31293 STATISTICS_MAXIMUM=32420 STATISTICS_MEAN=216.83673469388 STATISTICS_STDDEV=18087.589140406
Change History (10)
by , 15 years ago
comment:1 by , 15 years ago
|Component:||default → GDAL_Raster|
comment:2 by , 15 years ago
is there a particular reason for the AAIGRID driver to default to Int16 type and not Int32 ?
Another question, not directly related to this bug report, but in the ::CreateCopy, I see that we do the read RasterIO on the source as GDT_CFloat64. But it seems that we only use the real part, so that looks very weird.
comment:3 by , 15 years ago
The CFloat64 stuff is pretty scary, and apparently only works because we provide a "step size" that is sizeof(double) so all the imaginary values are getting written over. Please correct this, or let me know you don't plan to and I will do so. This fix ought to go into 1.5 and trunk.
The other issue is data type. The issue I have with defaulting to Int32 is that it produces output files that can't be read with most software after a translation and most grids are 8bit or 16bit values. But since there is no good way of knowing in advance what the range is (and prescanning is not an appealing option) I am ok with changing to GDT_Int32 as the default. This behavior change should only be made in trunk though.
The other possibility is that we just always treat the data as GDT_Float32, which gets rid of the other mess - trying to figure out if a file is floating point or integer. If you would like to change this, it should also only be done in trunk.
comment:4 by , 15 years ago
Thanks for looking into this Frank and others. Forcing the read as 32bit int or 32bit float seems like a workable solution, but will this lead to inflexibility later on? In other words, for funky formats like AAGRID it would be nice to have a flag for defining the data type on the fly. Defaulting to Float32 might work well in the general case, but if the user knows that the AAGRID is only an Int16, it would be helpful to only allocate the space that is really needed.
Just an idea.
comment:5 by , 15 years ago
Generally speaking I don't like users having to provide such information in order to open a dataset, though it could be accomplished via a configuration option.
Note that with gdal_translate you can convert pixel types on the fly, so if you know the data range is suitable for Int16, you could do "gdal_translate -ot Int16 in.grd out.tif" for instance.
comment:6 by , 15 years ago
|Status:||new → assigned|
comment:7 by , 15 years ago
comment:8 by , 15 years ago
|Status:||assigned → closed|
comment:9 by , 15 years ago
The applied fix works well for the test cases I was using when I had originally created the ticket. Thanks rouault!
an AAGRID file with a mix of 16 and 32 bit integers