With floating-point exceptions enabled, gdaladdo crashes (SIGFPE) on rasterband without NODATA (AVERAGE algorithm)

[match]

When I try to generate the overviews using the "AVERAGE" algorithm using raster(TIFF or JPEG) with rasterband of type GDT_Byte and without NODATA value defined, floating point exception happens inside GDALDownsampleChunk32R_AverageT<unsigned int, float>

The relevant source code line:

T      tNoDataValue = (T)fNoDataValue;  <=== crashes here
if (!bHasNoData)
    tNoDataValue = 0; 

fNoDataValue has the value of -1e10 obtained from GDALRasterBand::GetNoDataValue, bHasNoData is zero and exception happens when converting -1e10 to unsigned int

Maybe peforming check of bHasNoData before converting would correct the problem.

[Even Rouault]

It is strange that you are the first to report this problem, because it is a pretty common use case. What is your platform, OS, compiler, compiler version ?

Could you actually test that replacing the above snippet by the following fixes the issue ? :

T      tNoDataValue;
if (!bHasNoData)
   tNoDataValue = 0;
else
   tNoDataValue = (T)fNoDataValue;

[match]

Yes, the snippet fixes the issue.

It is Intel Core 2 Quad, Windows XP SP3 32 bit GDAL itself is build with Visual Studio 2008 and gdaladdo with Embarcadero C++Builder 2010. It seems that C++Builder has floating-point exceptions enabled by default, which causes the described issue, when negative float value is converted to unsigned int.

[Even Rouault]

I've managed to reproduce under Linux/GCC when enabling floating-point exceptions :

#define _GNU_SOURCE 1
#include <fenv.h>

feenableexcept(FE_DIVBYZERO | FE_INVALID | FE_OVERFLOW | FE_UNDERFLOW );

When running GDAL autotest suite, there are quite a few places where float to int overflow occurs (I've seen some in GEOS code too), and even more when it could occur but isn't triggered by the autotest suite... So my advice is to *disable* floating-point exceptions for production use.

This could be interesting to catch and fix those potential wrong float to int conversions, but I'm not sure if this is a battle worth pursuing... This complicates the code a lot to deal with situations where such overflows are detected.

[Even Rouault]

I'm unlikely pursuing that battle myself, so closing unless someone contributes a reasonable patch