Segfault in Python with large datasets

[Sam Gillingham]

Because of the order of operations used when calculating buffer size for read, the result is likely to overflow. This can happen well before the 2GB size limit (at 256 MB!). Proposed patch to the swig .i files addresses this and also adds a test for malloc failing.

[Sam Gillingham]

Contains patches for swig bindings

[hobu]

I think this is a duplicate of #2140. Please confirm.

[Even Rouault]

Yes, this is a duplicate of #2140. So I'm closing this bug. However, please note that it will only be fixed into 1.6.0, unless someone really wans to backport #2140.

[Mateusz Łoskot]

Even,

IMO, r14016 submission is a classic bug fix and I'd backport it to branches/1.5. This way, we will get Python bindings fixed in line 1.5 and 1.6. It makes sense to me because since 1.5 we have next-generation python as default. I vote to target 1.5.3.

Frank, what you think?

I can backport it.

[Even Rouault]

Adding Frank in CC so he can read Mateusz's comment.

Mateusz, the backport will not be trivially direct because VSIMalloc3(x, y, z) is new to GDAL 1.6. So you'd have to use VSIMalloc(x * y * z) and hope they don't overflow.

[Sam Gillingham]

Everyone,

Yes I see it is fixed for 1.6.0 - I should have checked the trunk first. Sorry about that.