Opened 14 years ago

Closed 14 years ago

#3173 closed defect (fixed)

ReadRaster() crashes when GDAL is built with MinGW

Reported by: dhelfman Owned by: Even Rouault
Priority: normal Milestone: 1.6.3
Component: PythonBindings Version: 1.6.2
Severity: major Keywords: python ReadRaster BandReadAsArray crash hang
Cc:

Description

When GDAL and its Python bindings are built with MinGW (gcc 3.4.5) on Windows XP, and I use the official Python 2.5 built with MSVC, then calling band.ReadRaster() or gdal_array.BandReadAsArray() from Python crashes some non-zero percentage of the time it's run.

Here is a sample test program that demonstrates the crash:

import sys
from osgeo import gdal

dataset = gdal.Open( sys.argv[ 1 ] )
band = dataset.GetRasterBand( 1 )

for i in range( 0, 4 ):
    print "reading", i
    band.ReadRaster( 0, 0, 256, 256, 256, 256, gdal.GDT_Float32 )

print "done"

When running this test program (in MSYS), you should provide any GDAL-readable raster filename as a command-line argument (at least 256x256). I've tried this both with PNG files and BSB KAP files, with the same results: If I run the program ten times, then at least one of those times it will crash and exit during the third iteration of the loop. Nothing is output on stdout or stderr when this happens.

If I run the test program in GDB, here's what I get on the first iteration:

warning: HEAP[python.exe]:
warning: Invalid Address specified to RtlFreeHeap( 00960000, 00D2E008 )

Program received signal SIGTRAP, Trace/breakpoint trap.
0x7c90120f in _libmsvcrt_a_iname ()

Based on this, I suspect that the crash is due to the fact that MinGW and MSVC are using different memory allocators, yet ReadRaster() is passing a chunk of allocated memory across that boundary in the form of the returned raster data. When Python takes "ownership" of this memory and subsequently tries to deallocate it, a crash can occur.

This problem essentially makes GDAL compiled with MinGW unusable with Python. One potential solution is briefly described in #3028.

Note that the crash does not occur when GDAL is built with MSVC.

Change History (6)

comment:1 by dhelfman, 14 years ago

#2658 is also related and proposes a specific solution.

comment:2 by Even Rouault, 14 years ago

I can't reproduce any crash with GDAL trunk compiled under MinGW GCC 4.4, and Python 2.6 downloaded from the python website. As far as memory allocation are concerned, for band.ReadRaster(), a buffer is allocated in the python bindings by VSIMalloc3() (that calls malloc() inside GDAL library) and freed in the binding code by free(), but that shouldn't be a problem if both GDAL library and binding objects are built with the same compiler. I tried to compile the python bindings with MSVC but that failed at the linking stage when trying to link against GDAL compiled with GCC.

Could you try to hand edit swig/python/extensions/gdal_wrap.cpp and replace

*buf = (char*) VSIMalloc3( buf_xsize, buf_ysize, (GDALGetDataTypeSize( buf_type ) / 8) * band_list );

with

*buf = (char*) malloc( buf_xsize * buf_ysize * (GDALGetDataTypeSize( buf_type ) / 8) * band_list );

and recompile and report if that improves things ? If so, for some reason, we should try to make allocation/deallocation more consistant

comment:3 by dhelfman, 14 years ago

Good call! That fixed it. I replaced both calls to VSIMalloc3() in gdal_wrap.cpp with malloc(), and the crash goes away. By the way, this was with Python 2.5.

comment:4 by dhelfman, 14 years ago

Some more info about my test cases:

  • Official Python 2.5 (MSVC) + GDAL 1.6.2 (MinGW) + GDAL Python bindings (MinGW): crash
  • Official Python 2.6 (MSVC) + GDAL 1.6.2 (MSVC) + GDAL Python bindings (MSVC): no crash
  • Official Python 2.5 (MSVC) + GDAL 1.6.2 (MinGW) + GDAL Python bindings (MinGW) + your VSIMalloc3 patch: no crash

So it's possible that the MinGW thing has nothing to do with it, and this is simply an issue with Python 2.5. I don't have MSVC myself, nor GDAL 1.6.2 Python 2.5 bindings built with MSVC. The MSVC GDAL binaries I used are from here: http://vbkto.dyndns.org/sdk/

I should also mention that I'm on 32-bit Windows XP.

comment:5 by Even Rouault, 14 years ago

Owner: changed from hobu to Even Rouault
Status: newassigned

comment:6 by Even Rouault, 14 years ago

Resolution: fixed
Status: assignedclosed

Should be fixed in trunk (r17765) and in branches/1.6 (r17766, r17767) now

Note: See TracTickets for help on using tickets.