Opened 14 years ago
Closed 9 years ago
#3390 closed defect (fixed)
HDF4 create test crashes for int32 & uint32, OSX 64bit
Reported by: | kyngchaos | Owned by: | warmerdam |
---|---|---|---|
Priority: | normal | Milestone: | |
Component: | GDAL_Raster | Version: | 1.7.0 |
Severity: | normal | Keywords: | |
Cc: |
Description
As title says. It looks like it's happening when the dataset is closed. From the autotest run:
TEST: Create: int32.tif rank=3 ... python2.6(60497,0x7fff7101abe0) malloc: *** error for object 0x1009a1e08: incorrect checksum for freed object - object was probably modified after being freed.
and the OSX crash report:
Exception Type: EXC_CRASH (SIGABRT) Exception Codes: 0x0000000000000000, 0x0000000000000000 Crashed Thread: 0 Dispatch queue: com.apple.main-thread Application Specific Information: *** error for object 0x1009a1e08: incorrect checksum for freed object - object was probably modified after being freed. Thread 0 Crashed: Dispatch queue: com.apple.main-thread 0 libSystem.B.dylib 0x00007fff884e8fe6 __kill + 10 1 libSystem.B.dylib 0x00007fff88589e32 abort + 83 2 libSystem.B.dylib 0x00007fff88578ae5 szone_error + 519 3 libSystem.B.dylib 0x00007fff884a562f small_free_list_remove_ptr + 154 4 libSystem.B.dylib 0x00007fff884a212f szone_free_definite_size + 3254 5 org.gdal.gdal 0x0000000101621136 sd_NC_free_cdf + 50 6 org.gdal.gdal 0x0000000101623a70 sd_ncclose + 232 7 org.gdal.gdal 0x00000001016293a2 SDend + 200 8 org.gdal.gdal 0x00000001010c58ce HDF4ImageDataset::~HDF4ImageDataset() + 66 9 org.gdal.gdal 0x00000001011b66ea GDALClose + 152 10 _gdal.so 0x00000001005010e5 _wrap_delete_Dataset + 306 11 org.python.python 0x0000000100017173 PyObject_Call + 112 12 org.python.python 0x0000000100017367 PyObject_CallFunctionObjArgs + 215 13 _gdal.so 0x00000001004fa2b3 SwigPyObject_dealloc + 117 14 org.python.python 0x0000000100044fb4 PyDict_Contains + 2517
Same HDF4.2r4 binaries that worked in GDAL 1.6. All other HDF4 tests with int32 and uint32 succeed (I blocked out the failures in hdf4_write.py so it would continue with the tests). I couldn't figure out how to emulate this test outside of Python to see if it was the python interface that's the problem.
Attachments (1)
Change History (12)
comment:1 by , 14 years ago
comment:2 by , 14 years ago
I have run autotest/gcore/hfa_write.py under valgrind on 64bit linux with nothing showing up. Without access to 64bit macos x there isn't much more that I can do.
comment:3 by , 14 years ago
William,
are you 200% positive that you have linked against HDF4.2r2 ? Because, on my 64bit Linux, when linking against an older HDF4 (4.1r4), I got the following crash :
TEST: Create: int32.tif rank=3 ... * glibc detected * /usr/bin/python: double free or corruption (out): 0x00000000008fe370 *
and Valgrind reported :
==13402== Invalid write of size 1 ==13402== at 0x4C250AF: memcpy (mc_replace_strmem.c:402) ==13402== by 0x8AF173F: DFKnb4b (in /usr/lib/libdf.so.4.1r4) ==13402== by 0x88913B2: (within /usr/lib/libmfhdf.so.4.1r4) ==13402== by 0x88923BE: NCvario (in /usr/lib/libmfhdf.so.4.1r4) ==13402== by 0x889A41A: SDreaddata (in /usr/lib/libmfhdf.so.4.1r4) ==13402== by 0x631E85E: HDF4ImageRasterBand::IReadBlock(int, int, void*) (hdf4imagedataset.cpp:303) ==13402== by 0x64DAD89: GDALRasterBand::GetLockedBlockRef(int, int, int) (gdalrasterband.cpp:1249) ==13402== by 0x64E9A19: GDALRasterBand::IRasterIO(GDALRWFlag, int, int, int, int, void*, int, int, GDALDataType, int, int) (rasterio.cpp:100) ==13402== by 0x65129F2: GDALChecksumImage (gdalchecksum.cpp:147) ==13402== by 0x5E498CA: _wrap_Band_Checksum (gdal_wrap.cpp:3792) ==13402== by 0x417E32: PyObject_Call (in /usr/bin/python2.5) ==13402== by 0x487072: PyEval_EvalFrameEx (in /usr/bin/python2.5) ==13402== Address 0x5d5ea27 is 3 bytes after a block of size 4 free'd ==13402== at 0x4C22B2E: free (vg_replace_malloc.c:323) ==13402== by 0x88903E4: SDPfreebuf (in /usr/lib/libmfhdf.so.4.1r4) ==13402== by 0x88911FA: (within /usr/lib/libmfhdf.so.4.1r4) ==13402== by 0x88923BE: NCvario (in /usr/lib/libmfhdf.so.4.1r4) ==13402== by 0x889A41A: SDreaddata (in /usr/lib/libmfhdf.so.4.1r4) ==13402== by 0x631E85E: HDF4ImageRasterBand::IReadBlock(int, int, void*) (hdf4imagedataset.cpp:303) ==13402== by 0x64DAD89: GDALRasterBand::GetLockedBlockRef(int, int, int) (gdalrasterband.cpp:1249) ==13402== by 0x64E9A19: GDALRasterBand::IRasterIO(GDALRWFlag, int, int, int, int, void*, int, int, GDALDataType, int, int) (rasterio.cpp:100) ==13402== by 0x65129F2: GDALChecksumImage (gdalchecksum.cpp:147) ==13402== by 0x5E498CA: _wrap_Band_Checksum (gdal_wrap.cpp:3792) ==13402== by 0x417E32: PyObject_Call (in /usr/bin/python2.5) ==13402== by 0x487072: PyEval_EvalFrameEx (in /usr/bin/python2.5)
When upgrading to HDF4.2r2, no problem any more.
comment:4 by , 14 years ago
I'm positive it's 4.2r4. No other 4.x version installed.
I could rebuild on OSX 10.5 to see what valgrind has to say (valgrind does not run on 10.6 yet), if that'll help.
comment:5 by , 14 years ago
With CPL_DEBUG on, and some debug print statements added to gdaltest.py, I see that strangely it's not happening when closing the created dataset, but after reopening it, when checking the checksum (maybe that is closing the bands?):
GDAL: GDALOpen(tmp/int32.tif.tst, this=0x104023e00) succeeds as HDF4. checksuming band 1 python2.6(86017,0x7fff7101abe0) malloc: *** error for object 0x104057c08: incorrect checksum for freed object - object was probably modified after being freed.
And also strange, just before this I had run the int32 test without it crashing, and it did report the checksum failed. It will only crash now.
comment:6 by , 14 years ago
OK, can't check valgrind - [u]int32 tests succeed on OSX 10.5 64bit...
Small changes in the source could have triggered an optimization bug in GCC 4.2, it's happened before.
I compiled my HDF4 library last Sep, at the first release of OSX 10.6 Xcode 3.2, and I'm at 3.2.1 (I think GCC was 4.2.1 from the start, just a different build). I can try recompiling HDF4.
comment:7 by , 14 years ago
optimization isn't the problem. With unoptimized HDF4 and GDAL, same crash. But there are more details in the trace:
0 libSystem.B.dylib 0x00007fff884e8fe6 __kill + 10 1 libSystem.B.dylib 0x00007fff88589e32 abort + 83 2 libSystem.B.dylib 0x00007fff88578ae5 szone_error + 519 3 libSystem.B.dylib 0x00007fff884a568b small_free_list_remove_ptr + 246 4 libSystem.B.dylib 0x00007fff884a212f szone_free_definite_size + 3254 5 org.gdal.gdal 0x0000000101869cc7 free_biobuf + 28 6 org.gdal.gdal 0x000000010186a532 xdrposix_destroy + 85 7 org.gdal.gdal 0x000000010184be7b sd_NC_free_cdf + 112 8 org.gdal.gdal 0x000000010185378e sd_ncclose + 295 9 org.gdal.gdal 0x0000000101857ab4 SDend + 299 10 org.gdal.gdal 0x0000000101141ab7 HDF4ImageDataset::~HDF4ImageDataset() + 97 11 org.gdal.gdal 0x00000001012bfcd0 GDALClose + 234 12 _gdal.so 0x00000001004fbb5a delete_GDALDatasetShadow(void*) + 39 13 _gdal.so 0x0000000100511457 _wrap_delete_Dataset + 197
I do remember that figuring out how to fix the XDR stuff in HDF4 for OSX was a PITA, and I may have missed something that only showed up now.
Maybe it's something in WriteRaster that writes bad data which causes the checksum to crash? I see that the Create test also copies but as a separate step, with WriteRaster.
comment:8 by , 14 years ago
With CPL_DEBUG on so I could save the test files, I compared the successful test with the failure. It looks like the failed HDF output reports as a 3-band image, though looking at it with a hex editor I see only a single bands' worth of data. The data is exactly the same as the correct HDF file (I did find and fix one 64bit issue I missed in HDF4 lib). But there is a bunch of extra junk in the trailer (?). And gdalinfo -checksum crashes on it.
comment:9 by , 9 years ago
Is there anybody who could repeat the test with OSX 64bit and refresh the ticket with the results?
comment:11 by , 9 years ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
P.S. Implied, but not stated above: it works on OSX 32bit.