GUI segfaults after iconv error during load

[pramsey]

From Nik: One thing I discoverd now. If I close the options window with the cross in the corner, it won't open next time I try, so I have to restart the loader gui. Since there is no other "cancel" button, I think closing with the cross should be a valid way of closing the window.

[pramsey]

OK, this was a larger change than I anticipated, so test carefully that other things haven't broken. I've made the dialogue an actual gtk_dialog, and made it complete create/destroy on each invocation. Known limitation: the null insert policy is not persisted. All the other options are. Can we remove the null policy from the GUI? (r5061)

[robe]

This may not be related, I'll have to test my really old copy, but I was able to crash this brought up by VS debugger.

First typing in UTF8 (which correctly gave an error like invalid encoding)

and then changing it to

UTF-8

Though I couldn't crash it with starting at UTF-8 first. So its almost like dirty memory or something I guess left after an error.

[nicklas]

First tests worked fine. Window reopens nicely

But I get the same crash as Regina.

[robe]

I have no clue what the null policy numbers mean. Never used it. I'm okay with getting rid of it. We can always introduce it back later I guess. I've never even seen a null geometry in a shape file. They always put some garbage geometry in there as a placeholder that breaks my validaton anyway (I presume so the index gets created correctly) so skipping nulls never does me any good personally. Can't speak for others. Anyrate if its some cryptic number that has to be typed in, its probably going to raise more questions than its worth.

[pramsey]

I can't replicate this on OS/X. I've added some minimal extra initialization stuff, but if it's still broken I'll have to see if I can replicate on Win32. Oddly, I don't even get errors from putting in bogus iconv encoding strings! It just runs through.

[robe]

Nope still crashes if I toggle between a wrong and right encoding on my windows xp and windows 7 boxes.

Also did you intentionally change the default character encoding to be the default for the OS. Its coming up as WINDOWS-1252 for me now on all my windows and windows 7. I could have sworn it used to be LATIN1, but perhaps I was mistaken.

I wonder if its the version of iconv you are using? Which version are you using or if its the OS? I'm using libiconv-1.13 I think.

[robe]

BTW -- this is not a reason to hold up PostGIS 1.5. The sequence of events I have to do to make this crash I doubt too many people will be exercising and if they do tough. I'm just being my usual sadist self :)

The window isn't exploding for me anymore so I'm content. Though would be nice to set the default back to LATIN1.

[pramsey]

Windows 1252 is more likely to be what's actually in a file than LATIN1. It's LATIN1+ where the + is some extra characters that MS included instead of ctrl characters in the hibit space. Things like the "66" "99" quotation marks, etc.

[robe]

Ah okay then disregard my comment

[nicklas]

I get the crash at once I try with UTF8. It takes a couple of seconds after the error message, then it crashes. I attach a picture with the crash which shows the error message and the err-log from windows if it can help. /Nicklas

[nicklas]

the error message from crash

[pramsey]

OK, I replicated and stepped into this. I found an unrelated crasher (well, it only crashed when running in gdb) that occurred just by opening and closing the options. The malformed iconv key problem is not related to iconv, it's related to the shutdown of the process. However, it's pretty hard to understand what's going wrong, the fault is actually occurring down in SHPClose.

pgui_action_import (widget=0xb59758, data=0x0) at shp2pgsql-gui.c:832
832             ShpLoaderDestroy(state);
(gdb) s
ShpLoaderDestroy (state=0x1324120) at shp2pgsql-core.c:1828
1828    {
(gdb) s
1831            if (state != NULL)
(gdb) s
1833                    if (state->hSHPHandle)
(gdb) s
1834                            SHPClose(state->hSHPHandle);
(gdb) s
SHPClose (psSHP=0xc01fe0) at shpopen.c:574
574     {
(gdb) s
578             if ( psSHP->bUpdated )
(gdb) s
584             free( psSHP->panRecOffset );
(gdb) s
585             free( psSHP->panRecSize );
(gdb) s
587             fclose( psSHP->fpSHX );
(gdb) s
588             fclose( psSHP->fpSHP );
(gdb) s
590             if ( psSHP->pabyRec != NULL )
(gdb) s
595             free( psSHP );
(gdb) s
0x00425ad0 in free ()
(gdb) s
Single stepping until exit from function free, which has no line number information.
0x77c2c21b in msvcrt!free () from C:\WINDOWS\system32\msvcrt.dll
(gdb) s
Single stepping until exit from function msvcrt!free, which has no line number information.
0x77c37420 in strerror () from C:\WINDOWS\system32\msvcrt.dll
(gdb) s
Single stepping until exit from function strerror, which has no line number information. 
0x77c2c227 in msvcrt!free () from C:\WINDOWS\system32\msvcrt.dll
(gdb) s
Single stepping until exit from function msvcrt!free, which has no line number information.
0x7c90ff2d in ntdll!RtlFreeThreadActivationContextStack ()
   from C:\WINDOWS\system32\ntdll.dll
(gdb) s
Single stepping until exit from function ntdll!RtlFreeThreadActivationContextStack, which has no line number information.
0x7c90e8cb in strchr () from C:\WINDOWS\system32\ntdll.dll
(gdb) s
Single stepping until exit from function strchr, which has no line number information.
0x7c90ff3c in ntdll!RtlFreeThreadActivationContextStack ()
   from C:\WINDOWS\system32\ntdll.dll
(gdb) s
Single stepping until exit from function ntdll!RtlFreeThreadActivationContextStack, which has no line number information.
0x7c927569 in ntdll!RtlPcToFileHeader () from C:\WINDOWS\system32\ntdll.dll
(gdb) s
Single stepping until exit from function ntdll!RtlPcToFileHeader, which has no line number information.

Program received signal SIGSEGV, Segmentation fault.
0x7c96f749 in ntdll!RtlpNtMakeTemporaryKey ()
   from C:\WINDOWS\system32\ntdll.dll
(gdb)

[pramsey]

This is so random, and makes so little sense, I wonder if it is a symptom of DLL hell. Or, rather, compiler hell. We have a mix of libraries behind this GUI, some compiled with mingw-gcc and some compiled with msvc. I have been warned in the past that mixing compilers was "dangerous" but never really given any specific cases or reasons why.

[pramsey]

I took rburhum's advice and put a watch on the memory area used by the hSHPHandle and it looks like it's windows clobbering it

(gdb) break shp2pgsql-core.c:896
Breakpoint 1 at 0x4082be: file shp2pgsql-core.c, line 896.
(gdb) r
Starting program: z:\win32build\output\pgsql-postgis\bin/./shp2pgsql-gui.exe
[New Thread 1572.0x1f4]
[New Thread 1572.0x6f8]

Breakpoint 1, ShpLoaderOpenShape (state=0x1330870) at shp2pgsql-core.c:897
897                     if (state->hSHPHandle == NULL)
(gdb) print state->hSHPHandle
$4 = (SHPHandle) 0x0
(gdb) break shp2pgsql-core.c:905
Breakpoint 2 at 0x407e56: file shp2pgsql-core.c, line 905.
(gdb) c
Continuing.

Breakpoint 2, ShpLoaderOpenShape (state=0x1330870) at shp2pgsql-core.c:907
907             state->hDBFHandle = DBFOpen(state->config->shp_file, "rb");
(gdb) print state->hSHPHandle
$5 = (SHPHandle) 0x131fae0
(gdb) watch *0x131fae0+0x04
Hardware watchpoint 3: *0x131fae0+0x04
(gdb) c
Continuing.
Hardware watchpoint 3: *0x131fae0+0x04

Old value = 2009464036
New value = 4063772
0x7c9276f9 in ntdll!RtlPcToFileHeader () from C:\WINDOWS\system32\ntdll.dll
(gdb)

This is an environment issue, not a code issue, I'm thinking. We're going to have to look at our Win32 compile at some point.

[pramsey]

For completeness, here's the stack at the time when the hSHPHandle gets clobbered

(gdb) print state->hSHPHandle
$1 = (SHPHandle) 0x132b1c0
(gdb) watch *0x132b1c0+0x04
Hardware watchpoint 2: *0x132b1c0+0x04
(gdb) c
Continuing.
Hardware watchpoint 2: *0x132b1c0+0x04

Old value = 2009464036
New value = 4063772
0x7c9276f9 in ntdll!RtlPcToFileHeader () from C:\WINDOWS\system32\ntdll.dll
(gdb) bt
#0  0x7c9276f9 in ntdll!RtlPcToFileHeader ()
   from C:\WINDOWS\system32\ntdll.dll
#1  0x7c96f85a in ntdll!RtlpNtMakeTemporaryKey ()
   from C:\WINDOWS\system32\ntdll.dll
#2  0x7c94bc4c in ntdll!LdrFindEntryForAddress ()
   from C:\WINDOWS\system32\ntdll.dll
#3  0x003e0000 in ?? ()
#4  0x50000061 in ?? ()
#5  0x0132b1c0 in ?? ()
#6  0x003e0000 in ?? ()
#7  0x0132b1c0 in ?? ()
#8  0x40000060 in ?? ()
#9  0x0022f008 in ?? ()
#10 0x858c2df6 in ?? ()
#11 0x00047c5d in ?? ()
#12 0x00b423fc in ?? ()
#13 0x0022f018 in ?? ()
#14 0x685eb9c8 in g_get_current_time ()
   from z:\win32build\output\pgsql-postgis\bin\libglib-2.0-0.dll
#15 0x7c927573 in ntdll!RtlPcToFileHeader ()
   from C:\WINDOWS\system32\ntdll.dll
#16 0x77c2c2de in msvcrt!free () from C:\WINDOWS\system32\msvcrt.dll
#17 0x003e0000 in ?? ()
#18 0x00000000 in ?? ()
(gdb)

Nice, hey?

[mloskot]

I'm not really able to build and debug this issue, no MinGW for me, but from the reported details I suspect it is a memory corruption issue, presumably heap corruption.

Paul suggested it rather is not allocation and deallocation crossing boundaries of DLL modules as Shapelib is built-in to the application.

The value of pointer changes in the way that it is clearly not valid address in memory, what would suggest memory corruption at some place before SHPClose is called.

Old value = 2009464036
New value = 4063772

I have no idea if binaries built using MinGW are debuggable under Windows, but perhaps  pageheap or even using debugging instruments:  _CrtSetDbgFlag and  _CrtCheckMemory:

assert(_CrtCheckMemory());

[robe]

Not even sure if PostGIS 2.0 has the same issue -- but 2.0 is so different and so better than the 1.5 version that it's not worth fixing 1.5

[pramsey]

I'm going to close this as works for me and see if it doesn't work for someone else.