5.6.0beta2 with SDE connectiontype causes segmentation fault

[jmckenna]

During FOSS4G benchmarking exercise we discovered a problem with MapServer5.6.0beta2 and SDE connectiontype. shp2img and mapserv cgi causes a segmentation fault when trying to display the SDE layer. The same layer draws fine with MapServer 5.4.2

[warmerdam]

The problem shows up somewhat like this in valgrind:

==15842== Invalid read of size 1
==15842==    at 0x4158DE9: SES_alloc_copy_wstring_array (in /usr/local/lib/libsd
e.so)
==15842==    by 0x416832F: SE_queryinfo_set_columns (in /usr/local/lib/libsde.so
)
==15842==    by 0x8157433: getSDEQueryInfo (mapsde.c:964)
==15842==    by 0x8158684: msSDELayerWhichShapes (mapsde.c:1561)
==15842==    by 0x808563F: msLayerWhichShapes (maplayer.c:121)
==15842==    by 0x8093356: msDrawVectorLayer (mapdraw.c:949)
==15842==    by 0x8092CE6: msDrawLayer (mapdraw.c:811)
==15842==    by 0x8091CE2: msDrawMap (mapdraw.c:457)
==15842==    by 0x8053AB3: main (shp2img.c:296)
==15842==  Address 0x72CCCA8 is 0 bytes inside a block of size 8 free'd
==15842==    at 0x4004FDA: free (vg_replace_malloc.c:233)
==15842==    by 0x8056B75: msFree (mapfile.c:117)
==15842==    by 0x8056BB7: msFreeCharArray (mapfile.c:130)
==15842==    by 0x8086699: msLayerWhichItems (maplayer.c:489)
==15842==    by 0x80931E2: msDrawVectorLayer (mapdraw.c:928)
==15842==    by 0x8092CE6: msDrawLayer (mapdraw.c:811)
==15842==    by 0x8091CE2: msDrawMap (mapdraw.c:457)
==15842==    by 0x8053AB3: main (shp2img.c:296)

The problem seems to be that an items array is being freed in msLayerWhichItems() but still used later in mapsde.c. I found commenting out the line:

  if(freeitems) msFreeCharArray(items, numitems);

avoids the crash though it is likely this causes a memory leak in some circumstances. I'm going to turn this over to Steve who did the new one pass query which seems to be responsible for the issue. I can provide further support on the benchmarking system if needed. I'm taking the liberty of making this a high priority 5.6 milestone bug since it appears to be a significant regression.

[sdlime]

Problem looks to be that SDE is the only driver that overrides the default item creation function for some reason. That code doesn't allocate new memory for the item list which leaves items and layer->items pointing at the same memory which is freed later resulting in the crash. I believe the solution will be to avoid the vtable->createItems() function and just do it myself- much safer... Will update this if that works.

Steve

[sdlime]

I committed a fix in r9406 that should do the trick. I can't test from home though since our friggin' SDE server is locked and I don't have administrative access. This should avoid the double free and any memory leak. Other drivers should not be impacted since they were using essentially the same code albeit via the default create items functions.

Will confirm first thing in the morning...

Steve

[sdlime]

Ok, this fixes the segfault but now msSDELayerInitItemInfo() is throwing an error:

"A specified CLASSITEM, FILTERITEM or expression key cannot be found."

CC'ing hobu on the significance. Doing a bit of clean up (e.g. freeing any item into) seems to get past that error and I can get things to draw. However, there looks to be a problem with the computed index values (e.g. labelitemindex). Same map pulls the wrong item between 5.4 and 5.6.

Steve

[hobu]

 http://trac.osgeo.org/mapserver/browser/trunk/mapserver/mapsde.c#L1920 and  http://trac.osgeo.org/mapserver/browser/trunk/mapserver/mapsde.c#L1937 kind of describe what's going on. We needed to fetch all of the items because the user might be taking advantage of the SDE driver's join support.

[sdlime]

Thanks Howard, I changed msLayerWhichItems() to grab everything for the SDE case (we do the same for inline layers) and things seem ok now. Committed in r9415. I rolled back r9406 since that ultimately wasn't the right fix. Marking as fixed...

Steve