mgserver lock up using GDAL Plugins (mrsid)

[andrewd]

Using the MGOS 2.2 RC1 with GDAL plugins (for mrsid), the mgserver service completely locks up when trying to access a mrsid layer, or when accessing a weblayout with a sid layer in it (even if it is not on). The only solution is to kill and restart the service.

The mrsids work in 2.0.2. I will attach an example sid...

Here are my steps for installing the plugins:

The download links all came from ​http://trac.osgeo.org/mapguide/wiki/MGOS21GdalProviderExtending, which is linked to from ​http://trac.osgeo.org/mapguide/wiki/MGOS21GdalProvider

1) Downloaded lti_dsdk_cdll.dll (6.0.7.1407) and lti_dsdk_dll.dll (6.0.7.1407) from ​http://download.osgeo.org/osgeo4w/release/gdal/gdal16-mrsid/ and then put them into the bin folder. 2) Downloaded Gdal_MrSID.dll (11/28/2008) from ​http://download.osgeo.org/osgeo4w/release/gdal/gdal16-mrsid/ and put that into bin\gdalplugins. 3) Now, right then, I tried things out (with debug on) - I tried it at this point, because these are the only files that first webpage talks about. My normal rasters worked (jpg), but I got errors complaining about my sid layers - the errors were saying that I was missing geotiff.dll... 4) Downloaded geotiff.dll, jpeg_osgeo.dll, libtiff.dll and zlib_osgeo.dll from ​http://www.jasonbirch.com/fdogdal/gdal_wavelet_3.4.zip as mentioned on the 2nd page. Put these into fdo folder. 5) Tried again - this time, no more complaints about missing dlls - only I get that same message as the other day, saying that the server connection's been lost...and then my mgserver.exe completely freezes. It freezes when trying to preview a sid layer or load a layout that has a sid layer in it (even though it's off).

When running the mgservice from the command line and then trying to either access a sid layer, or weblayout (with a sid layer), this is the output:

(5936) Server starting... (5936) Mentor Coordinate System Library (5936) Server started. GDAL: Auto register C:\Program Files (x86)\OSGeo\MapGuide\Server\bin\gdalplugins\gdal_ECW_JP2ECW.dll using GDALRegister_ECW_JP2ECW. GDAL: Auto register C:\Program Files (x86)\OSGeo\MapGuide\Server\bin\gdalplugins\gdal_MrSID.dll using GDALRegister_MrSID. GDAL: Auto register C:\Program Files (x86)\OSGeo\MapGuide\Server\bin\gdalplugins\gdal_ECW_JP2ECW.dll using GDALRegister_ECW_JP2ECW. GDAL: Auto register C:\Program Files (x86)\OSGeo\MapGuide\Server\bin\gdalplugins\gdal_MrSID.dll using GDALRegister_MrSID. GDAL: GDALOpen(E:\drawings and files\drawings\4300\438253\gis/airphotos/Google20102010.jpg, this=03DD57C8) succeeds as JPEG. GDAL: GDALClose(E:\drawings and files\drawings\4300\438253\gis/airphotos/Google20102010.jpg, this=03DD57C8) (3928) Problem:\nCannot establish connection to the MapGuide Server.\n\nPossible Causes:\n-Your session has been idle for too long;\n-MapGuide Server is no longer responding;\n-Internet connection problems.\n\nSolution:\nRestart your application / session, restart the MapGuide Server service, or contact with the server administrator.

[andrewd]

problem sid

[andrewd]

Hopefully this ticket doesn't get lost. I know it's only been a few weeks, but with no owner, I'm a little afraid!

[trevorwekel]

Hi Andrew, I have a testing environment set up on a 64 bit OS and I am having zero luck getting the MrSID plug in to work. The MapGuide Server freezes when the SID is accessed and throws an invalid heap error. I suspect it might be WOW64 conflict with the msvcr71.dll (circa Visual Studio .Net 2003) which is needed by the plugin.

However, I can access the SID on a 32 bit OS. Did you install MapGuide 32 bit on 64 bit OS?

[andrewd]

I was working with 32bit only.

[andrewd]

...though I suppose that I should point out that this is on a 64bit OS (Server 2008 x64).

[trevorwekel]

I think we have discovered the problem. I did some checking on Visual Studio .Net 2003 and Microsoft dropped support for it on Windows Vista. They strongly suggest moving to Visual Studio 2005 ​http://msdn.microsoft.com/en-us/vstudio/aa948854.aspx.

This leads me to believe that using Visual Studio 2003 dlls on any operating system newer than Vista is a dangerous approach since the compiler that uses those dlls is "not supported". I think it is reasonable to assume that VS 2003 dlls will not work under WOW64. I seriously doubt that Microsoft has tested it.

What does this mean for the GDAL plugins? Basically, we need to compile all the plug in libraries with Visual Studio 2008. This will include the SDKs from ERDAS and LizardTech. The compilation work will also be required to resolve threading issues. Most of the thread checking tools expect the same system and c-runtime allocators to be used throughout the application.

This could be a lot of work if no one has started on it. Hopefully ERDAS and LizardTech have VS 2008 SDKs available.

[andrewd]

Thanks for looking into it, Trevor! Given that this likely won't be fixed for v2.2 (final), maybe there should be some mention / warning about the GDAL plugins (and the GDAL not working in x64, either) in the release notes. Some users may depend on that functionality, and if an update breaks that, that will be a bad user experience...

I've been burnt by this just about every time I've tried a new version of MGOS (since 1.x) and as much as it pains me to say it, I'm very close to giving up on those formats that aren't supported out of the MGOS box. I'll always keep my fingers crossed, of course...

[trevorwekel]

I was digging around the GDAL project website and found a link to some updated GDAL 1.6 binaries. These binaries are compiled with Visual Studio 2008. They also include ECW and SID libraries compiled with Visual Studio 2008.

I used the 32 bit GDAL 1.6 binaries from ​http://vbkto.dyndns.org/sdk/release-1500-gdal-1-6-mapserver-5-6.zip and dumped them on my Windows 2008 R2 64 bit test machine.

The consistent dll set makes a difference. SID, ECW, and JPEG2000 are working with a single user hitting them. I haven't tried multi-user yet.

Here's the dlls I copied over:

Server\bin\libecwj2.dll
Server\bin\lti_dsdk_dll.dll
Server\bin\gdalplugins\gdal_ECW_JP2ECW.dll
Server\bin\gdalplugins\gdal_MrSID.dll
Server\Fdo\gdal16.dll
Server\Fdo\geos_c.dll
Server\Fdo\iconv.dll
Server\Fdo\libcurl.dll
Server\Fdo\libexpat.dll
Server\Fdo\libpq.dll
Server\Fdo\libtiff.dll
Server\Fdo\proj.dll
Server\Fdo\spatialite.dll
Server\Fdo\xerces-c_2_8.dll
Server\Fdo\zlib1.dll

[andrewd]

That certainly does the trick for me. Works like a charm now. I gave it a fair work out (though only just the 1 user), and things kept working. I was trying a project that has a bunch of sids like the one from this ticket...lots of panning and zooming, really quickly. I couldn't break it.

It should be noted that a few of those files that you listed didn't exist in my FDO folder at all...

[trevorwekel]

Using the updated GDAL 1.6 binaries seems to correct the issue. A wiki item has been created ​http://trac.osgeo.org/mapguide/wiki/MGOS22GdalProvider