Opened 8 years ago

Last modified 8 years ago

#820 new enhancement

FDO GDAL as multithreading data provider

Reported by: sgamperl Owned by: warmerdam
Priority: major Milestone: 3.7.0
Component: GDAL Provider Version: 3.7.0
Severity: 2 Keywords:
Cc: trevorwekel, zspitzer, sgamperl External ID:

Description

As the connection capabilities of the open source GDAL provider implies the GDAL provider is a single threaded data provider. Are there any attempts to make the provider capable of handling multiple request through different threads?

We use the FDO GDAL provider in combination with MapGuide? Open Source. Many users will use access the MapGuide? simultaneously to get an image. Serial access through the FDO GDAL provider is a big bottleneck . What have to be done, to make the FDO GDAL provider threadsafe? GDAL itself can be used by many different threads. Thanks for your comments.

Change History (4)

comment:1 Changed 8 years ago by warmerdam

I will note that GDAL 1.7 and newer are pretty safe in a multithreaded for read only access to common formats like GeoTIFF. Not all drivers are threadsafe though and things are definately not threadsafe in the context of *writing* through GDAL. Possibly the provider could avoid acquiring the GDAL lock for read access to some whitelisted file formats or alternatively a flag could be exposed providing some control over this to the user.

comment:2 Changed 8 years ago by zspitzer

Milestone: 3.6.03.7.0
Version: 3.6.03.7.0

Another Mapguide user recently commissioned Trevor to upgrade FDO GDAL to 1.9, there are a series of patches pending for MapGuide? which haven't been merged or tested yet over in http://trac.osgeo.org/mapguide/ticket/462

Would you be interested in contributing either funding or developer time towards this work?

I also bumped release and milestone to 3.7

comment:3 Changed 8 years ago by warmerdam

Cc: trevorwekel zspitzer added

comment:4 Changed 8 years ago by sgamperl

Cc: sgamperl added

At the moment it will be difficult to contribute developer time because I have not enough knowledge of the internals of GDAL and the FDO GDAL Provider to do code changes. In the near future we would like to test the new Autodesk Infrastructure Server 2013 and a new version of the MapGuide? OpenSource? server. We can run several tests on the server if would be a multithreading GDAL FDO provider and provide the test results to you.

In another project we use the GDAL provider from several threads to visualize GeoTIFF or MrSID. It seems to work proberly and the performance is very good.

Note: See TracTickets for help on using tickets.