Opened 18 years ago

Last modified 18 years ago

#1132 closed defect (fixed)

multi-threaded reading

Reported by: list672000@… Owned by: warmerdam
Priority: high Milestone:
Component: GDAL_Raster Version: unspecified
Severity: normal Keywords:
Cc:

Description

The FAQ mentions that only a few drivers have the multi-threaded read 
capability.

I wrote a program to test a driver.  The program takes a filename as input.  
The program starts a few threads.  Each thread reads rows from the specified 
file.

I intend to attach the program.

The program currently crashes when I specify a ERDAS .img file.  I looked into 
the HFA driver code.  There are some a few lstatic variables that seem to cause 
the problems.

Attachments (1)

testMultiThreadedRead.cpp (4.5 KB ) - added by list672000@… 18 years ago.
small program that starts numerous threads that read from a specified file

Download all attachments as: .zip

Change History (6)

by list672000@…, 18 years ago

Attachment: testMultiThreadedRead.cpp added

small program that starts numerous threads that read from a specified file

comment:1 by mike.dunne@…, 18 years ago

Quick note to say we (R&D at ER Mapper) have also run into this bug with our 
read support for ERDAS Imagine files in ER Mapper. We are using GDAL 1.3.1.
We have worked around it via mutex at a higher level(with some performance 
hit), and will be happy to test any fixes that come thru (not really in a 
position to fix it - Imagine files are not our strength:).

Mike Dunne




comment:2 by warmerdam, 18 years ago

I have spent an hour trying to reproduce problems with my "multireadtest" 
program (which seems to do what your provided program does and more) against
GDAL 1.3.2 (nearly released) and a few Imagine files I have sitting around
but I have not encountered any problems.  I *suspect* I may have fixed the
issue.

Could either of you test with GDAL 1.3.2 beta2 and let me know if you still
see problems?  If so, and if you are able to reproduce them with multireadtest,
an example file and commandline likely to tickle the error would be appreciated.

Note, my testing was all on Linux with a debug build. 

comment:3 by warmerdam, 18 years ago

Frank,

I just checked my email and saw your reply.  I've
attached a sample .img file with a corresponding .tif
file.  I ran the multireadtest program (compiled from
the GDAL 2006-05-04 cvs download) with those two
files.  When running with the .tif file the program
did not crash.  When running the .img file with the
following command line, the program sometimes crashes,
sometimes reports, "Checksum error in worker thread",
and sometimes (rarely in the small sample of tests I
ran) completes without an apparent error.

./multireadtest.exe -t 40 -i 40
c:/somePath/smallImage.img

    Mike

comment:4 by warmerdam, 18 years ago

I have confirmed seeing very intermittent errors with this configuration
on linux.  I will dig into it more.  

comment:5 by warmerdam, 18 years ago

The ExtractInstValue() functions were returning values in a static which
was ... bad.  

The problem has been corrected in CVS and will likely appear in a 
GDAL 1.3.2.1 or perhaps GDAL 1.3.3 release. 

Whew, this has been a long standing bug. 
Note: See TracTickets for help on using tickets.