Opened 13 years ago

Closed 13 years ago

Last modified 13 years ago

#2542 closed defect (fixed)

Opening large GIF files causes GDALOpen() to hang.

Reported by: warmerdam Owned by: warmerdam
Priority: normal Milestone: 1.6.1
Component: GDAL_Raster Version: unspecified
Severity: normal Keywords: gif biggif
Cc: gaopeng

Description

Attempting to open a large GIF file (24000x18000) results in GDALOpen() appearing to hang. While it does eventually succeed if sufficient memory is available it is very intrusive to the application.

Change History (7)

comment:1 Changed 13 years ago by warmerdam

Owner: changed from gaopeng to warmerdam
Status: newassigned

The way the GIF driver is currently setup it reads the whole GIF file into memory in the open function. For the sample image this means using 400MB of RAM and having to do a substantial amount of decoding before processing can continue.

In theory it might be possible to restructure to avoid using DGifSlurp() but this appears to be quite a substantial job. Instead, I'm just modifying the slurp function to error out for files larger than 100 megapixels. This is very rare in GIF format.

comment:2 Changed 13 years ago by warmerdam

Ok, I have applied this change in trunk (r15186).

I'm a bit hesitant to migrate this back into 1.5 branch for now.

comment:3 Changed 13 years ago by warmerdam

Cc: gaopeng added

comment:4 Changed 13 years ago by warmerdam

Milestone: 1.6.0
Resolution: fixed
Status: assignedclosed

Closing as completed. Reopen if there are problems.

comment:5 Changed 13 years ago by gaopeng

Resolution: fixed
Status: closedreopened

The current fix by setting a hard limit doesn't work, sometimes it's too low, and other times, it's too high. Look at the possibility to restructure to avoid using DGifSlurp() (but this appears to be quite a substantial job).

comment:6 Changed 13 years ago by warmerdam

Keywords: biggif added
Resolution: fixed
Status: reopenedclosed

I have implemented a "BIGGIF" driver that reads scanline by scanline. It has a number of deficiencies (no support for nodata transparency blocks, and only supports one image-band even in multi-image files) so it is registered after the normal GIF driver and only used for really big files.

To support interlaced files effectively the BIGGIF driver will create a temporary TIFF file to hold the image data for random access if anything but a once-through sequential read is done.

This code is added in trunk (r16023,r16025) and a small test for this driver is added in trunk (r16024). The new driver has also been added in 1.6 branch (r16026).

comment:7 Changed 13 years ago by Even Rouault

Create the temporary TIFF file with SPARSE_OK=YES to avoid filling up the whole file with zeros even if we have read a small part of it, just before destroying it. Fixed in trunk in r16236 and in branches/1.6 in r16237

Note: See TracTickets for help on using tickets.