Opened 17 years ago
Last modified 17 years ago
#1418 closed defect (fixed)
Conversion of MrSID datasets from NASA Zulu to JP2ECW hangs
Reported by: | Mateusz Łoskot | Owned by: | warmerdam |
---|---|---|---|
Priority: | highest | Milestone: | |
Component: | GDAL_Raster | Version: | 1.4.0 |
Severity: | major | Keywords: | |
Cc: |
Description
During tests of MrSID to JP2ECW translator, some issue has been discovered when trying to convert big MrSID images downloaded from NASA Zulu server. The conversion process hangs for 2-3 hours with 100% CPU time but without any progress of the translation process. Here is the full command line using GDAL utilities from FWTools 1.1.0, on Windows XP: gdal_translate -of JP2ECW -co "LARGE_OK=YES" N-35-50_2000.sid out.jp2 Mark Sheridan noticed the same issue when trying to convert images from the NASA Zulu server. Mark and I tested on different images from the Zulu server, so it seems does not depend on a particular image but it applies to all images from Zulu server. Thus, It should be sufficient to download any image from the Zulu: https://zulu.ssc.nasa.gov/mrsid/ I'm going to attach two screenshots taken at the beginning and at the end of the conversion process.
Attachments (2)
Change History (5)
by , 17 years ago
Attachment: | nasa_test_1.png added |
---|
comment:1 by , 17 years ago
Here I paste last comments from Mark Sheridan: -------- However with this version, on the NASA Zulu data (mentioned in previous email), it seems to hang at about 10%. I tested the tile "S-50-30_2000" (Perth in Western Australia), 190Mb compressed, which is 5.2Gb uncompressed, which would normally compress in no more than an hour, but the process was still going with no visible progress and 100% CPU after 3 hours, so I killed it. --------
comment:2 by , 17 years ago
I believe the problem is classic cache thrashing. The default cache is too small to hold one full row of 128 pixel tall tiles for the input mrsid, so the first row of tiles is re-read for each output scanline being generated. I would suggest calling GDALSetCacheMax( 100000000 ); See if that helps.
comment:3 by , 17 years ago
Frank, Thanks for the hint. Indeed, the problem was with too tiny cache size. I think the bug can be closed, so I'm doing it now. BTW, on my P4 3GHz with 512MB RAM, converting ~190 MB MrSID file to JP2ECW with target ratio 20:1 takes ~ 1hr 20min. The output is ~400MB Do you think it's because I have not enough RAM or this timing is normal?
Note:
See TracTickets
for help on using tickets.
Beginning of the conversion process