Opened 19 years ago
Closed 15 years ago
#864 closed defect (fixed)
gdalwarp: overflow from dstfile name
| Reported by: | Owned by: | warmerdam | |
|---|---|---|---|
| Priority: | high | Milestone: | |
| Component: | GDAL_Raster | Version: | unspecified |
| Severity: | normal | Keywords: | tps |
| Cc: | Markus Neteler |
Description
A couple of weird bugs, both when using 'gdalwarp -tps'. 1) processing runs slower and output appears as a series of swirly lines if the GCPs are not sorted by line (*as below) before being applied with gdal_translate. (previously the GCPs were in somewhat random order) * [-gcp pixel line easting northing] 2) this is the really weird one, maybe a memory buffer getting stomped on? If the gdalwarp dstfile is less then 20 chars then the output appears as a series of swirly lines. If I make the output filename 20 chars+, it works fine. ?! GDAL 1.2.6.0, released 2005/03/13 on Debian (from debian package). reproduceable on multiple machines. more details available upon request. thanks, Hamish
Attachments (3)
Change History (23)
by , 19 years ago
| Attachment: | geo18_tps.jpg added |
|---|
comment:1 by , 19 years ago
recompiled on another machine from source (previous was with the Debian
packages). Same result but now I can use gdb for two segfaults I can
trigger, in addition to the weird output bug mentioned above.
Both segfaults trace back to the GDALWarpOperation::WarpRegion stage.
Segfault 1) Different GeoTIFF image than the above failure, it gets to
100% complete, then segfaults. Leaves behind a corrupted output GeoTiff.
Here's the gdb session:
$ gdb `which gdalwarp`
(gdb) run -tps -co compress=lzw 8right_gcp.tif 8right_warped_xyz123def.tif
Starting program: /usr/local/bin/gdalwarp -tps -co compress=lzw 8right_gcp.tif
8right_warped_xyz123def.tif
[Thread debugging using libthread_db enabled]
[New Thread 16384 (LWP 9987)]
Creating output file that is 9627P x 6717L.
:0...10...20...30...40...50...60...70...80...90...100 - done.
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 16384 (LWP 9987)]
0x4070756b in memset () from /lib/libc.so.6
(gdb) bt f
#0 0x4070756b in memset () from /lib/libc.so.6
No symbol table info available.
#1 0x405138ad in _TIFFmemset () from /usr/lib/libtiff.so.4
No symbol table info available.
#2 0x40508833 in TIFFInitSGILog () from /usr/lib/libtiff.so.4
No symbol table info available.
#3 0x40511f85 in TIFFReadBufferSetup () from /usr/lib/libtiff.so.4
No symbol table info available.
#4 0x4051131c in TIFFReadEncodedStrip () from /usr/lib/libtiff.so.4
No symbol table info available.
#5 0x400f5e05 in GTiffRasterBand::IReadBlock (this=0x80512e8, nBlockXOff=0,
nBlockYOff=0,
pImage=0x8a9ab00) at geotiff.cpp:518
poGDS = (GTiffDataset *) 0x80511d8
nBlockBufSize = 9627
nBlockId = 0
nBlockIdBand0 = 0
eErr = CE_None
#6 0x401a8d3a in GDALRasterBand::GetBlockRef (this=0x80512e8, nXBlockOff=0,
nYBlockOff=0,
bJustInitialize=0) at gdalrasterband.cpp:1142
poBlock = (class GDALRasterBlock *) 0x8706db0
#7 0x401ac1da in GDALRasterBand::IRasterIO (this=0x80512e8, eRWFlag=GF_Write,
nXOff=4813, nYOff=0,
nXSize=4814, nYSize=6717, pData=0x41409008, nBufXSize=4814, nBufYSize=6717,
eBufType=GDT_Byte,
nPixelSpace=1, nLineSpace=4814) at rasterio.cpp:175
bJustInitialize = 0
nSrcByteOffset = 0
nBandDataSize = 1
nBufDataSize = 1
pabySrcBlock = (GByte *) 0x0
poBlock = (class GDALRasterBlock *) 0x40701176
nLBlockX = -1
nLBlockY = 0
iBufYOff = 0
iBufXOff = 1081866368
iSrcY = 0
iSrcX = 0
dfSrcX = 2.1219957909652723e-314
dfSrcY = 7.7813168390427143e-292
dfSrcXInc = 48.371032714843786
dfSrcYInc = 263.61883544908824
#8 0x401a00b0 in GDALDataset::IRasterIO (this=0x80511d8, eRWFlag=GF_Write,
nXOff=4813, nYOff=0,
nXSize=4814, nYSize=6717, pData=0x41409008, nBufXSize=4814, nBufYSize=6717,
eBufType=GDT_Byte,
nBandCount=1, panBandMap=0x80508c0, nPixelSpace=1, nLineSpace=4814,
nBandSpace=32335638)
at gdaldataset.cpp:1290
poBand = (class GDALRasterBand *) 0x80512e8
pabyBandData = (GByte *) 0x41409008 ""
iBandIndex = 0
eErr = CE_None
#9 0x401a048c in GDALDataset::RasterIO (this=0x80511d8, eRWFlag=GF_Write,
nXOff=4813, nYOff=0,
nXSize=4814, nYSize=6717, pData=0x41409008, nBufXSize=4814, nBufYSize=6717,
eBufType=GDT_Byte,
nBandCount=1, panBandMap=0x80508c0, nPixelSpace=1, nLineSpace=4814,
nBandSpace=32335638)
at gdaldataset.cpp:1481
i = 1
bNeedToFreeBandMap = 0
eErr = CE_None
#10 0x401a0531 in GDALDatasetRasterIO (hDS=0x80511d8, eRWFlag=GF_Write,
nXOff=4813, nYOff=0,
nXSize=4814, nYSize=6717, pData=0x41409008, nBufXSize=4814, nBufYSize=6717,
eBufType=GDT_Byte,
nBandCount=1, panBandMap=0x80508c0, nPixelSpace=0, nLineSpace=0, nBandSpace=0)
at gdaldataset.cpp:1515
poDS = (GDALDataset *) 0x80511d8
#11 0x401cf63e in GDALWarpOperation::WarpRegion (this=0xbffff9b0, nDstXOff=4813,
nDstYOff=0,
nDstXSize=4814, nDstYSize=6717, nSrcXOff=0, nSrcYOff=0, nSrcXSize=1200,
nSrcYSize=11100)
at gdalwarpoperation.cpp:1190
eErr = CE_None
iBand = 1
pDstBuffer = (void *) 0x41409008
nWordSize = 1
nBandSize = 32335638
pszInitDest = 0x8055d8a "0"
#12 0x401ce7d9 in GDALWarpOperation::ChunkAndWarpImage (this=0xbffff9b0,
nDstXOff=0, nDstYOff=0,
nDstXSize=9627, nDstYSize=6717) at gdalwarpoperation.cpp:683
panThisChunk = (int *) 0x80570d8
dfChunkPixels = 32335638
eErr = CE_None
iChunk = 1
dfPixelsProcessed = 32328921
dfTotalPixels = 64664559
#13 0x0804aca6 in main (argc=6, argv=0x80504a8) at gdalwarp.cpp:657
hSrcDS = 0x80509f8
hDstDS = 0x80511d8
pszFormat = 0x804b6cc "GTiff"
pszTargetSRS = 0x0
pszSourceSRS = 0x0
pszSrcFilename = 0x804fdc0 "8right_gcp.tif"
pszDstFilename = 0x80504c8 "8right_warped_xyz123def.tif"
bCreateOutput = 1
i = 1
nOrder = -1
hTransformArg = (void *) 0x8057358
hGenImgProjArg = (void *) 0x8055f60
hApproxArg = (void *) 0x8057358
papszWarpOptions = (char **) 0x8055d60
dfErrorThreshold = 0.125
dfWarpMemoryLimit = 0
pfnTransformer = 0x8049454 <GDALApproxTransform>
papszCreateOptions = (char **) 0x0
eOutputType = GDT_Unknown
eWorkingType = GDT_Unknown
eResampleAlg = GRA_NearestNeighbour
pszSrcNodata = 0x0
pszDstNodata = 0x0
bMulti = 0
psWO = (GDALWarpOptions *) 0x80574d0
oWO = {_vptr.GDALWarpOperation = 0x403735d0, psOptions = 0x8056b60,
dfProgressBase = 0.49994806274020981, dfProgressScale = 0.50005193725979014,
hThread1Mutex = 0x0,
hThread2Mutex = 0x0, hIOMutex = 0x0, hWarpMutex = 0x0, nChunkListCount = 2,
nChunkListMax = 3,
panChunkList = 0x80570b8, bReportTimings = 0, nLastTimeReported = 0}
(gdb) l
657 oWO.ChunkAndWarpImage( 0, 0,
658 GDALGetRasterXSize( hDstDS ),
659 GDALGetRasterYSize( hDstDS ) );
660 }
661
662 /* -------------------------------------------------------------------- */
663 /* Cleanup */
664 /* -------------------------------------------------------------------- */
665 if( hApproxArg != NULL )
666 GDALDestroyApproxTransformer( hApproxArg );
(gdb) frame 5
#5 0x400f5e05 in GTiffRasterBand::IReadBlock (this=0x80512e8, nBlockXOff=0,
nBlockYOff=0,
pImage=0x8a9ab00) at geotiff.cpp:518
518 if( TIFFReadEncodedStrip( poGDS->hTIFF, nBlockId, pImage,
(gdb) l
513 eErr = CE_Failure;
514 }
515 }
516 else
517 {
518 if( TIFFReadEncodedStrip( poGDS->hTIFF, nBlockId, pImage,
519 nBlockBufSize ) == -1 )
520 {
521 memset( pImage, 0, nBlockBufSize );
522 CPLError( CE_Failure, CPLE_AppDefined,
(gdb) frame 6
#6 0x401a8d3a in GDALRasterBand::GetBlockRef (this=0x80512e8, nXBlockOff=0,
nYBlockOff=0,
bJustInitialize=0) at gdalrasterband.cpp:1142
1142 if( !bJustInitialize
(gdb) l
1137 CPLError( CE_Failure, CPLE_AppDefined, "Internalize failed",
1138 nXBlockOff, nYBlockOff);
1139 return( NULL );
1140 }
1141
1142 if( !bJustInitialize
1143 && IReadBlock(nXBlockOff,nYBlockOff,poBlock->GetDataRef()) !=
CE_None)
1144 {
1145 delete poBlock;
1146 CPLError( CE_Failure, CPLE_AppDefined,
$ gdalinfo 8right_gcp.tif
Driver: GTiff/GeoTIFF
Size is 1200, 11100
Coordinate System is `'
GCP Projection =
GCP[ 0]: Id=1, Info=
(1198,3) -> [...]
[...]
GCP[ 23]: Id=24, Info=
(970,11040) -> [...]
Corner Coordinates:
Upper Left ( 0.0, 0.0)
Lower Left ( 0.0,11100.0)
Upper Right ( 1200.0, 0.0)
Lower Right ( 1200.0,11100.0)
Center ( 600.0, 5550.0)
Band 1 Block=1200x6 Type=Byte, ColorInterp=Gray
--------
Segfault 2) Another segfault in another way using a different file.
I guess this one's only a symptom of the matrix gen failing:
$ gdb `which gdalwarp`
[...]
(gdb) run -tps -co compress=lzw 18left_gcp.tif 18left_warped_abcdefg.tif
Starting program: /usr/local/bin/gdalwarp -tps -co compress=lzw 18left_gcp.tif
18left_warped_abcdefg.tif
[Thread debugging using libthread_db enabled]
[New Thread 16384 (LWP 9969)]
There is a problem to invert the interpolation matrix
Creating output file that is 4513P x 12251L.
There is a problem to invert the interpolation matrix
:0.
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 16384 (LWP 9969)]
0x401ca487 in GWKNearestNoMasksByte (poWK=0xbffff7d0) at gdalwarpkernel.cpp:2065
2065 poWK->papabyDstImage[iBand][iDstOffset] =
(gdb) bt f
#0 0x401ca487 in GWKNearestNoMasksByte (poWK=0xbffff7d0) at gdalwarpkernel.cpp:2065
iSrcX = -2147483648
iSrcOffset = -2147483648
iSrcY = -2147483648
iBand = 0
iDstOffset = 0
iDstX = 0
iDstY = 0
nDstXSize = 4513
nDstYSize = 12251
nSrcXSize = 0
nSrcYSize = 0
eErr = CE_None
padfX = (double *) 0x80594f8
padfY = (double *) 0x8062208
padfZ = (double *) 0x806af18
pabSuccess = (int *) 0x8073c28
#1 0x401c695e in GDALWarpKernel::PerformWarp (this=0xbffff7d0) at
gdalwarpkernel.cpp:591
eErr = CE_None
#2 0x401cfda6 in GDALWarpOperation::WarpRegionToBuffer (this=0xbffff9b0,
nDstXOff=0, nDstYOff=0,
nDstXSize=4513, nDstYSize=12251, pDataBuf=0x40f04008, eBufDataType=GDT_Byte,
nSrcXOff=0,
nSrcYOff=0, nSrcXSize=0, nSrcYSize=0) at gdalwarpoperation.cpp:1475
eErr = CE_None
i = 1
nWordSize = 1
oWK = {_vptr.GDALWarpKernel = 0x40373570, papszWarpOptions = 0x8056b80,
eResample = GRA_NearestNeighbour, eWorkingDataType = GDT_Byte, nBands = 1,
nSrcXSize = 0,
nSrcYSize = 0, papabySrcImage = 0x8056200, papanBandSrcValid = 0x0,
panUnifiedSrcValid = 0x0,
pafUnifiedSrcDensity = 0x0, nDstXSize = 4513, nDstYSize = 12251,
papabyDstImage = 0x8056f40,
panDstValid = 0x0, pafDstDensity = 0x0, nSrcXOff = 0, nSrcYOff = 0, nDstXOff =
0, nDstYOff = 0,
pfnTransformer = 0x8049454 <GDALApproxTransform>, pTransformerArg = 0x8056ef0,
pfnProgress = 0x8049834 <GDALTermProgress>, pProgress = 0x0, dfProgressBase = 0,
dfProgressScale = 1}
#3 0x401cf5a8 in GDALWarpOperation::WarpRegion (this=0xbffff9b0, nDstXOff=0,
nDstYOff=0,
nDstXSize=4513, nDstYSize=12251, nSrcXOff=0, nSrcYOff=0, nSrcXSize=0,
nSrcYSize=0)
at gdalwarpoperation.cpp:1181
eErr = CE_None
iBand = 1
pDstBuffer = (void *) 0x40f04008
nWordSize = 1
nBandSize = 55288763
pszInitDest = 0x8056b9a "0"
#4 0x401ce7d9 in GDALWarpOperation::ChunkAndWarpImage (this=0xbffff9b0,
nDstXOff=0, nDstYOff=0,
nDstXSize=4513, nDstYSize=12251) at gdalwarpoperation.cpp:683
panThisChunk = (int *) 0x8056f18
dfChunkPixels = 55288763
eErr = 134532600
iChunk = 0
dfPixelsProcessed = 0
dfTotalPixels = 55288763
#5 0x0804aca6 in main (argc=6, argv=0x80504a8) at gdalwarp.cpp:657
hSrcDS = 0x80509f8
hDstDS = 0x8056230
pszFormat = 0x804b6cc "GTiff"
pszTargetSRS = 0x0
pszSourceSRS = 0x0
pszSrcFilename = 0x804fdc0 "18left_gcp.tif"
pszDstFilename = 0x80504c8 "18left_warped_abcdefg.tif"
bCreateOutput = 1
i = 1
nOrder = -1
hTransformArg = (void *) 0x8056ef0
hGenImgProjArg = (void *) 0x80511d8
hApproxArg = (void *) 0x8056ef0
papszWarpOptions = (char **) 0x8050750
dfErrorThreshold = 0.125
dfWarpMemoryLimit = 0
pfnTransformer = 0x8049454 <GDALApproxTransform>
papszCreateOptions = (char **) 0x0
eOutputType = GDT_Unknown
eWorkingType = GDT_Unknown
eResampleAlg = GRA_NearestNeighbour
pszSrcNodata = 0x0
pszDstNodata = 0x0
bMulti = 0
psWO = (GDALWarpOptions *) 0x8057268
oWO = {_vptr.GDALWarpOperation = 0x403735d0, psOptions = 0x8056af0,
dfProgressBase = 0,
dfProgressScale = 1, hThread1Mutex = 0x0, hThread2Mutex = 0x0, hIOMutex = 0x0,
hWarpMutex = 0x0,
nChunkListCount = 1, nChunkListMax = 1, panChunkList = 0x8056f18,
bReportTimings = 0,
nLastTimeReported = 0}
(gdb) l
657 oWO.ChunkAndWarpImage( 0, 0,
658 GDALGetRasterXSize( hDstDS ),
659 GDALGetRasterYSize( hDstDS ) );
660 }
661
662 /* -------------------------------------------------------------------- */
663 /* Cleanup */
664 /* -------------------------------------------------------------------- */
665 if( hApproxArg != NULL )
666 GDALDestroyApproxTransformer( hApproxArg );
(gdb) frame 0
#0 0x401ca487 in GWKNearestNoMasksByte (poWK=0xbffff7d0) at gdalwarpkernel.cpp:2065
2065 poWK->papabyDstImage[iBand][iDstOffset] =
(gdb) l
2060
2061 iDstOffset = iDstX + iDstY * nDstXSize;
2062
2063 for( iBand = 0; iBand < poWK->nBands; iBand++ )
2064 {
2065 poWK->papabyDstImage[iBand][iDstOffset] =
2066 poWK->papabySrcImage[iBand][iSrcOffset];
2067 }
2068 }
2069
(gdb) frame 1
#1 0x401c695e in GDALWarpKernel::PerformWarp (this=0xbffff7d0) at
gdalwarpkernel.cpp:591
591 return GWKNearestNoMasksByte( this );
(gdb) l
586 && papanBandSrcValid == NULL
587 && panUnifiedSrcValid == NULL
588 && pafUnifiedSrcDensity == NULL
589 && panDstValid == NULL
590 && pafDstDensity == NULL )
591 return GWKNearestNoMasksByte( this );
592
593 if( eWorkingDataType == GDT_Byte
594 && eResample == GRA_Bilinear
595 && papanBandSrcValid == NULL
So I guess this segfault happens as iSrcOffset being used uninitialized
in gdalwarpkernel.cpp line 2066 ......
thanks,
Hamish
comment:2 by , 19 years ago
Hamish, On the "swirl" effects with -tps depending on the order of the GCPs, I am not sure what is going on. I have found the -tps transformation to be quite unstable in some situations to my great disappointment. I somewhat suspect the other problem is related to a bug in the gdal warper fixed after 1.2.6 release. Any chance you could retry it with the nightly snapshot of GDAL?
comment:3 by , 19 years ago
Hamish, I tried some of your examples, and they worked OK for me _if_ I did not try to compress the output file. Currently the GDAL/libtiff support for inplace update of compressed TIFF files is not working and you should not use it. You will need to gdalwarp to an uncompressed file, and then gdal_translate that to a compressed file.
comment:4 by , 19 years ago
Hi,
I don't think that disabling the compress= option is the whole story.
Did you get "8right" to work (with good output)? Doing as you suggest
makes it run to completion but it just came out as swirls.
e.g. I have tested with & without. Sometimes with the **same input file and
same command line** it worked or didn't.
synopsis: (all import maps are lzw compressed (I think))
18left -- same result with & without lzw
8left -- worked one way then stopped working that way.
8right -- got it to process by removing lzw and making name longer, but
result is swirled
Also I put up 18_R (18right) which works unswirled if the output filename is
20chars+. Smaller output filename leads to slower processing & result is
swirled.
details:
[18left] same result with & without lzw:
$ gdalwarp -tps -co compress=lzw 18left_gcp_1.tif 18left_warped_abcdefg3.tif
There is a problem to invert the interpolation matrix
Creating output file that is 4513P x 12251L.
There is a problem to invert the interpolation matrix
:0.Segmentation fault
$ gdalwarp -tps 18left_gcp_1.tif 18left_warped_abcdefg4.tif
There is a problem to invert the interpolation matrix
Creating output file that is 4513P x 12251L.
There is a problem to invert the interpolation matrix
:0.Segmentation fault
[8left] WTF?!
$ gdalwarp -tps -co compress=lzw 8left_gcp.tif 8left_warped_abcdefghi2.tif
Creating output file that is 12185P x 0L.
ERROR 1: _TIFFVSetField:8left_warped_abcdefghi2.tif: Bad value 0 for "RowsPerStrip"
Floating point exception
$ gdalwarp -tps 8left_gcp.tif 8left_warped_abcdefghi2.tif
Creating output file that is 9749P x 7476L.
:0...10...20...30...40...50...60...70...80...90...100 - done.
^^ worked!!
$ gdalwarp -tps 8left_gcp.tif 8left_warped_abcdefg.tif
Creating output file that is 0P x 0L.
ERROR 1: _TIFFVSetField:8left_warped_abcdefg.tif: Bad value 0 for "RowsPerStrip"
Floating point exception
^^ only output filename length changes!
$ gdalwarp -tps 8left_gcp.tif 8left_warped_abcdefghi2.tif
Creating output file that is 0P x 0L.
ERROR 1: _TIFFVSetField:8left_warped_abcdefghi2.tif: Bad value 0 for "RowsPerStrip"
Floating point exception
^^ only time changed! Same input and command line as working version!
[8right] breaks in 3 different ways depending on command line:
$ gdalwarp -tps -co compress=lzw 8right_gcp.tif 8right_warped_xyz123deY2.tif
Creating output file that is 11165P x 0L.
ERROR 1: _TIFFVSetField:8right_warped_xyz123deY2.tif: Bad value 0 for "RowsPerStrip"
Floating point exception
$ gdalwarp -tps 8right_gcp.tif 8right_warped_xyz123deZ.tif
Creating output file that is -2147483648P x -2147483648L.
ERROR 1: 8right_warped_xyz123deZ.tif:Integer overflow in TIFFScanlineSize
:0.ERROR 1: 8right_warped_xyz123deZ.tif:Integer overflow in TIFFScanlineSize
ERROR 1: 8right_warped_xyz123deZ.tif:Integer overflow in TIFFScanlineSize
ERROR 1: TIFFReadDirectory:8right_warped_xyz123deZ.tif: cannot handle zero
scanline size
$ gdalwarp -tps 8right_gcp.tif 8right_warped_xyz123defghijk.tif
Creating output file that is 9627P x 6717L.
:0...10...20...30...40...50...60...70...80...90...100 - done.
^^ completed, but output is swirled
comment:5 by , 19 years ago
[previous email added to the bug tracker]
re. swirling & segfaulting gdalwarp -tps:
Sometimes the program runs to completion, I hit the up arrow at the
console, change one letter in the output file name, and it fails. So me
thinks there's a variable somewhere being used uninitialized. So I try
running with electric-fence to detect that. Lo and behold it runs and
for the first time (for me) everything works properly & creates nice
output (without compress=lzw). ?!? I hesitate to believe that Debian
stable's glibc malloc is buggy ... No idea why it works ok when using
EF's. Maybe EF flushes all memory to 0 before allocating it or something?
Also works more reliably if you run it in gdb.
btw, Electric-fence catches this with compress=lzw turned on:
(exactly the same error with 8left.tif and 8right.tif)
$ ulimit -c unlimited
$ LD_PRELOAD=libefence.so.0.0 gdalwarp -tps -co compress=lzw \
-srcnodata 255 -dstnodata 255 8right_gcp.tif 8right_warped_xyy.tif
(EF core dumps when 0 bytes are attempted to be allocated)
$ gdb `which gdalwarp` ./core
[...]
1331 oWK.papabySrcImage[0] = (GByte *)
1332 VSIMalloc( nWordSize * nSrcXSize * nSrcYSize *
psOptions->nBandCount );
Summary: 0 bytes allocated as it thinks nSrcXSize = 0
at gdalwarpoperation.cpp:1331.
(AMD k6, hardware known to be 0.5% flakey)
Hamish
comment:6 by , 19 years ago
Here's a backtrace of that 'gdalwarp -co compress=lzw' Segfaulting:
$ LD_PRELOAD=libefence.so.0.0 gdalwarp -tps -co compress=lzw \
-srcnodata 255 -dstnodata 255 6left_warped2.tif_gcp 6left_warped_xyy.tif
Electric Fence 2.1 Copyright (C) 1987-1998 Bruce Perens.
Creating output file that is 13257P x 8609L.
:0...10...20...30...40...50...60...70..Segmentation fault (core dumped)
[...]
(gdb) bt full
#0 0x4072e56b in memset () from /lib/libc.so.6
No symbol table info available.
#1 0x405398ad in _TIFFmemset () from /usr/lib/libtiff.so.4
No symbol table info available.
#2 0x4052e833 in TIFFInitSGILog () from /usr/lib/libtiff.so.4
No symbol table info available.
#3 0x40537f85 in TIFFReadBufferSetup () from /usr/lib/libtiff.so.4
No symbol table info available.
#4 0x4053731c in TIFFReadEncodedStrip () from /usr/lib/libtiff.so.4
No symbol table info available.
#5 0x40101929 in GTiffRasterBand::IReadBlock (this=0x418e0f90, nBlockXOff=0,
nBlockYOff=0,
pImage=0x43e2bc34) at geotiff.cpp:530
poGDS = (GTiffDataset *) 0x418d6f08
nBlockBufSize = 13257
nBlockId = 0
nBlockIdBand0 = 0
eErr = CE_None
#6 0x401c3919 in GDALRasterBand::GetLockedBlockRef (this=0x418e0f90,
nXBlockOff=0, nYBlockOff=0,
bJustInitialize=0) at gdalrasterband.cpp:1194
poBlock = (class GDALRasterBlock *) 0x45512fcc
#7 0x401c7cd2 in GDALRasterBand::IRasterIO (this=0x418e0f90, eRWFlag=GF_Write,
nXOff=6628, nYOff=0,
nXSize=6629, nYSize=4304, pData=0x41a605f0, nBufXSize=6629, nBufYSize=4304,
eBufType=GDT_Byte,
nPixelSpace=1, nLineSpace=6629) at rasterio.cpp:184
bJustInitialize = 0
nSrcByteOffset = -1073744120
nBandDataSize = 1
nBufDataSize = 1
pabySrcBlock = (GByte *) 0x0
poBlock = (class GDALRasterBlock *) 0x0
nLBlockX = -1
nLBlockY = 0
iBufYOff = 0
iBufXOff = 1171103744
iSrcY = 0
iSrcX = 0
dfSrcX = 1.8960908253239957e+28
dfSrcY = 2.3574204445744105
dfSrcXInc = 1.0000002384615696
dfSrcYInc = 52.730308532697961
#8 0x401b4430 in GDALDataset::IRasterIO (this=0x418d6f08, eRWFlag=GF_Write,
nXOff=6628, nYOff=0,
nXSize=6629, nYSize=4304, pData=0x41a605f0, nBufXSize=6629, nBufYSize=4304,
eBufType=GDT_Byte,
nBandCount=1, panBandMap=0x419a2ffc, nPixelSpace=1, nLineSpace=6629,
nBandSpace=28531216)
at gdaldataset.cpp:1251
poBand = (class GDALRasterBand *) 0x418e0f90
pabyBandData = (
GByte *) 0x41a605f0 "&\" *!)$'-1$&,(\036 '\036 %$%)*')
,',03$#!%$$)%'$($-#&$#%'!\034\034\"% $#'%)#*(&&\035 \021\035\031\035\036# &/')#$
#\"\037\024#0\017\030!$)-\"$&.#\" &\027\033\034\036!*##(\032%'(+
''ÿÿÔÍÌØÝãÞÛàÞßãâáäãâßäÝäßàååæååæáåæãÞéæããâáäããáâÞäãáèáâäæÞââÝââäæäåæåïåäæϾÞÚ\214Nàá"...
iBandIndex = 0
eErr = CE_None
#9 0x401b480c in GDALDataset::RasterIO (this=0x418d6f08, eRWFlag=GF_Write,
nXOff=6628, nYOff=0,
nXSize=6629, nYSize=4304, pData=0x41a605f0, nBufXSize=6629, nBufYSize=4304,
eBufType=GDT_Byte,
nBandCount=1, panBandMap=0x419a2ffc, nPixelSpace=1, nLineSpace=6629,
nBandSpace=28531216)
at gdaldataset.cpp:1442
i = 1
bNeedToFreeBandMap = 0
eErr = CE_None
#10 0x401b48b1 in GDALDatasetRasterIO (hDS=0x418d6f08, eRWFlag=GF_Write,
nXOff=6628, nYOff=0,
nXSize=6629, nYSize=4304, pData=0x41a605f0, nBufXSize=6629, nBufYSize=4304,
eBufType=GDT_Byte,
nBandCount=1, panBandMap=0x419a2ffc, nPixelSpace=0, nLineSpace=0, nBandSpace=0)
at gdaldataset.cpp:1477
poDS = (GDALDataset *) 0x418d6f08
#11 0x401eb57a in GDALWarpOperation::WarpRegion (this=0xbffff960, nDstXOff=6628,
nDstYOff=0,
nDstXSize=6629, nDstYSize=4304, nSrcXOff=0, nSrcYOff=7748, nSrcXSize=1129,
nSrcYSize=7602)
at gdalwarpoperation.cpp:1190
eErr = CE_None
iBand = 1
pDstBuffer = (void *) 0x41a605f0
nWordSize = 1
nBandSize = 28531216
pszInitDest = 0x4199eff6 "NO_DATA"
#12 0x401ea715 in GDALWarpOperation::ChunkAndWarpImage (this=0xbffff960,
nDstXOff=0, nDstYOff=0,
nDstXSize=13257, nDstYSize=8609) at gdalwarpoperation.cpp:683
panThisChunk = (int *) 0x419acf60
dfChunkPixels = 28531216
eErr = CE_None
iChunk = 2
dfPixelsProcessed = 57060452
dfTotalPixels = 114129513
#13 0x0804aca6 in main (argc=10, argv=0x40d2cfd4) at gdalwarp.cpp:657
hSrcDS = 0x40d3cf08
hDstDS = 0x418d6f08
pszFormat = 0x804b6cc "GTiff"
pszTargetSRS = 0x0
pszSourceSRS = 0x0
pszSrcFilename = 0x40d28fe8 "6left_warped2.tif_gcp"
pszDstFilename = 0x40d2afe8 "6left_warped_xyy.tif"
bCreateOutput = 1
i = 1
nOrder = -1
hTransformArg = (void *) 0x41982fdc
hGenImgProjArg = (void *) 0x418e6f18
hApproxArg = (void *) 0x41982fdc
papszWarpOptions = (char **) 0x418e8ff8
dfErrorThreshold = 0.125
dfWarpMemoryLimit = 0
pfnTransformer = 0x8049454 <GDALApproxTransform>
papszCreateOptions = (char **) 0x0
eOutputType = GDT_Unknown
eWorkingType = GDT_Unknown
eResampleAlg = GRA_NearestNeighbour
pszSrcNodata = 0x40d22ffc "255"
pszDstNodata = 0x40d26ffc "255"
bMulti = 0
psWO = (GDALWarpOptions *) 0x40d32f78
oWO = {_vptr.GDALWarpOperation = 0x40395510, psOptions = 0x41996f78,
dfProgressBase = 0.49996228407633703, dfProgressScale = 0.24998981639394185,
hThread1Mutex = 0x0,
hThread2Mutex = 0x0, hIOMutex = 0x0, hWarpMutex = 0x0, nChunkListCount = 4,
nChunkListMax = 7,
panChunkList = 0x419acf20, bReportTimings = 0, nLastTimeReported = 0}
(gdb) frame 5
#5 0x40101929 in GTiffRasterBand::IReadBlock (this=0x418e0f90, nBlockXOff=0,
nBlockYOff=0,
pImage=0x43e2bc34) at geotiff.cpp:530
530 if( TIFFReadEncodedStrip( poGDS->hTIFF, nBlockId, pImage,
(gdb) l
525 eErr = CE_Failure;
526 }
527 }
528 else
529 {
530 if( TIFFReadEncodedStrip( poGDS->hTIFF, nBlockId, pImage,
531 nBlockBufSize ) == -1 )
532 {
533 memset( pImage, 0, nBlockBufSize );
534 CPLError( CE_Failure, CPLE_AppDefined,
comment:7 by , 19 years ago
Output to compressed TIFF files with random access applications like gdalwarp is not currently supported, due to known bugs. Best not to complicate this bug report with stuff related to random output to compressed TIFF issues.
comment:8 by , 19 years ago
Ok, will keep it to the main issue that doesn't have a work around. I put up that last one as I got the same exact segfault in exactly the same line number and variable with a couple of input different images. Avoiding compression applies only to the gdalwarp command line, not the input image, correct? Generally running 'gdalwarp -tps' without compress=lzw and within electric-fence works just beautifully -- most of the time. Which makes it all the more frustrating when it doesn't. I have been able to get through quite a bit of my data so far using the above method.. and again the results are better than we had hoped for. Really puts the previous attempts with ERDAS to shame. My latest problem is two similar tracks of data (6left & 6right) going haywire in another form of the swirling image bug. These two produce slanting line output, as the attached image shows. I've put the input 6left_gcp.tif image up on my website if you want to have a try yourself. I saw this before and was able to fix it by disabling a wayward GCP. This time I don't see any points in the wrong place and turning off the 3 points with the greatest RMS error (from GRASS's i.points analysis) didn't help. (they weren't that bad..) command line for that image looked like: gdalwarp -tps -srcnodata 255 -dstnodata 255 6left_gcp.tif 6left_warped.tif thanks, Hamish
comment:9 by , 19 years ago
Here is another case where I can get a crash as opposed to bad ouput.
gdalwarp.cpp line 657:
oWO.ChunkAndWarpImage( 0, 0,
GDALGetRasterXSize( hDstDS ),
GDALGetRasterYSize( hDstDS ) );
feeds in bogus values for nDstXSize and nDstYSize, so it fails.
$ LD_PRELOAD=libefence.so.0.0 gdalwarp -tps -srcnodata 255 -dstnodata 255 \
22right_gcp.tif 22right_warped_warped_warp.tif
Electric Fence 2.1 Copyright (C) 1987-1998 Bruce Perens.
Creating output file that is -2147483648P x -2147483648L.
ERROR 1: 22right_warped_warped_warp.tif:Integer overflow in TIFFScanlineSize
ElectricFence Aborting: Allocating 0 bytes, probably a bug.
Illegal instruction
I have put 22right_gcp.tif up on my website. Let me know if you no longer have
the URL.
The other day I did the same but got a memory error when it tried to allocate 0
bytes for a supposed 0x0 output map:
(using a diff't but very similar input file)
$ LD_PRELOAD=libefence.so.0.0 gdalwarp -tps -srcnodata 255 -dstnodata 255
22left_warped.tif_gcp 22l.tif
Electric Fence 2.1 Copyright (C) 1987-1998 Bruce Perens.
Creating output file that is 0P x 0L.
ERROR 1: _TIFFVSetField:22l.tif: Bad value 0 for "RowsPerStrip"
Floating point exception
$ LD_PRELOAD=libefence.so.0.0 gdalwarp -tps -srcnodata 255 -dstnodata 255
22left_warped.tif_gcp 22l_tps.tif
Electric Fence 2.1 Copyright (C) 1987-1998 Bruce Perens.
:0...11...21...30...40...50...61...71...80...90...100 - done.
^^^^ 11% done?!
--only difference there is the output file name.
(ignore the input file name, it's just a temporary file in my script: raw image
with GCPs applied by gdal_translate)
Removing the 8 byte output file and running the same command again on the
command line (up arrow so no typos) it can fail with 0Px0L or -2147483648Px...L
for the output image size.
If I don't use Electric Fence it runs but the output comes out as swirls.
gdal compiled with -ggdb so untouched variables should be initialized to 0.
'gdalwarp -order 2' looks ok, but isn't appropriate for my data.
(these are long & thin sidescan sonar tracks)
Here is a backtrace on the core dump from the above (top-most) 22right run.
(gdb) bt full
#0 0x406e07c1 in kill () from /lib/libc.so.6
No symbol table info available.
#1 0x4002de86 in EF_Abort () from /usr/lib/libefence.so.0.0
No symbol table info available.
#2 0x4002d3ca in memalign () from /usr/lib/libefence.so.0.0
No symbol table info available.
#3 0x4002da3e in malloc () from /usr/lib/libefence.so.0.0
No symbol table info available.
#4 0x401d4d81 in VSIMalloc (nSize=0) at cpl_vsisimple.cpp:322
No locals.
#5 0x401eb0d1 in GDALWarpOperation::WarpRegion (this=0xbffff980, nDstXOff=0,
nDstYOff=0, nDstXSize=-2147483648,
nDstYSize=-2147483648, nSrcXOff=0, nSrcYOff=0, nSrcXSize=0, nSrcYSize=0) at
gdalwarpoperation.cpp:1085
eErr = CE_None
iBand = -1073743488
pDstBuffer = (void *) 0x4000bcd0
nWordSize = 1
nBandSize = 0
pszInitDest = 0x40016540 "Øn\001@\034"
#6 0x401ea715 in GDALWarpOperation::ChunkAndWarpImage (this=0xbffff980,
nDstXOff=0, nDstYOff=0, nDstXSize=-2147483648,
nDstYSize=-2147483648) at gdalwarpoperation.cpp:683
panThisChunk = (int *) 0x414b0fe0
dfChunkPixels = 4.6116860184273879e+18
eErr = 134532600
iChunk = 0
dfPixelsProcessed = 0
dfTotalPixels = 4.6116860184273879e+18
#7 0x0804aca6 in main (argc=8, argv=0x40d28fdc) at gdalwarp.cpp:657
hSrcDS = 0x40d34f08
hDstDS = 0x41418f08
pszFormat = 0x804b6cc "GTiff"
pszTargetSRS = 0x0
pszSourceSRS = 0x0
pszSrcFilename = 0x40d24ff0 "22right_gcp.tif"
pszDstFilename = 0x40d26fe0 "22right_warped_warped_warp.tif"
bCreateOutput = 1
i = 1
nOrder = -1
hTransformArg = (void *) 0x40d2afdc
hGenImgProjArg = (void *) 0x41426f18
hApproxArg = (void *) 0x40d2afdc
papszWarpOptions = (char **) 0x41428ff8
dfErrorThreshold = 0.125
dfWarpMemoryLimit = 0
pfnTransformer = 0x8049454 <GDALApproxTransform>
papszCreateOptions = (char **) 0x0
eOutputType = GDT_Unknown
eWorkingType = GDT_Unknown
eResampleAlg = GRA_NearestNeighbour
pszSrcNodata = 0x40d1effc "255"
pszDstNodata = 0x40d22ffc "255"
bMulti = 0
psWO = (GDALWarpOptions *) 0x40d2cf78
oWO = {_vptr.GDALWarpOperation = 0x40395510, psOptions = 0x4149af78,
dfProgressBase = 0, dfProgressScale = 1,
hThread1Mutex = 0x0, hThread2Mutex = 0x0, hIOMutex = 0x0, hWarpMutex = 0x0,
nChunkListCount = 1, nChunkListMax = 1,
panChunkList = 0x414b0fe0, bReportTimings = 0, nLastTimeReported = 0}
(gdb) frame 5
#5 0x401eb0d1 in GDALWarpOperation::WarpRegion (this=0xbffff980, nDstXOff=0,
nDstYOff=0, nDstXSize=-2147483648,
nDstYSize=-2147483648, nSrcXOff=0, nSrcYOff=0, nSrcXSize=0, nSrcYSize=0) at
gdalwarpoperation.cpp:1085
1085 pDstBuffer = VSIMalloc( nBandSize * psOptions->nBandCount );
(gdb) l
1080 /* -------------------------------------------------------------------- */
1081 void *pDstBuffer;
1082 int nWordSize = GDALGetDataTypeSize(psOptions->eWorkingDataType)/8;
1083 int nBandSize = nWordSize * nDstXSize * nDstYSize;
1084
1085 pDstBuffer = VSIMalloc( nBandSize * psOptions->nBandCount );
1086 if( pDstBuffer == NULL )
1087 {
1088 CPLError( CE_Failure, CPLE_OutOfMemory,
1089 "Out of memory allocatint %d byte destination buffer.",
?
Hamish
comment:10 by , 19 years ago
So I tried running through kdbg to get a better handle on why for this image it
thought the output would be 0P x 0L. Now it doesn't write out the "Creating
output file that is 0P x 0L." at all! And the percent done is still funny,
although my guess is that it is process so few lines that it jumps from 9% to
11% done, so 10% done is not reported.
Also running through kdbg I got it to create a 8217P x 36L output file which
comes out as a wide strip (apparently the correct width) with a miniature
version of the warped image in the middle of it scaled to 36 pixel height.
Correct angles, everything, just really small. Result should be roughly square.
$ LD_PRELOAD=libefence.so.0.0 gdalwarp -tps -srcnodata 255 -dstnodata 255 \
22right_gcp.tif 22right_warped_warped_warp.tif
Electric Fence 2.1 Copyright (C) 1987-1998 Bruce Perens.
:0...11...22...30...41...50...61...72...80...91...100 - done.
$ gdalinfo 22right_warped_warped_warp.tif
Driver: GTiff/GeoTIFF
Size is 8217, 36
Coordinate System is `'
Origin = (2128719.429993,5413622.181495)
Pixel Size = (26.24841097,-26.24841097)
Corner Coordinates:
Upper Left ( 2128719.430, 5413622.181)
Lower Left ( 2128719.430, 5412677.239)
Upper Right ( 2344402.623, 5413622.181)
Lower Right ( 2344402.623, 5412677.239)
Center ( 2236561.026, 5413149.710)
Band 1 Block=8217x1 Type=Byte, ColorInterp=Gray
NoData Value=255
Hamish
comment:11 by , 19 years ago
In the above case "Creating output file that is 8217P x 36L." is not displayed as I didn't delete the output file before re-running (figured it would just blindly overwrite), so this test gets it wrong on gdalwarp.cpp line 417: /* Does the output dataset already exist? (as bCreateOutput == 0) and I never get the "Output dataset %s exists,\n" error. If I add "-of GTiff" to the command line bCreateOutput does get set to TRUE and I do get the error. Hamish
comment:12 by , 19 years ago
After another run it decided to get the size right (AFAICT), but the result is
swirled.
$ gdalinfo 22right_warped_warped_warp.tif
Driver: GTiff/GeoTIFF
Size is 5196, 3249
Coordinate System is `'
Origin = (2264630.752256,5413622.181495)
Pixel Size = (0.28892316,-0.28892316)
Corner Coordinates:
Upper Left ( 2264630.752, 5413622.181)
Lower Left ( 2264630.752, 5412683.470)
Upper Right ( 2266131.997, 5413622.181)
Lower Right ( 2266131.997, 5412683.470)
Center ( 2265381.375, 5413152.826)
Band 1 Block=5196x1 Type=Byte, ColorInterp=Gray
NoData Value=255
so wrong output size isolated to GDALSuggestedWarpOutput() call..
/* -------------------------------------------------------------------- */
/* Get approximate output definition. */
/* -------------------------------------------------------------------- */
if( GDALSuggestedWarpOutput( hSrcDS,
GDALGenImgProjTransform, hTransformArg,
adfDstGeoTransform, &nPixels, &nLines )
!= CE_None )
return NULL;
re-run a second time with the same correct size but swirled result.
re-run from the command line (using Electric fence for mallocs, this time with
-of GTiff) it reports a 0P x 0L output file and then gets a floating point
exception.
"Creating output file that is 0P x 0L.
ERROR 1: _TIFFVSetField:22right_warped_warped_warp.tif: Bad value 0 for
"RowsPerStrip"
Floating point exception"
same re-run from the command line & same error but this time because
"Creating output file that is -2147483648P x -2147483648L.
ERROR 1: 22right_warped_warped_warp.tif:Integer overflow in TIFFScanlineSize
ElectricFence Aborting: Allocating 0 bytes, probably a bug.
Illegal instruction"
re-run in Kdbg & I get the 5196P x 3249L swirled output.
re-run from the command line with EF & I get 0P x 0L error.
up arrow on the command line & repeated -2147483648P x -2147483648L errors.
So I'm still thinking something is reading from an uninitialized variable..
Hamish
comment:13 by , 19 years ago
... and if I run from the command line without electric fence it comes out as the same 5196P x 3249L swirled output as kdbg gives. (kdbg isn't using EF) Hamish
comment:14 by , 19 years ago
I tried using the final dimensions from the swirled output Tiff with the -ts
command line option to get around the 0x0 target size problem. No luck. Src size
isn't getting there either, and I can't override that.
$ LD_PRELOAD=libefence.so.0.0 gdalwarp -tps -of GTiff -srcnodata 255 \
-dstnodata 255 -ts 5196 3249 22right_gcp.tif 22right_warped_warped_warpT.tif
Electric Fence 2.1 Copyright (C) 1987-1998 Bruce Perens.
Creating output file that is 5196P x 3249L.
ElectricFence Aborting: Allocating 0 bytes, probably a bug.
Illegal instruction (core dumped)
(because nSrcXSize and nSrcYSize are 0)
gdb:
#5 0x401eb7f0 in GDALWarpOperation::WarpRegionToBuffer (this=0xbffff940,
nDstXOff=0, nDstYOff=0,
nDstXSize=5196, nDstYSize=3249, pDataBuf=0x414e2774, eBufDataType=GDT_Byte,
nSrcXOff=0,
nSrcYOff=0, nSrcXSize=0, nSrcYSize=0) at gdalwarpoperation.cpp:1331
1331 oWK.papabySrcImage[0] = (GByte *)
(gdb) l
1326 oWK.nSrcXSize = nSrcXSize;
1327 oWK.nSrcYSize = nSrcYSize;
1328
1329 oWK.papabySrcImage = (GByte **)
1330 CPLCalloc(sizeof(GByte*),psOptions->nBandCount);
1331 oWK.papabySrcImage[0] = (GByte *)
1332 VSIMalloc( nWordSize * nSrcXSize * nSrcYSize *
psOptions->nBandCount );
1333
1334 if( oWK.papabySrcImage[0] == NULL )
1335 {
Adding some printfs, I see that panThisChunk[0-7] looks like this:
0 0 5196 3249 0 0 [0 0]
for the gdalwarpoperation.cpp line ~685 WarpRegion() call.
Shouldn't those last two match the dimensions of the input image? (1200x5720)
(2,3 filled by -ts command)
I notice in alg/gdalwarper.h that the WarpRegion() definition sets nSrc* to 0.
It won't compile without those being set & I don't know much C++, but don't
think that is what is doing the resetting, probably panThisChunk isn't getting
filled correctly? I tried hardcoding in my Src x,y into [7] and [8], but that
segfaulted badlyafter ":0..". Guess I don't understand what that is doing fully.
As you may notice I'm somewhat shooting in the dark here...
Hamish
comment:15 by , 18 years ago
Hi, problem persists if input map is compressed with lzw. Note input was created with i.warp61 shell script: http://grass.gdf-hannover.de/wiki/GRASS_AddOns#Imagery_add-ons gdal_translate -co compress=lzw $GDAL_GCP "$INFILE" "$TMP".tif gdalwarp -tps -co compress=lzw "$TMP".tif "$OUTFILE" (GCPs are parsed from a GRASS i.points session POINTS file) removing "-co compress=lzw" from the gdal_translate line (gdalwarp input map) makes it succeed for the two maps I tried it with today. (as suggested in FW comment 2005-06-11) I have left _warp_warp_warp_warp in the output filename, for some reason that helps the LZW case work ~50% of the time. Hamish
comment:16 by , 18 years ago
adding "-rcs" made 2 out of 8 text images produce bad results. (e.g. output width=2) (uncompressed input image) Hamish
comment:17 by , 17 years ago
| Summary: | overflow → gdalwarp: overflow from dstfile name |
|---|
follow-up: 19 comment:18 by , 16 years ago
| Keywords: | tps added |
|---|
Some of the problems exposed in this bug report are probably related to a bug with the -tps transformer which has just been fixed with #2300. The fix is available in trunk, branches/1.5 and branches/1.4.
Hamish, if you have the opportunity of building GDAL from SVN or can wait for a next GDAL release, could you see if that fixes your problem ?
comment:19 by , 16 years ago
Replying to rouault:
Hamish, if you have the opportunity of building GDAL from SVN or can wait for a next GDAL release, could you see if that fixes your problem ?
Ok, I will test and report back.
thanks, Hamish
comment:20 by , 15 years ago
| Resolution: | → fixed |
|---|---|
| Status: | assigned → closed |
Hi,
I can confirm that after re-testing the above images that things now work properly using the latest GDAL SVN/trunk, and that gdalwarp from 1.5.0 segfaults.
It was a somewhat intermittent thing, so I can't say for sure, but the segfaults have now abated to unmeasurable levels. :)
Closing bug in the belief that init'ng rhs[] to 0 as discussed in ticket #2300 fixed it.
thanks, Hamish
Note:
See TracTickets
for help on using tickets.
example of gdalwarp -tps gone odd