Opened 21 years ago

Closed 14 years ago

#403 closed defect (invalid)

DumpModeDecode: Not enough data for scanline 0

Reported by: matt.wilkie@… Owned by: warmerdam
Priority: normal Milestone:
Component: default Version: unspecified
Severity: normal Keywords:
Cc: Jeff McKenna, hobu, Mateusz Łoskot

Description (last modified by Mateusz Łoskot)

when translating a greyscale tiff I get the message:

gdal_translate -of png cowley.tif test.png
Input file size is 10060, 14800
ERROR 1: cowley.tif:DumpModeDecode: Not enough data for scanline 0
ERROR 1: TIFFReadEncodedTile() failed.

ERROR 1: IReadBlock failed at X offset 0, Y offset 115
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 115
ERROR 1: cowley.tif:DumpModeDecode: Not enough data for scanline 0
ERROR 1: TIFFReadEncodedTile() failed.
...

The output format does not seem to matter. The output image is created but the bottom strip is streaked. I can send a screen shot if you wish. The source image does not have the streaks. I'm using OpenEV for viewing.

> gdalinfo cowley.tif       # <-- input image
Driver: GTiff/GeoTIFF
Size is 10060, 14800
Coordinate System is `'
Origin = (503830.589326,6711515.646365)
Pixel Size = (0.489962,-0.489962)
Corner Coordinates:
Upper Left  (  503830.589, 6711515.646)
Lower Left  (  505305.912, 6704264.204)
Upper Right (  508759.611, 6712518.467)
Lower Right (  510234.933, 6705267.024)
Center      (  507032.761, 6708391.335)
Band 1 Block=128x128 Type=Byte, ColorInterp=Palette
  Overviews: 106x155
  Color Table (RGB with 256 entries)
    0: 0,0,0,255
    1: 1,1,1,255
...blah blah blah...

>gdalinfo test.jp2      # <-- output image
Driver: JP2KAK/JPEG-2000 (based on Kakadu)
Size is 10060, 14800
Coordinate System is `'
Origin = (503830.589326,6711515.646365)
Pixel Size = (0.489962,-0.489962)
Corner Coordinates:
Upper Left  (  503830.589, 6711515.646)
Lower Left  (  505305.912, 6704264.204)
Upper Right (  508759.611, 6712518.467)
Lower Right (  510234.933, 6705267.024)
Center      (  507032.761, 6708391.335)
Band 1 Block=512x128 Type=Byte, ColorInterp=Palette
  Overviews: 5030x7400, 2515x3700, 1258x1850, 629x925, 315x463
  Color Table (RGB with 256 entries)
    0: 0,0,0,255
    1: 1,1,1,255
    2: 2,2,2,255
...blah blah blah...

All programs are from openev_fw_162.zip

OS is WindowsXP (not avail. in the bug form)

Attachments (4)

scanline0.png (209.1 KB ) - added by maphew 17 years ago.
4% of original size, viewable in openev only, can't be translated to gtiff
gimp_flat_tiny.tif (166.7 KB ) - added by maphew 17 years ago.
gimp save as flat tif, then gdal_translate -outsize 3% 3%…
from_gimp_tiny.xcf (132.9 KB ) - added by maphew 17 years ago.
gimp open original.tif, save as 2 layer .xcf, scale to 3%, save as tiny 2 layer .xcf
tail.zip (233.9 KB ) - added by maphew 17 years ago.
results of "tail -c 340787 badimg.tif > tail"

Download all attachments as: .zip

Change History (27)

comment:1 by jmckenna@…, 18 years ago

i just hit this same error trying to add overviews to a geotiff.

comment:2 by jmckenna@…, 18 years ago

but since this bug is so old i'm assuming i'm doing something wrong (?)

comment:3 by jmckenna@…, 18 years ago

$ gdaladdo -r nearest etopo2-bathy.tif 2 4 8

   ERROR 1: etopo2-bathy.tif:DumpModeDecode: Not enough data for scanline 0
   ERROR 1: TIFFReadEncodedTile() failed.
   ERROR 1: etopo2-bathy.tif:DumpModeDecode: Not enough data for scanline 128
   ERROR 1: TIFFReadEncodedTile() failed.
   ERROR 1: etopo2-bathy.tif:DumpModeDecode: Not enough data for scanline 256

$ gdalinfo etopo2-bathy.tif

   Driver: GTiff/GeoTIFF
   Size is 10800, 5400
   Coordinate System is `'
   Origin = (-180.000000,90.000000)
   Pixel Size = (0.03333333,-0.03333333)
   Metadata:
     TIFFTAG_XRESOLUTION=1200
     TIFFTAG_YRESOLUTION=1200
     TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch)
   Corner Coordinates:
     Upper Left  (-180.0000000,  90.0000000)
     Lower Left  (-180.0000000, -90.0000000)
     Upper Right ( 180.0000000,  90.0000000)
     Lower Right ( 180.0000000, -90.0000000)
   Center      (   0.0000000,   0.0000000)
   Band 1 Block=10800x1 Type=Byte, ColorInterp=Red
   Band 2 Block=10800x1 Type=Byte, ColorInterp=Green
   Band 3 Block=10800x1 Type=Byte, ColorInterp=Blue

comment:4 by warmerdam, 18 years ago

Matt / Jeff, 

I would really need to examine the input files to have any idea what is
going on.  I *suspect* the files have null tiles or strips.  I think I 
made a correction in GDAL to support these null tiles/strips properly
but I'm not sure. 

Note, gdal_merge used to (and may still) produce null tiles or strips for
areas with no data. 

comment:6 by maphew, 17 years ago

Priority: highnormal

I still get this error with a current gdal_translate, and some new symptoms (which may or not have been present 3 years ago):

  • if the output format is gtiff, the output file does not get written.
  • if output is png, out file is written and can be viewed with OpenEV,

but xnview reports "error reading test.png"

  • a further translate of the output png fails with a different message:
    gdal_translate scanline0.png test.tif
    Input file size is 10060, 14800
    0...10...20...30...40...50...60...70...80...90...ERROR 1: 
    libpng: Not enough image data
    ERROR 1: IReadBlock failed at X offset 0, Y offset 14717
    ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 14717
    

The source image is 150mb. Output png is 128mb. I can make them freely available to anyone who wants to look at this more deeply. The attached image was generated via:

gdal_translate -of png cowley.tif scanline0.png
gdal_translate -of png -outsize 10% 10% scanline0.png scanline0_small.png

by maphew, 17 years ago

Attachment: scanline0.png added

4% of original size, viewable in openev only, can't be translated to gtiff

comment:7 by maphew, 17 years ago

curiously, the image itself can't be viewed in xnview or windows explorer, but in Thumbnail Mode, the image *is* shown, although there a strip at the bottom missing.

comment:8 by warmerdam, 17 years ago

Description: modified (diff)

I have run into this with other TIFF files, and it turns out they actually write out the last strip incomplete. So the "not enough data" message is accurate.

I can't really think of a good fix for this in GDAL. In theory libtiff could be modified to allow this, but I'm not sure that is a good idea.

It would be helpful if someone could offer a small sample file for experimenting with.

comment:9 by maphew, 17 years ago

so the attached png can't be used to debug this? I'll try use a utility other than gdal to resize the problem image, hopefully the problem part is copied over <x-fingers>

comment:10 by warmerdam, 17 years ago

Matt,

My understanding is that the png file is the result of a partially failed translation from the corrupt tiff to png, isn't that right? It doesn't help us address the issue with the tiff driver. If I have misunderstood, please let me know.

comment:11 by maphew, 17 years ago

yes the png is an incomplete copy of the original, but most of it is still there. I'd much rather have >90% than 0%.

I find it curious that openev can view the broken png, and gdal_translate can copy it to select other formats (so far: jp2kak, ecw, png, gsag). I suppose this might more properly be filed as a separate bug.

when I open problem.tif in Gimp there is the same scanline error, only repeated in increments (scanline 0,128,256,512...). Gimp also says there are two layers: Background which is 10060x1480 pixels and Page 1 which is only 106x155. When I save a copy gimp tells me tiff can't handle layers and flattens it. There are no errors gdal_translating flattened.tif, though of course there is no georeferencing.

There is a scrambled bit at the bottom of flat.tif, also visible in the original. Openev does not show the scrambled bit on the original. I can't tell if it is drawing the strip properly or quietly omitting it.

by maphew, 17 years ago

Attachment: gimp_flat_tiny.tif added

gimp save as flat tif, then gdal_translate -outsize 3% 3%...

by maphew, 17 years ago

Attachment: from_gimp_tiny.xcf added

gimp open original.tif, save as 2 layer .xcf, scale to 3%, save as tiny 2 layer .xcf

comment:12 by maphew, 17 years ago

I dunno if anything about the original can be determined from this gimp .xcf file, but here is is just in case...

comment:13 by warmerdam, 17 years ago

Matt,

I have downloaded the gimp_flat_tiny.tif, but I don't see any dumpmode errors when I work with it. Do you? If this is just intended to be another picture of the resulting image corruption, I don't need that. I just need a smallish TIFF file that produced the error message!

comment:14 by maphew, 17 years ago

no I don't get any errors with gimp_flat_tiny.tif. Gimp does something in the flatten process that fixes it, well fixes the error but adds the scrambled strip at the bottom.

I can't figure out how to reduce the size of the source tif. All of the programs I have either don't read it or change it (gdal_translate, openev, xnview, nconvert, gimp, arcmap).

The scanline error happens right at the end, ...90...[!here!], if I just cut the tail off, the last x bytes, would that help?

by maphew, 17 years ago

Attachment: tail.zip added

results of "tail -c 340787 badimg.tif > tail"

comment:15 by warmerdam, 17 years ago

Matt,

No, I don't believe there is anything that can be done short of going back to whatever program is generating these corrupt files, and producing a small one.

comment:16 by Jeff McKenna, 17 years ago

Frank I am again hitting something similar with a geotiff when trying to gdaladdo:

ERROR 1: IReadBlock failed at X offset 0, Y offset 37 ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 37 ERROR 1: elu90m_land.tif:DumpModeDecode: Not enough data for scanline 0 ERROR 1: TIFFReadEncodedTile() failed.

This is with GDAL 1.4.2

It is a large file (140 MB), but let me know and I can post it somewhere for you or make it smaller.

comment:17 by maphew, 17 years ago

Jeff" thanks for reminding me about this bug.

The cowley.tif image where I encountered this can be reached at ftp://ftp.geomaticsyukon.ca/Users/matt/NotEnoughDataForScanline0/

comment:18 by dfuhry, 17 years ago

I can provide plenty of test images which demonstrate this bug. One is: http://kddb1.kddb.cs.kent.edu/gdal/n2270610.tif

$ gdaladdo n2270610.tif 2 2>err.log 0...10...20...30...40...50...60...70...80...90...100 - done.

But the stderr output in err.log reads:
Warning 1: TIFFReadDirectory:n2270610.tif: Wrong "StripByteCounts" field, ignoring and calculating from imagelength
Warning 1: TIFFReadDirectory:n2270610.tif: Wrong "StripByteCounts" field, ignoring and calculating from imagelength
ERROR 1: n2270610.tif:DumpModeDecode: Not enough data for scanline 0
ERROR 1: TIFFReadEncodedTile() failed.
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
ERROR 1: n2270610.tif:DumpModeDecode: Not enough data for scanline 128
ERROR 1: TIFFReadEncodedTile() failed.
ERROR 1: IReadBlock failed at X offset 1, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 1, Y block offset 0
ERROR 1: n2270610.tif:DumpModeDecode: Not enough data for scanline 256
(and so on for scanlines 384,512,640,...,1920)
Warning 1: TIFFReadDirectory:n2270610.tif: Wrong "StripByteCounts" field, ignoring and calculating from imagelength
ERROR 1: n2270610.tif:DumpModeDecode: Not enough data for scanline 2048 (and so on for scanline numbers in increments 128)

Image metadata (sorry, trac mangles the output a bit):

$ gdalinfo n2270610.tif
Warning 1: TIFFReadDirectory:n2270610.tif: Wrong "StripByteCounts" field, ignoring and calculating from imagelength
Driver: GTiff/GeoTIFF
Size is 5000, 5000
Coordinate System is `'
Origin = (2270000.000000000000000,615000.000000000000000)
Pixel Size = (1.000000000000000,-1.000000000000000)
Metadata:

AREA_OR_POINT=Area TIFFTAG_SOFTWARE=IMAGINE TIFF Support

Copyright 1991 - 1999 by ERDAS, Inc. All Rights Reserved @(#)$RCSfile: etif.c $ $Revision: 1.9.1.3 $ $Date: 2002/07/29 15:39:06EDT $

TIFFTAG_XRESOLUTION=1 TIFFTAG_YRESOLUTION=1 TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch)

Corner Coordinates: Upper Left ( 2270000.000, 615000.000) Lower Left ( 2270000.000, 610000.000) Upper Right ( 2275000.000, 615000.000) Lower Right ( 2275000.000, 610000.000) Center ( 2272500.000, 612500.000) Band 1 Block=5000x1 Type=Byte, ColorInterp=Red

Overviews: 2500x2500

Band 2 Block=5000x1 Type=Byte, ColorInterp=Green

Overviews: 2500x2500

Band 3 Block=5000x1 Type=Byte, ColorInterp=Blue

Overviews: 2500x2500

It does put some kind of invalid overview layer; if I tiffsplit the file and try to view xaab.tif in any typical image viewer (which presumably uses libtiff), I get the output:

TIFF image: DumpModeDecode: Not enough data for scanline 0
TIFF image: DumpModeDecode: Not enough data for scanline 128
...
TIFF image: DumpModeDecode: Not enough data for scanline 2432
TIFF image: DumpModeDecode: Not enough data for scanline 0
TIFF image: DumpModeDecode: Not enough data for scanline 128
...

I'm using

$ gdaladdo --version
GDAL 1.4.1.0, released 2007/04/09

$ tiffinfo 2>&1 | grep Version
LIBTIFF, Version 3.8.2

on Debian.

comment:19 by dfuhry, 17 years ago

Scratch (most of) my previous comments - gdaladdo 1.4.1.0 only gives those errors if operating on a TIFF which had been previously processed with gdaladdo 1.3.x

Starting over with the original (unoverviewed) TIFF with ONLY gdaladdo 1.4.1.0, the process works fine.

Sorry for the noise.

comment:20 by Jeff McKenna, 17 years ago

Cc: Jeff McKenna added; jmckenna@… removed

I have hit this same error enough, to now realize that it usually happens with MS4W's GDAL utilities, yet not with the FWTools utilities (with the same image). Can anyone think of a reason why this error would be thrown on an image in MS4W but never in FWTools? I mean ignoring the obvious GDAL versions differences.

comment:21 by warmerdam, 17 years ago

Cc: hobu added

Jeff,

I believe the issue is with libtiff. FWTools builds with "libtiff development head", and I think this problem is fixed there. I presume your libtiff is inherited from "buildkit"? Perhaps we need to issue a new libtiff4 release (beta1) and migrate buildkit to that?

comment:22 by Jeff McKenna, 17 years ago

ah true Howard, you are probably exactly right. The buildkit uses 'tiff-3.8.2'. Do you want me to test with the "libtiff development head", or wait for the new libtiff4 release? Let me know what you think.

comment:23 by Mateusz Łoskot, 16 years ago

Cc: Mateusz Łoskot added
Description: modified (diff)

comment:24 by Even Rouault, 14 years ago

Resolution: invalid
Status: assignedclosed

Old issue. Hopefully fixed by libtiff 3.9.1

Note: See TracTickets for help on using tickets.