Adding overviews to byte swapped 16bit TIFF files mixed up

[warmerdam]

A GDAL user added overviews to a bigendian sixteen bit per sample GeoTIFF file 
on a little endian system, and it appears the data byte order is inconsistent
(little endian) on the overviews.

[warmerdam]

tiffdump report on the problem file.

[warmerdam]

Thanks Frank.

Your suggestion works great. :o)

I'll post it to the list.

Cheers,

Antti

-----Original Message-----
From: Frank Warmerdam [mailto:warmerdam@pobox.com]
Sent: Wednesday, 26 June 2002 11:58 AM
To: Antti.Roppola@brs.gov.au
Subject: Re: [mapserver-users] Using gdaladdo on large TIFFs?



Antti,

I think this is a hint of the problem:
   hard0613g.tif:
   Magic: 0x4d4d <big-endian> Version: 0x2a
   Directory 0: offset 8 (0x8) next 300946982 (0x11f01626)

GDAL (and the underlying libtiff) support big endian or little endian TIFF
files; however, I think it likely doesn't properly handle adding overviews
to a bigendian TIFF file on a little endian system.  I think this has result
in some byte order problems with the file.

I would suggest you copy the file on Linux and then add the overviews.

eg.

gdal_translate hard0613g.tif hard0613g_le.tif
gdaladdo hard0613g_le.tif 2 4 8 16 32 64 ...

This might not even have been a problem if the file was eight bits per
sample.

I have created a bug report on this problem, but I am not promising a timely
fix, so the work around (copy and then create overviews) is likely your best
bet.

   http://bugzilla.remotesensing.org/show_bug.cgi?id=171

Best regards,

--
---------------------------------------+-------------------------------------- I
set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com light
and sound - activate the windows | http://pobox.com/~warmerdam and watch the
world go round - Rush | Geospatial Programmer for Rent

[warmerdam]

I have found that this error is in libtiff itself.  The various libtiff
functions such as TIFFWriteEncodedStrip(), TIFFWriteEncodedTile() and 
TIFFWriteScanline were not doing any "swabbing" of data being written out.
The swabbing function (TIFF->tif_postdecode()) is only applied when reading. 

The libtiff change notice looks like this:

	* libtiff/tif_write.c: modified tif_write.c so that the various
	encoded write functions use tif_postdecode() to apply byte order
	swapping (swab) to the application passed data buffer if the same
	would be done when reading.  This allows us to write pixel data with
	more than 8 bits per sample to existing files of a non-native byte 
	order.  One side effect of this change is the applications buffer
	itself is altered in this case by the act of writing. 

This now works fine, but the side effect of changing the applications buffer
may mean we need to do some additional work in this area in the future.