Opened 9 years ago

Closed 9 years ago

#5928 closed defect (invalid)

Can't covert 14 GB tiff to NETCDF

Reported by: tsapin Owned by: warmerdam
Priority: highest Milestone:
Component: default Version: 1.11.1
Severity: blocker Keywords:
Cc:

Description


Attachments (4)

Output.jpg (190.8 KB ) - added by tsapin 9 years ago.
The error message I got during the conversion
Checksum output.jpg (231.1 KB ) - added by tsapin 9 years ago.
checksum (25.3 KB ) - added by tsapin 9 years ago.
dump.txt (1.5 KB ) - added by oleg531 9 years ago.
dump my.tif

Download all attachments as: .zip

Change History (28)

by tsapin, 9 years ago

Attachment: Output.jpg added

The error message I got during the conversion

comment:1 by tsapin, 9 years ago

I still can successfully convert small files. But with the big one - I am challenging issues. The file contains 14 GB of data

Thank you in advance!

comment:2 by Even Rouault, 9 years ago

Are you sure the TIFF is not corrupted ?

comment:3 by tsapin, 9 years ago

Hi! Well, I guess so. It's just too big.

comment:4 by tsapin, 9 years ago

BTW - this is a GEOTIFF. May be it should be converted differently?

in reply to:  4 comment:5 by Even Rouault, 9 years ago

Replying to tsapin:

BTW - this is a GEOTIFF. May be it should be converted differently?

No, TIFF or GeoTIFF, both are fine. try "gdalinfo -checksum 2013_30m_cdls.tif"

by tsapin, 9 years ago

Attachment: Checksum output.jpg added

by tsapin, 9 years ago

Attachment: checksum added

comment:6 by tsapin, 9 years ago

Attached checksum output as well as stderr message

comment:7 by Even Rouault, 9 years ago

Reviewing the code, I would say that the file is corrupted. It seams that the declared tile length of one tile is over 2GB, which cause the "Integer overflow" error.

comment:8 by tsapin, 9 years ago

well it is a big file... what is the max value for tile length it could process? thanks! You really helping me with this )

comment:9 by Even Rouault, 9 years ago

As I said, the maximum size of a tile (for a 32bit process) is 2GB. But that should never be reached in normal file. For your file, the tile dimension is 304x96, so each tile size should be constant at 29184 bytes, so very far from 2GB. So the TileCountBytes TIFF tag is likely corrupted. Try to check with the person that provided the file to you

comment:10 by tsapin, 9 years ago

Will do. Thank you!

in reply to:  9 comment:11 by oleg531, 9 years ago

Replying to rouault:

As I said, the maximum size of a tile (for a 32bit process) is 2GB. But that should never be reached in normal file. For your file, the tile dimension is 304x96, so each tile size should be constant at 29184 bytes, so very far from 2GB. So the TileCountBytes TIFF tag is likely corrupted. Try to check with the person that provided the file to you

How can I get the value of TileCountBytes from TiFF file? I want to make sure, that TIFF file is corrupted. These file open successful by the Geomatica Freeview( http://go.pcigeomatics.com/e/13502/14Windows64SoftwareInstall-exe/9gxt7/234293350).

Thanks!

comment:12 by Even Rouault, 9 years ago

Can you paste the output of "tiffdump the.tif" ? tiffdump is a utility of libtiff

in reply to:  12 comment:13 by oleg531, 9 years ago

Replying to rouault:

Can you paste the output of "tiffdump the.tif" ? tiffdump is a utility of libtiff

So, I downloaded libtiff and try to execute the command :

tiffdump the.tif"

But, tiffdump give me a error: my.tif: Not a TIFF file, bad version number 43 (0x2b). I want to try the big tiff version of this library: http://bigtiff.org/

comment:14 by Even Rouault, 9 years ago

Use rather 4.0.4beta from http://download.osgeo.org/libtiff/

by oleg531, 9 years ago

Attachment: dump.txt added

dump my.tif

comment:15 by Even Rouault, 9 years ago

The dump looks right. Did you try with a recent version of GDAL ?

in reply to:  15 comment:16 by oleg531, 9 years ago

Replying to rouault:

The dump looks right. Did you try with a recent version of GDAL ?

gdalinfo --version GDAL 1.11.1, released 2014/09/24

comment:17 by Even Rouault, 9 years ago

If you compiled it yourself, could you try configuring "--with-libtiff=internal --with-geotiff=internal --with-rename-internal-libtiff-symbols --with-rename-internal-libgeotiff-symbols" ?

in reply to:  17 comment:18 by oleg531, 9 years ago

Replying to rouault:

If you compiled it yourself, could you try configuring "--with-libtiff=internal --with-geotiff=internal --with-rename-internal-libtiff-symbols --with-rename-internal-libgeotiff-symbols" ?

How can I do this? Thanks!

comment:19 by oleg531, 9 years ago

I use command

gdal_translate.exe -of netCDF ndv.tif  ndv.nc

to convert TIF to netCDF.

But large(14GB) files can't be converted and my TIF file is not corrupted. So, may be some additional command required? Thanks!

comment:20 by oleg531, 9 years ago

Priority: normalhighest
Severity: normalblocker
Version: unspecified1.11.1

comment:21 by Even Rouault, 9 years ago

It's just too hard to make you remotly debug this, so unless you share the file for download, I don't see how progress can be made on this

comment:22 by oleg531, 9 years ago

I compiled the latest version of gdal and trying to convert my file to netCDF. But I have a error:Output driver `netCDF' not recognised. How can I register this netCDF driver ? I also downloaded and installed the netCDF. So, that command I should execute to register netCDF driver? Thanks!

comment:23 by Even Rouault, 9 years ago

If you're on Linux, you should install the netcdf development package and ./configure should auto-detect it. Otherwise for Windows, the packages marked "-development" on http://archive.gisinternals.com/sdk/ are already compiled with netCDF support. But before trying conversion to netCDF, you should try with gdalinfo -checksum to see if the TIF is properly read

comment:24 by Even Rouault, 9 years ago

Resolution: invalid
Status: newclosed

No feedback. Closing

Note: See TracTickets for help on using tickets.