64bit LFS support for G_ftell() and PRI_OFF_T gis.h
|Reported by:||hamish||Owned by:|
|Keywords:||r.in.bin, LFS, G_ftell(), PRI_OFF_T||Cc:|
as reported on the users' ML, r.in.bin is failing for input files bigger than 2GB. thread: "ERROR: Bytes do not match file size with r.in.bin (but file size is correct!!)"
It works in trunk, but not 6.x. Just after 6.4.1 was released r.in.bin got upgraded to off_t, but PRI_OFF_T in GRASS 6 set by gis.h remains as "ld" because
LFS_CFLAGS = -D_FILE_OFFSET_BITS=64
is missing from Grass.make (it is there in G7). That may just be the overflow in the error message though, I think the real problem is G_ftell() is always returning int, even when ftello() should be returning off_t.
fwiw in G6 only the flags for wxWidgets get D_FILE_OFFSET'd. In G7 PRI_OFF_T is "lld" and r.in.bin works on a >2GB test file.
here's a little Matlab/Octave code to make one:
%%%% make a >2GB binary data file fd1 = fopen('lfs_test.bin', 'w'); rows=19450 cols=29404 for i = 1:rows row_content = [1:cols] * sqrt(i); fwrite(fd1, row_content, 'real*4'); end fclose(fd1) %%%%
and import command:
$ ls -l lfs_test.bin -rw-r--r-- 1 hamish hamish 2287631200 May 10 12:43 lfs_test.bin GRASS> r.in.bin -f input=lfs_test.bin output=outputmap bytes=4 \ n=51:05:20.4N s=41:21:50.4N w=5:08:31.2W e=9:33:36E \ r=19450 c=29404 anull=-9999.0
the error in GRASS 6 is:
WARNING: File Size -2007336096 ... Total Bytes 2287631200 ERROR: Bytes do not match file size
This is a 64bit system, and
GRASS6> g.version -b | tr ' ' '\n' has the --enable-64bit option, but I'm not really sure why that's needed, as autoconf already knows the platform. (for cross-compiling from 32bit?)