NetCDF Driver should read/write data in stripes instead of the whole array

[rsignell]

When I tried to convert a 20000x15000 file using the GMT driver (which writes NetCDF with Z as a 1D variable) it works fine, but the NetCDF driver gives an "Out of memory" error:

gdal_translate psdem_2005_geo.tif -of netcdf psdem_2005.nc
Input file size is 20000, 15000
0ERROR 2: CPLMalloc(): Out of memory allocating 1200000000 bytes.

It looks like the NetCDF driver is trying to allocate memory for the entire data grid being processed instead of reading/writing by column, row or chunk. This should be fixed, to allow processing of much larger grids on machines with limited memory.

[warmerdam]

Denis,

Any thoughts on this issue? Skimming the CreateCopy? code in the netcdf driver, it looks like things are done one scanline at a time which should be OK.

I'll try and reproduce ...

[warmerdam]

It turns out the data was being written one line at a time, but for no good apparent reason the working buffer was also dimensioned by the number of lines. I have corrected this in trunk (r15476) and 1.5 branch (r15477).