Opened 16 years ago

Last modified 16 years ago

#2591 closed defect

NetCDF Driver should read/write data in stripes instead of the whole array — at Version 1

Reported by: rsignell Owned by: warmerdam
Priority: normal Milestone: 1.5.3
Component: GDAL_Raster Version: 1.5.0
Severity: normal Keywords: netcdf
Cc: dnadeau

Description (last modified by warmerdam)

When I tried to convert a 20000x15000 file using the GMT driver (which writes NetCDF with Z as a 1D variable) it works fine, but the NetCDF driver gives an "Out of memory" error: 

gdal_translate psdem_2005_geo.tif -of netcdf psdem_2005.nc
Input file size is 20000, 15000
0ERROR 2: CPLMalloc(): Out of memory allocating 1200000000 bytes.

It looks like the NetCDF driver is trying to allocate memory for the entire data grid being processed instead of reading/writing by column, row or chunk. This should be fixed, to allow processing of much larger grids on machines with limited memory.

Change History (1)

comment:1 by warmerdam, 16 years ago

Cc: dnadeau added
Component: default → GDAL_Raster
Description: modified (diff)
Keywords: netcdf added
Priority: high → normal
Status: new → assigned

Denis,

Any thoughts on this issue? Skimming the CreateCopy code in the netcdf driver, it looks like things are done one scanline at a time which should be OK.

I'll try and reproduce ...

Note: See TracTickets for help on using tickets.