Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#5082 closed defect (fixed)

netcdf driver does not set chunksize properly when ndims > 2, output is corrupted

Reported by: etourigny Owned by: etourigny
Priority: normal Milestone: 1.10.1
Component: GDAL_Raster Version: unspecified
Severity: normal Keywords:
Cc:

Description

when a netcdf file is copied with createcopy() and there are several dimensions (e.g. time), chunksize is not properly set. This can lead to files which are not properly read by gdal afterwards.

$ gdal_translate --debug off -co COMPRESS=DEFLATE -co FORMAT=NC4C -of netcdf test2.nc test2-cp.nc
Warning 1: No UNIDATA NC_GLOBAL:Conventions attribute
Input file size is 720, 360
0...10...20...30...40...50ERROR 1: netcdf error #-127 : NetCDF: Bad chunk sizes. .
at (netcdfdataset.cpp,DefVarDeflate,5745)

...60...70...80...90...100 - done.
$ gdalinfo -stats test2-cp.nc 
Driver: netCDF/Network Common Data Format
Files: test2-cp.nc
Size is 720, 360
Coordinate System is `'
Origin = (-180.000000000000000,90.000000000000000)
Pixel Size = (0.500000000000000,-0.500000000000000)
Metadata:
  lat#long_name=latitude
  lat#standard_name=latitude
  lat#units=degrees_north
  lon#long_name=longitude
  lon#standard_name=longitude
  lon#units=degrees_east
  NC_GLOBAL#Conventions=CF-1.5
  NC_GLOBAL#GDAL=GDAL 1.10.0, released 2013/04/13
  NC_GLOBAL#history=Wed May 15 19:02:07 2013: GDAL CreateCopy( test2-cp.nc, ... )
  NETCDF_DIM_EXTRA={time}
  NETCDF_DIM_time_DEF={2,6}
  NETCDF_DIM_time_VALUES={-363,-362}
  npptot#_FillValue=8.999999828524175e+20
  npptot#long_name=total npp
  npptot#units=kg m-2 year-1
  time#long_name=time
  time#units=days since 1980-12-31
Corner Coordinates:
Upper Left  (-180.0000000,  90.0000000) 
Lower Left  (-180.0000000, -90.0000000) 
Upper Right ( 180.0000000,  90.0000000) 
Lower Right ( 180.0000000, -90.0000000) 
Center      (   0.0000000,   0.0000000) 
Band 1 Block=720x360 Type=Float64, ColorInterp=Undefined
ERROR 1: nBlockYSize = 360, only 1 supported when reading bottom-up dataset
ERROR 1: test2-cp.nc, band 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: test2-cp.nc, band 1: Failed to compute statistics, no valid pixels found in sampling.
  NoData Value=8.9999998285241752e+20
  Metadata:
    _FillValue=8.999999828524175e+20
    long_name=total npp
    NETCDF_DIM_time=-363
    NETCDF_VARNAME=npptot
    units=kg m-2 year-1
Band 2 Block=720x360 Type=Float64, ColorInterp=Undefined
ERROR 1: nBlockYSize = 360, only 1 supported when reading bottom-up dataset
ERROR 1: test2-cp.nc, band 2: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: test2-cp.nc, band 2: Failed to compute statistics, no valid pixels found in sampling.
  NoData Value=8.9999998285241752e+20
  Metadata:
    _FillValue=8.999999828524175e+20
    long_name=total npp
    NETCDF_DIM_time=-362
    NETCDF_VARNAME=npptot
    units=kg m-2 year-1

Workaround is to set WRITE_BOTTOMUP=NO as creation option.

Proper fix is to set chunking as 1 for all dims, except for xdim (where it is xsize).

Attachments (1)

test2.nc (927.0 KB) - added by etourigny 5 years ago.
test file

Download all attachments as: .zip

Change History (3)

Changed 5 years ago by etourigny

Attachment: test2.nc added

test file

comment:1 Changed 5 years ago by etourigny

Resolution: fixed
Status: newclosed

fixed in r26003 and added creation option CHUNKING(default=YES), set to NO to use netcdf defaults

comment:2 Changed 5 years ago by etourigny

fixed in 1.10 (r26007)

Note: See TracTickets for help on using tickets.