Problem creating netcdf files with numerous bands

[tbouffon]

gdal_translate has trouble creating netcdf files with more than 2053 bands :

$ gdal_translate -of NETCDF 2053.grb 2053.nc
Input file size is 1, 1
$ gdal_translate -of NETCDF 2054.grb 2054.nc 2>&1 | head
Input file size is 1, 1
0ERROR 5: Access window out of range in RasterIO().  Requested
(0,1) of size 1x1 on raster of 1x1.
ERROR 5: Access window out of range in RasterIO().  Requested
(0,1) of size 1x1 on raster of 1x1.
ERROR 5: Access window out of range in RasterIO().  Requested
(0,1) of size 1x1 on raster of 1x1.
ERROR 5: Access window out of range in RasterIO().  Requested
(0,1) of size 1x1 on raster of 1x1.
ERROR 5: Access window out of range in RasterIO().  Requested

Conversion to Geotiff from the same files do not present any problem. Is there a hard limit for netcdf creation ?

[tbouffon]

grib file with 2053 bands

[tbouffon]

Grib file with 2054 bands which causes troubles

[Kyle Shannon]

What are the particulars of your netcdf build? Version, platform, etc. If possible, run nc-config --all. I didn't have a problem creating the nc file on my build. I will look into this further.

kss

[tbouffon]

I don't have nc-config. My netcdf is 3.6.2 built on x86_64 rhel 4.8 netcdf has been built with :

./configure --enable-shared

I managed to translate the file on a fedora box, too. I'm trying to build gdal 1.8 against a different netcdf, to see if it changes something.

Thanks, Thomas

[tbouffon]

HI, I rebuilt gdal 1.8 statically linked against netcdf 4.1.1. I stillo have the same error. nc-config from this netcdf install gives

  --cc        -> cc
  --cflags    ->  -I/home/mfdp/mfdp013/netcdf/include
  --libs      -> -L/home/mfdp/mfdp013/netcdf/lib -lnetcdf -L/usr/lib64 -lcurl -L/usr/kerberos/lib -L/usr/kerberos/lib64 -lssl -lcrypto -lgssapi_krb5 -lkrb5 -lcom_err -lk5crypto -lresolv -ldl -lz -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lresolv -L/usr/kerberos/lib -L/usr/kerberos/lib64 -lidn -lssl -lcrypto -lssl -lcrypto -lgssapi_krb5 -lkrb5 -lcom_err -lk5crypto -lresolv -ldl -lz -lz

  --cxx       -> c++
  --has-c++   -> yes

  --fc        -> gfortran
  --fflags    -> -g -O2 -I/home/mfdp/mfdp013/netcdf/include
  --flibs     -> -L/home/mfdp/mfdp013/netcdf/lib -lnetcdf -L/usr/lib64 -lcurl -L/usr/kerberos/lib -L/usr/kerberos/lib64 -lssl -lcrypto -lgssapi_krb5 -lkrb5 -lcom_err -lk5crypto -lresolv -ldl -lz -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lresolv -L/usr/kerberos/lib -L/usr/kerberos/lib64 -lidn -lssl -lcrypto -lssl -lcrypto -lgssapi_krb5 -lkrb5 -lcom_err -lk5crypto -lresolv -ldl -lz -lz
  --has-f77   -> yes
  --has-f90   -> yes

  --has-dap   -> yes
  --has-nc2   -> yes
  --has-nc4   -> no
  --has-hdf5  -> no
  --has-hdf4  -> no
  --has-szlib -> no

  --prefix    -> /home/mfdp/mfdp013/netcdf
  --includedir-> /home/mfdp/mfdp013/netcdf/include
  --version   -> netCDF 4.1.1

I'm building gdal with rpmbuild, which adds lots of options to configure. I'm going to try and build it manually.

[tbouffon]

I seem to have found the problem : on line 2355 in frmts/netcdf/netcdf.cpp, anBandMap is initialized as an array of NC_MAX_DIMS integers, and on line 2662 a loop through the bands writes anBandMap[j-1]=j; These are the only references to this variable in the whole gdal tarball.

So I removed the definition of anBandMap and the loop and everything worked now.

I guess the strange number of bands needed to have gdal_translate are due to the way gcc organizes compiled code : with different -O and -g combinations, the results where varying from segmentation faults to incomplete files with damaged geography.

Moreover, the bands are registered as variables, so the relevant constant should be NC_MAX_VARS, which is never used.

Is this reference to anBandMap a remnant from an old, unused part of code, or a typo against panBandMap ? Why is it never used ?

Cheers, Thomas

[Kyle Shannon]

Thanks for investigating. I am not sure why that is the way it is. I agree with the logic above, referring to vars vs dims. I will check on it and update the ticket soon, I am a little under the weather, so it probably won't be until next week.

kss

[Kyle Shannon]

After looking at this a bit, I am not certain what the anBandMap is for or why it was implemented. Bands may or may not represent variables, such as when the data dimensions are x, y, and time. The bands would represent a dimension. I am not sure if the band map is there for a reason or not. If I can't find a valid reason for having it, I will comment it out. Thanks again, and sorry for the delay.

kss

[etourigny]

I can't reproduce the error in gdal-1.8.1 (which has anBandMap).

It has been removed entirely in trunk (r23265).

Closing this bug, if the problems comes back please reopen.