#6870 closed defect (fixed)
Segmentation fault when trying to open a NetCDF subdataset with a 3rd (time) dimension
| Reported by: | atedstone | Owned by: | warmerdam |
|---|---|---|---|
| Priority: | normal | Milestone: | 2.1.4 |
| Component: | default | Version: | unspecified |
| Severity: | normal | Keywords: | |
| Cc: |
Description
I am trying to create a NetCDF file which will be compliant with the NetCDF Conventions CF-1.4, including defining the projection of the geo-referenced subdatasets.
I have found that GDAL crashes with a segmentation fault when trying to access a subdataset in this file. I posted a question on GIS Exchange https://gis.stackexchange.com/questions/237951/gdal-crashes-with-segmentation-fault-when-trying-to-open-a-netcdf-subdataset-wit and received a reply suggesting that this is a problem with GDAL and that I should file a bug report.
The dimensions of the main subdataset are (TIME, Y, X) where Y and X are the polar stereographic grid.
The file that I generate contains three variables: runoff (TIME, Y, X), lat (Y, X) and lon (Y, X). With gdalinfo I can access the lat and lon grids, but not runoff:
$ gdalinfo NETCDF:"myfile.nc":runoff Segmentation fault (core dumped)
In other words it looks like the TIME dimension is causing problems.
The file was created using xarray (http://xarray.pydata.org), with output set to NETCDF4. The ncdump of the file is as follows, and I've attached the file to the ticket.
N.b. this is a test-only file: runoff has dimensions TIME=2 and both matrices are filled with zeros. I can provide the creation script if useful.
netcdf myfile {
dimensions:
X = 756 ;
Y = 786 ;
TIME = 2 ;
variables:
int64 X(X) ;
X:units = "meters" ;
X:standard_name = "projection_x_coordinate" ;
X:point_spacing = "even" ;
X:axis = "X" ;
int64 Y(Y) ;
Y:units = "meters" ;
Y:standard_name = "projection_y_coordinate" ;
Y:point_spacing = "even" ;
Y:axis = "Y" ;
int64 TIME(TIME) ;
TIME:standard_name = "time" ;
TIME:axis = "TIME" ;
TIME:units = "days since 1958-01-01 00:00:00" ;
TIME:calendar = "proleptic_gregorian" ;
byte polar_stereographic ;
polar_stereographic:grid_mapping_name = "polar_stereographic" ;
polar_stereographic:scale_factor_at_central_origin = 0. ;
polar_stereographic:standard_parallel = 70. ;
polar_stereographic:straight_vertical_longitude_from_pole = -45. ;
polar_stereographic:latitude_of_projection_origin = 90. ;
polar_stereographic:false_easting = 0. ;
polar_stereographic:false_northing = 0. ;
double lon(Y, X) ;
lon:_FillValue = -9999. ;
lon:grid_mapping = "polar_stereographic" ;
lon:units = "degrees" ;
lon:standard_name = "longitude" ;
byte runoff(TIME, Y, X) ;
runoff:_FillValue = -15b ;
runoff:long_name = "runoff" ;
runoff:units = "mmWE" ;
runoff:standard_name = "runoff" ;
runoff:grid_mapping = "polar_stereographic" ;
double lat(Y, X) ;
lat:_FillValue = -9999. ;
lat:grid_mapping = "polar_stereographic" ;
lat:units = "degrees" ;
lat:standard_name = "latitude" ;
// global attributes:
:Conventions = "CF-1.4" ;
:history = "redacted" ;
:proj4 = "+init=epsg:3413" ;
:nx = 756. ;
:ny = 786. ;
:xmin = -1790000. ;
:ymax = -65000. ;
:spacing = 5000. ;
}
The file was created using the same conda environment that also provides GDAL, although I've tried to open the file on a different server with a different GDAL version and I also get the same problem.
Conda environment:
Running a conda environment: basemap 1.0.8.dev0 np112py35_3 conda-forge basemap-data-hires 1.0.8.dev0 2 conda-forge blas 1.1 openblas conda-forge bleach 1.5.0 py35_0 conda-forge bokeh 0.12.4 py35_0 bottleneck 1.2.0 np112py35_0 conda-forge ca-certificates 2017.1.23 0 conda-forge cairo 1.14.6 4 conda-forge cartopy 0.15.1 np112py35_1 conda-forge certifi 2017.1.23 py35_0 conda-forge cloudpickle 0.2.2 py35_2 conda-forge curl 7.52.1 0 conda-forge cycler 0.10.0 py35_0 conda-forge dask 0.13.0 py35_0 conda-forge decorator 4.0.11 py35_0 conda-forge descartes 1.1.0 py35_0 conda-forge dill 0.2.6 py35_0 conda-forge entrypoints 0.2.2 py35_1 conda-forge expat 2.1.0 2 conda-forge fastcache 1.0.2 py35_1 fontconfig 2.12.1 4 conda-forge freetype 2.7 1 conda-forge freexl 1.0.2 1 conda-forge gdal 2.1.3 np112py35_1 conda-forge geos 3.5.1 1 conda-forge gettext 0.19.7 1 conda-forge giflib 5.1.4 0 conda-forge glib 2.51.4 0 conda-forge gmp 6.1.2 0 conda-forge graphite2 1.3.9 0 conda-forge h5netcdf 0.3.1 py35_0 conda-forge h5py 2.6.0 np112py35_7 conda-forge harfbuzz 1.4.3 0 conda-forge hdf4 4.2.12 0 conda-forge hdf5 1.8.17 9 conda-forge html5lib 0.999 py35_0 conda-forge hyperspy 1.2 py35_0 conda-forge icu 58.1 1 conda-forge imageio 2.1.2 py35_0 conda-forge ipykernel 4.6.0 py35_0 conda-forge ipyparallel 6.0.1 py35_0 conda-forge ipython 5.3.0 py35_0 conda-forge ipython_genutils 0.2.0 py35_0 conda-forge ipywidgets 6.0.0 py35_0 conda-forge jinja2 2.9.5 py35_0 conda-forge jpeg 9b 0 conda-forge json-c 0.12 0 conda-forge jsonschema 2.5.1 py35_0 conda-forge jupyter 1.0.0 py35_0 conda-forge jupyter_client 5.0.1 py35_0 conda-forge jupyter_console 5.1.0 py35_0 conda-forge jupyter_core 4.3.0 py35_0 conda-forge kealib 1.4.6 3 conda-forge libdap4 3.18.3 2 conda-forge libffi 3.2.1 3 conda-forge libgfortran 3.0.0 1 libiconv 1.14 4 conda-forge libnetcdf 4.4.1.1 2 conda-forge libpng 1.6.28 0 conda-forge libpq 9.5.4 3 conda-forge libsodium 1.0.10 0 conda-forge libspatialite 4.3.0a 14 conda-forge libtiff 4.0.6 7 conda-forge libxml2 2.9.4 4 conda-forge libxslt 1.1.29 3 conda-forge locket 0.2.0 py35_1 conda-forge lxml 3.7.3 py35_0 conda-forge markupsafe 0.23 py35_1 conda-forge matplotlib 2.0.0 np112py35_2 conda-forge mistune 0.7.4 py35_0 conda-forge mpmath 0.19 py35_2 conda-forge natsort 5.0.2 py35_0 conda-forge nbconvert 5.1.1 py35_1 conda-forge nbformat 4.2.0 py35_0 conda-forge ncurses 5.9 10 conda-forge netcdf4 1.2.7 np112py35_0 conda-forge networkx 1.11 py35_0 conda-forge notebook 5.0.0 py35_0 conda-forge numpy 1.12.1 py35_blas_openblas_200 [blas_openblas] conda-forge olefile 0.44 py35_0 conda-forge openblas 0.2.19 1 conda-forge openjpeg 2.1.2 2 conda-forge openssl 1.0.2h 3 conda-forge owslib 0.14.0 py35_0 conda-forge pandas 0.19.2 np112py35_1 conda-forge pandoc 1.19.2 0 conda-forge pandocfilters 1.4.1 py35_0 conda-forge pango 1.40.4 0 conda-forge partd 0.3.7 py35_0 conda-forge patsy 0.4.1 py35_0 conda-forge pcre 8.39 0 conda-forge pexpect 4.2.1 py35_0 conda-forge pickleshare 0.7.3 py35_0 conda-forge pillow 4.0.0 py35_2 conda-forge pip 9.0.1 py35_0 conda-forge pixman 0.34.0 0 conda-forge proj4 4.9.3 2 conda-forge prompt_toolkit 1.0.13 py35_0 conda-forge ptyprocess 0.5.1 py35_0 conda-forge pyepsg 0.3.1 py35_0 conda-forge pyface 5.1.0 py35_1 conda-forge pygments 2.2.0 py35_0 conda-forge pyparsing 2.2.0 py35_0 conda-forge pyproj 1.9.5.1 py35_0 conda-forge pyqt 4.11.4 py35_2 conda-forge pyshp 1.2.10 py35_0 conda-forge python 3.5.3 1 conda-forge python-dateutil 2.6.0 py35_0 conda-forge pytz 2016.10 py35_0 conda-forge pywavelets 0.5.2 np112py35_0 conda-forge pyyaml 3.12 py35_0 conda-forge pyzmq 16.0.2 py35_1 conda-forge qt 4.8.7 7 conda-forge qtconsole 4.3.0 py35_0 conda-forge readline 6.2 0 conda-forge requests 2.13.0 py35_0 conda-forge scikit-image 0.13.0 np112py35_0 conda-forge scikit-learn 0.18.1 np112py35_blas_openblas_200 [blas_openblas] conda-forge scipy 0.19.0 np112py35_blas_openblas_200 [blas_openblas] conda-forge seaborn 0.7.1 py35_0 conda-forge setuptools 33.1.1 py35_0 conda-forge shapely 1.5.17 np112py35_2 conda-forge simplegeneric 0.8.1 py35_0 conda-forge sip 4.18 py35_1 conda-forge six 1.10.0 py35_1 conda-forge spectral 0.18 <pip> sqlite 3.13.0 1 conda-forge statsmodels 0.8.0 np112py35_0 conda-forge sympy 1.0 py35_0 conda-forge terminado 0.6 py35_0 conda-forge testpath 0.3 py35_0 conda-forge tk 8.5.19 1 conda-forge toolz 0.8.2 py35_0 conda-forge tornado 4.4.2 py35_0 conda-forge tqdm 4.11.2 py35_0 conda-forge traitlets 4.3.2 py35_0 conda-forge traits 4.6.0 py35_1 conda-forge traitsui 5.1.0 py35_0 conda-forge util-linux 2.21 0 wcwidth 0.1.7 py35_0 conda-forge webencodings 0.5 py35_0 conda-forge wheel 0.29.0 py35_0 conda-forge widgetsnbextension 2.0.0 py35_0 conda-forge xarray 0.9.1 py35_0 conda-forge xerces-c 3.1.4 3 conda-forge xlrd 1.0.0 py35_1 conda-forge xlwt 1.2.0 py35_0 xz 5.2.2 0 conda-forge yaml 0.1.6 0 conda-forge zeromq 4.2.1 1 conda-forge zlib 1.2.11 0 conda-forge
Change History (6)
comment:1 by , 7 years ago
comment:2 by , 7 years ago
A quick stack (not at head, but I don't think it matters here)...
#0 0x00007ffff70f94da in strlen () from libc.so.6
#1 0x000000000051cc3b in NCDFSafeStrcat (ppszDest=0x7fffffffb530, pszSrc=0x13bfce9 <.L.str.51> "}", nDestSize=0x7fffffffb538) at frmts/netcdf/netcdfdataset.cpp:8101
#2 0x000000000050d0d1 in NCDFGet1DVar (nCdfId=65536, nVarId=2, pszValue=0x7fffffffd048) at frmts/netcdf/netcdfdataset.cpp:8758
#3 0x00000000005171b6 in netCDFDataset::Open (poOpenInfo=0x7fffffffd4f0) at frmts/netcdf/netcdfdataset.cpp:6542
#4 0x00000000005938eb in GDALOpenEx (pszFilename=0x7ffff6c68560 "NETCDF:/usr/local/google/home/schwehr/myfile.nc:runoff", nOpenFlags=66, papszAllowedDrivers=0x0, papszOpenOptions=0x0,
papszSiblingFiles=0x0) at gcore/gdaldataset.cpp:2781
A little debugging with cerr:
gdalinfo NETCDF:"myfile.nc":runoff Warning 1: NetCDF driver detected file type=5, but libnetcdf detected type=3 nVarLen: 2 nVarType: 10 pszVarValue: Segmentation fault (core dumped)
This is going through the default case of switch (nVarType)
default:
CPLDebug("GDAL_netCDF", "NCDFGetVar1D unsupported type %d", nVarType);
CPLFree(pszVarValue);
pszVarValue = NULL;
break;
}
(gdb) n 8590 switch (nVarType) (gdb) n 8752 CPLFree(pszVarValue); (gdb) print nVarType $5 = 10
#define NC_INT64 10
comment:6 by , 7 years ago
| Milestone: | → 2.1.4 |
|---|
Note:
See TracTickets
for help on using tickets.
I didn't appreciate that attached files had a < 1 mb limit. The file is 11 mb. I hope making it available via Dropbox is acceptable: https://dl.dropboxusercontent.com/u/93558837/myfile.nc.