62 | | |
63 | | === SRS Handling: Continue saving GDAL custom attribute tags by default to CRS definition, but add option not to save === |
| 63 | === 3) Changes to improve SRS Interoperability between GDAL and NetCDF CF-1 for Map Projected data === |
| 64 | |
| 65 | See GDAL ticket: #2893 |
| 66 | |
| 67 | While GDAL uses the OGC Well Known Text (WKT) format internally to store datasets' Spatial Reference Systems (SRS), the CF-1 conventions use a custom system based on a special 'grid_mapping' NetCDF variable with a set of projection attributes, and also Coordinate variables for both the projected coordinates and latitude and longitude. See [[http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/ch05s06.html CF1.5 ch5.6]] for an explanation and examples, and [[http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/apf.html CF1.5 App F]] for a list of supported projections. |
| 68 | |
| 69 | Unfortunately, the CF-1 conventions do not currently allow saving of all data needed for a fully georeferenced SRS, such as Named datums. For background on this, see CF convention trac tickets [[https://cf-pcmdi.llnl.gov/trac/ticket/9 #9]], [[https://cf-pcmdi.llnl.gov/trac/ticket/18 #18]] and [[https://cf-pcmdi.llnl.gov/trac/ticket/27 #27]]. |
| 70 | |
| 71 | The proposed changes in this section update the NetCDF driver when exporting to NetCDF to: |
| 72 | * save extra information so datasets are CF-1 compliant and can be viewed as gridded data in CF-1 compliant NetCDF applications built on NetCDF Java (X/Y projected coordinate arrays, and optionally full Lat/Lon arrays); |
| 73 | * save as much of the SRS information in CF-1 compliant format as possible (including ellipsoid parameters); |
| 74 | * continue exporting extra GDAL information by default that allows full re-import. However add to the driver a |
| 75 | creation option so the user can choose not to export them. |
| 76 | |
| 77 | The following subsections describe these changes specifically. |
| 78 | |
| 79 | ==== 3.1) Proj. SRS Handling: When exporting to NetCDF map projections via GDAL, save projection coordinate variables. Optionally, also save lat and lon mapping variables. ==== |
| 80 | |
| 81 | '''Related fix''': #2893 [[BR]] |
| 82 | '''New -co options''': |
| 83 | * WRITE_LONLAT=YES/NO/IFNEEDED (default: YES for geographic, IFNEEDED for projected) ->Note, this not implemented yet |
| 84 | * TYPE_LONLAT=float/double (default: double for geographic, float for projected) |
| 85 | |
| 86 | '''Has back-compat impact?''': No |
| 87 | |
| 88 | '''Change''': |
| 89 | |
| 90 | When exporting rasters to NetCDF, save Coordinate Variables for the projected SRS, as required by the CF-1 convention, normally saved as "x" and "y". |
| 91 | |
| 92 | Optionally using -co options listed above, also write 2D 'lat' and 'lon' arrays, which are specified as part of the CF-1 convention for map projections - but are not actually required for the Unidata-maintained NetCDF Java API and applications built upon them (eg ncWMS, Thredds Data Server). |
| 93 | |
| 94 | The 'IFNEEDED' option of WRITELONLAT is used in the case of exporting a projection which is not supported by the CF-1.5 standard (e.g. sinusoidal), so any CF-1.5 compliant software can locate the points on a lon/lat map. |
| 95 | |
| 96 | '''Rationale''': |
| 97 | |
| 98 | The CF-1 conventions specify these coordinate variables as necessary for grids, see [[http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/ch05s06.html ch5.6]] and [[http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/apf.html App F]] in the document. |
| 99 | |
| 100 | Tests with NetCDF CF-1 reading software indeed show that rasters with projected grids will not be correctly interpreted without the coordinate variables in the projected system (the NetCDF-Java API determines this, since it's used by ncWMS, ToolsUI and Thredds Data Server). |
| 101 | |
| 102 | While the CF-1 convention examples also contain 2D "lat" and "lon" Variables containing the latitude and longitude at each data point, tests have shown that the NetCDF Java API can correctly interpret CF-1 data without these added. The rationale for this point is given at http://www.mail-archive.com/cf-metadata@cgd.ucar.edu/msg00759.html. |
| 103 | |
| 104 | Thus we've chosen to make it ''optional'' to write these Variables, as they are redundant from GDAL's point of view (Lat and Lon are calculated on-the-fly), and they increase the size of a NetCDF file significantly. (E.g. if the actual band-data is stored as a 2D Byte array, but the lat and lon are stored as doubles, then the file-size can easily increase by 10X). |
| 105 | |
| 106 | ==== 3.2) Proj. SRS Handling: Add saving of reference Ellipsoid parameters in CF-1 compliant format used by dataset ==== |
| 107 | |
| 108 | '''Related fix''': #2893 [[BR]] |
| 109 | '''New -co options''': No, though related to the new proposed WRITE_GDAL_TAGS option above. |
| 110 | |
| 111 | '''Has back-compat impact?''': No |
| 112 | |
| 113 | '''Change''': |
| 114 | |
| 115 | When exporting to NetCDF, the new driver will utilise the CF-1 attributes that allow specification of the reference ellipsoid used by the dataset's CRS (e.g. 'semi_major_axis', 'inverse_flattening'). |
| 116 | |
| 117 | However, on importing data the driver will continue to use the full GDAL WKT specification in the 'spatial_ref' attribute where present. This is because the CF-1.5 conventions still don't include provision for saving full Datum information, such as the name and authority of the datum, which is needed for precise transformations. The CF-1 ellipsoid attributes will only be used as a fallback when the full WKT isn't present. |
| 118 | |
| 119 | {{{ |
| 120 | #!comment |
| 121 | ((?? pds: Should we add anything here about special import behaviour, e.g. whether to auto-detect datum on import based on ellipsoid values?? Or should there be an option for this?)) |
| 122 | |
| 123 | -> etienne: we could look into that for most common datums, but specify that there is no guarantee that a specific datum will be checked for. |
| 124 | }}} |
| 125 | |
| 126 | '''Rationale''': |
| 127 | |
| 128 | CF-1 convention's only support of datums currently is to specify a reference ellipsoid using parameters such as 'semi_major_axis' and 'inverse_flattening'. |
| 129 | |
| 130 | While these attributes alone do not fully specify the datum (which requires an EPSG code so control points can be referenced), they at least allow fairly accurate map projection of the created NetCDF file using tools such as ncWMS. |
| 131 | |
| 132 | In the longer-term it would be beneficial to work with the CF-1 community to allow saving of full datum information in a format readable by CF-1. |
| 133 | |
| 134 | ==== 3.3) Proj. SRS Handling: Continue saving GDAL custom attribute tags by default to CRS definition, but add option not to save ==== |
95 | | === SRS Handling: Add saving of reference Ellipsoid parameters in CF-1 compliant format used by dataset === |
96 | | |
97 | | '''Related fix''': #2893 [[BR]] |
98 | | '''New -co options''': No, though related to the new proposed WRITE_GDAL_TAGS option above. |
99 | | |
100 | | '''Has back-compat impact?''': No |
101 | | |
102 | | '''Change''': |
103 | | |
104 | | When exporting to NetCDF, the new driver will utilise the CF-1 attributes that allow specification of the reference ellipsoid used by the dataset's CRS (e.g. 'semi_major_axis', 'inverse_flattening'). |
105 | | |
106 | | However, on importing data the driver will continue to use the full GDAL WKT specification in the 'spatial_ref' attribute where present. This is because the CF-1.5 conventions still don't include provision for saving full Datum information, such as the name and authority of the datum, which is needed for precise transformations. The CF-1 ellipsoid attributes will only be used as a fallback when the full WKT isn't present. |
107 | | |
108 | | {{{ |
109 | | #!comment |
110 | | ((?? pds: Should we add anything here about special import behaviour, e.g. whether to auto-detect datum on import based on ellipsoid values?? Or should there be an option for this?)) |
111 | | |
112 | | -> etienne: we could look into that for most common datums, but specify that there is no guarantee that a specific datum will be checked for. |
113 | | }}} |
114 | | |
115 | | '''Rationale''': |
116 | | |
117 | | CF-1 convention's only support of datums currently is to specify a reference ellipsoid using parameters such as 'semi_major_axis' and 'inverse_flattening'. |
118 | | |
119 | | While these attributes alone do not fully specify the datum (which requires an EPSG code so control points can be referenced), they at least allow fairly accurate map projection of the created NetCDF file using tools such as ncWMS. |
120 | | |
121 | | In the longer-term it would be beneficial to work with the CF-1 community to allow saving of full datum information in a format readable by CF-1. |
122 | | |
123 | | === SRS Handling: When exporting to NetCDF map projections via GDAL, save projection coordinate variables. Optionally, also save lat and lon mapping variables. === |
124 | | |
125 | | '''Related fix''': #2893 [[BR]] |
126 | | '''New -co options''': |
127 | | * WRITE_LONLAT=YES/NO/IFNEEDED (default: YES for geographic, IFNEEDED for projected) ->Note, this not implemented yet |
128 | | * TYPE_LONLAT=float/double (default: double for geographic, float for projected) |
129 | | |
130 | | '''Has back-compat impact?''': No |
131 | | |
132 | | '''Change''': |
133 | | |
134 | | When exporting rasters to NetCDF, save Coordinate Variables for the projected SRS, as required by the CF-1 convention, normally saved as "x" and "y". |
135 | | |
136 | | Optionally using -co options listed above, also write 2D 'lat' and 'lon' arrays, which are specified as part of the CF-1 convention for map projections - but are not actually required for the Unidata-maintained NetCDF Java API and applications built upon them (eg ncWMS, Thredds Data Server). |
137 | | |
138 | | The ifneeded option of WRITELONLAT is used in the case of exporting a projection which is not supported by the CF-1.5 standard (e.g. sinusoidal), so any CF-1.5 compliant software can locate the points on a lon/lat map. |
139 | | |
140 | | '''Rationale''': |
141 | | |
142 | | The CF-1 conventions specify these coordinate variables as necessary for grids, see [[http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/ch05s06.html ch5.6]] and [[http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/apf.html App F]] in the document. |
143 | | |
144 | | Tests with NetCDF CF-1 reading software indeed show that rasters with projected grids will not be correctly interpreted without the coordinate variables in the projected system (the NetCDF-Java API determines this, since it's used by ncWMS, ToolsUI and Thredds Data Server). |
145 | | |
146 | | While the CF-1 convention examples also contain 2D "lat" and "lon" Variables containing the latitude and longitude at each data point, tests have shown that the NetCDF Java API can correctly interpret CF-1 data without these added. The rationale for this point is given at http://www.mail-archive.com/cf-metadata@cgd.ucar.edu/msg00759.html. |
147 | | |
148 | | Thus we've chosen to make it ''optional'' to write these Variables, as they are redundant from GDAL's point of view (Lat and Lon are calculated on-the-fly), and they increase the size of a NetCDF file significantly. (E.g. if the actual band-data is stored as a 2D Byte array, but the lat and lon are stored as doubles, then the file-size can easily increase by 10X). |
149 | | |
150 | | |
151 | | === Add support for different netcdf file types and compression === |
| 166 | |
| 167 | === 4) Add support for different netcdf file types and compression === |