Opened 13 years ago
Closed 13 years ago
#3673 closed defect (fixed)
GEORASTER - NODATA ELEMENT IN WRONG PLACE - ORA 13454
| Reported by: | dgeringe | Owned by: | ilucena |
|---|---|---|---|
| Priority: | normal | Milestone: | |
| Component: | GDAL_Raster | Version: | 1.7.2 |
| Severity: | normal | Keywords: | georaster |
| Cc: | ilucena, warmerdam |
Description
When loading a netcdf file into GeoRaster, validation fails with ORA 13454.
The cause is the NODATA element in the raster metadata is in the wrong place.
If it is manually moved right after the cellDepth element, the raster validates.
Change History (11)
comment:1 by , 13 years ago
| Owner: | changed from to |
|---|
comment:2 by , 13 years ago
That is just the tip of the iceberg. The order shouldn't be a matter but it is.
The real problem is that GeoRaster supports a much more complex form of nodata notation than GDAL.
In GeoRaster a nodata notation could be just a value, a value range or an array of value ranges.
Examples of sdo_geor.addNodata() command on a 3 band georaster:
DECLARE gr sdo_georaster; BEGIN SELECT raster INTO gr FROM gdal_import t WHERE t.raster.rasterid=151 FOR UPDATE; SDO_GEOR.addNODATA(gr, 0, sdo_range_array(sdo_range(0,10))); SDO_GEOR.addNODATA(gr, 1, sdo_range_array(sdo_range(2,12))); SDO_GEOR.addNODATA(gr, 2, sdo_range_array(sdo_range(1,11),sdo_range(21,31))); UPDATE gdal_import t SET raster=gr WHERE t.raster.rasterid=151; END;
That creates one notadata entry for each band and for each band there is a range of pixel values supposed to be treated as notadata. Band number 3 actually contains 2 value ranges.
<layerInfo>
<layerDimension>BAND</layerDimension>
<objectLayer>
<layerNumber>0</layerNumber>
<layerDimensionOrdinate>0</layerDimensionOrdinate>
<layerID />
<NODATA>
<range>
<min>0</min>
<max>10</max>
</range>
</NODATA>
</objectLayer>
<subLayer>
<layerNumber>1</layerNumber>
<layerDimensionOrdinate>0</layerDimensionOrdinate>
<layerID />
<NODATA>
<range>
<min>2</min>
<max>12</max>
</range>
</NODATA>
</subLayer>
<subLayer>
<layerNumber>2</layerNumber>
<layerDimensionOrdinate>1</layerDimensionOrdinate>
<layerID />
<NODATA>
<range>
<min>1</min>
<max>11</max>
</range>
<range>
<min>21</min>
<max>31</max>
</range>
</NODATA>
</subLayer>
</layerInfo>
That doesn't seems to be possible to support but it should be possible to fix the support for just one nodata value per band.
comment:3 by , 13 years ago
| Cc: | added |
|---|---|
| Keywords: | georaster added |
Ivan,
The GDAL data model only permits a single nodata value for a GDALRasterBand. So if you need to support multiple values or more complex combinations for georaster it will need to be addressed via special creation options and/or driver specific metadata.
Another possibility is to implement support for band masks for georaster bands. On read you could interpret the complex nodata rules, and return a mask indicating invalid pixels. Details on band masks are available at:
I will warn that relatively few applications currently utilize the masking capability, but it would be a step forward.
comment:4 by , 13 years ago
So in case we run this sequence:
% gdal_translate -of georaster input.fmt geor:usr/pwd@db,rdt,rid -co "NODATA=[50,105],[200,250]" % gdal_translate -of gtiff geor:usr/pwd@db,rdt,rid output.tif
The output will preserve the original values plus a mask band with the combined notada(s). Right?
comment:5 by , 13 years ago
Ivan,
I think the mask is preserved in geotiff by gdal_translate, but I am not certain.
comment:6 by , 13 years ago
I added support for mask band in GeoRaster about a year ago. So exporting from GeoRaster (with a real mask band) to a output driver that support mask band should work.
Bu the idea here is that if the GeoRaster driver finds any complex Nodata notation then it will lie saying "This dataset contain mask band!". And when gdal_translate is converting from GeoRaster to another format that support mask band, the GeoRaster driver will build the binary mask on the fly, in memory, in a ReadBlock operation. If understand it correctly.
It sounds like a interesting solution but if not a lot of other output formats support mask band and since most applications don't care about it too, I am thinking that it will be better just a apply the mask while reading the regular data blocks.
comment:7 by , 13 years ago
comment:8 by , 13 years ago
The changes on revision r20130 will add support for per-band NoData Value.
Note that even tough the GDAL API supports a single NoData value per band, not all raster formats support that. So, if a GeoRaster contains one NoData per band and it is converted to Geotiff, the Geotiff driver will use the first band Nodata as the dataset NoData. The HFA driver and the Erdas Imagine Image format will handle that more precisely. Example:
% gdalinfo geor:scott/tiger,,gdal_import,raster,t.raster.rasterid=177 -mdd oracle
Driver: GeoRaster/Oracle Spatial GeoRaster
Files: none associated
Size is 4304, 3788
Coordinate System is:
PROJCS["WGS 84 / UTM zone 50N",
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0,
AUTHORITY["EPSG","8901"]],
UNIT["degree",0.01745329251994328,
AUTHORITY["EPSG","9122"]],
AUTHORITY["EPSG","4326"]],
UNIT["metre",1,
AUTHORITY["EPSG","9001"]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",0],
PARAMETER["central_meridian",117],
PARAMETER["scale_factor",0.9996],
PARAMETER["false_easting",500000],
PARAMETER["false_northing",0],
AUTHORITY["EPSG","32650"],
AXIS["Easting",EAST],
AXIS["Northing",NORTH]]
Origin = (279036.375000000000000,-2288286.375000000000000)
Pixel Size = (57.000000000000000,-57.000000000000000)
Metadata (oracle):
TABLE_NAME=scott.gdal_import
COLUMN_NAME=raster
RDT_TABLE_NAME=GDAL_RDT
RASTER_ID=177
SRID=32650
WKT=PROJCS["WGS 84 / UTM zone 50N", GEOGCS [ "WGS 84", DATUM ["World Geodetic System 1984 (EPSG ID 6326)", SPHEROID ["WGS 84 (EPSG ID 7030)", 6378137.0, 298.257223563]], PRIMEM [ "Greenwich", 0.000000 ], UNIT ["Decimal Degree", 0.0174532925199433]], PROJECTION ["Transverse Mercator"], PARAMETER ["Latitude_Of_Origin", 0.0], PARAMETER ["Central_Meridian", 117.0], PARAMETER ["Scale_Factor", 0.9996], PARAMETER ["False_Easting", 500000.0], PARAMETER ["False_Northing", 0.0], UNIT ["Meter", 1.0]]
METADATA=<georasterMetadata xmlns="http://xmlns.oracle.com/spatial/georaster">
<objectInfo>
<rasterType>21001</rasterType>
<isBlank>false</isBlank>
<defaultRed>1</defaultRed>
<defaultGreen>1</defaultGreen>
<defaultBlue>1</defaultBlue>
</objectInfo>
<rasterInfo>
<cellRepresentation>UNDEFINED</cellRepresentation>
<cellDepth>8BIT_U</cellDepth>
<totalDimensions>3</totalDimensions>
<dimensionSize type="ROW">
<size>3788</size>
</dimensionSize>
<dimensionSize type="COLUMN">
<size>4304</size>
</dimensionSize>
<dimensionSize type="BAND">
<size>2</size>
</dimensionSize>
<ULTCoordinate>
<row>0</row>
<column>0</column>
<band>0</band>
</ULTCoordinate>
<blocking>
<type>REGULAR</type>
<totalRowBlocks>15</totalRowBlocks>
<totalColumnBlocks>17</totalColumnBlocks>
<totalBandBlocks>2</totalBandBlocks>
<rowBlockSize>256</rowBlockSize>
<columnBlockSize>256</columnBlockSize>
<bandBlockSize>1</bandBlockSize>
</blocking>
<interleaving>BSQ</interleaving>
<pyramid>
<type>NONE</type>
</pyramid>
<compression>
<type>NONE</type>
</compression>
</rasterInfo>
<spatialReferenceInfo>
<isReferenced>true</isReferenced>
<isRectified>true</isRectified>
<SRID>32650</SRID>
<spatialResolution dimensionType="X">
<resolution>57</resolution>
</spatialResolution>
<spatialResolution dimensionType="Y">
<resolution>57</resolution>
</spatialResolution>
<modelCoordinateLocation>CENTER</modelCoordinateLocation>
<modelType>FunctionalFitting</modelType>
<polynomialModel rowOff="0" columnOff="0" xOff="0" yOff="0" zOff="0" rowScale="1" columnScale="1" xScale="1" yScale="1" zScale="1">
<pPolynomial pType="1" nVars="2" order="1" nCoefficients="3">
<polynomialCoefficients>-40145.375 0 -0.01754385964912281</polynomialCoefficients>
</pPolynomial>
<qPolynomial pType="1" nVars="0" order="0" nCoefficients="1">
<polynomialCoefficients>1</polynomialCoefficients>
</qPolynomial>
<rPolynomial pType="1" nVars="2" order="1" nCoefficients="3">
<polynomialCoefficients>-4895.375 0.01754385964912281 0</polynomialCoefficients>
</rPolynomial>
<sPolynomial pType="1" nVars="0" order="0" nCoefficients="1">
<polynomialCoefficients>1</polynomialCoefficients>
</sPolynomial>
</polynomialModel>
</spatialReferenceInfo>
<layerInfo>
<layerDimension>BAND</layerDimension>
<subLayer>
<layerNumber>1</layerNumber>
<layerDimensionOrdinate>0</layerDimensionOrdinate>
<layerID />
<NODATA>
<value>10</value>
</NODATA>
</subLayer>
<subLayer>
<layerNumber>2</layerNumber>
<layerDimensionOrdinate>1</layerDimensionOrdinate>
<layerID />
<NODATA>
<value>20</value>
</NODATA>
</subLayer>
</layerInfo>
</georasterMetadata>
Image Structure Metadata:
INTERLEAVE=BAND
COMPRESSION=NONE
Corner Coordinates:
Upper Left ( 279036.375,-2288286.375) (114d52'43.45"E, 20d40'51.52"S)
Lower Left ( 279036.375,-2504202.375) (114d51'0.13"E, 22d37'49.46"S)
Upper Right ( 524364.375,-2288286.375) (117d14'2.25"E, 20d41'37.96"S)
Lower Right ( 524364.375,-2504202.375) (117d14'13.66"E, 22d38'40.72"S)
Center ( 401700.375,-2396244.375) (116d 2'59.68"E, 21d40'0.36"S)
Band 1 Block=256x256 Type=Byte, ColorInterp=Gray
NoData Value=10
Band 2 Block=256x256 Type=Byte, ColorInterp=Gray
NoData Value=20
Converted to Geotiff:
% gdal_translate -of gtiff geor:scott/tiger,,gdal_import,raster,t.raster.rasterid=177 out.tif
Input file size is 4304, 3788
0...10...20...30...40...50...60...70...80...90...100 - done.
% gdalinfo out.tif
Driver: GTiff/GeoTIFF
Files: out.tif
out.tif.aux.xml
Size is 4304, 3788
Coordinate System is:
PROJCS["WGS 84 / UTM zone 50N",
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433],
AUTHORITY["EPSG","4326"]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",0],
PARAMETER["central_meridian",117],
PARAMETER["scale_factor",0.9996],
PARAMETER["false_easting",500000],
PARAMETER["false_northing",0],
UNIT["metre",1,
AUTHORITY["EPSG","9001"]],
AUTHORITY["EPSG","32650"]]
Origin = (279036.375000000000000,-2288286.375000000000000)
Pixel Size = (57.000000000000000,-57.000000000000000)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left ( 279036.375,-2288286.375) (114d52'43.45"E, 20d40'51.52"S)
Lower Left ( 279036.375,-2504202.375) (114d51'0.13"E, 22d37'49.46"S)
Upper Right ( 524364.375,-2288286.375) (117d14'2.25"E, 20d41'37.96"S)
Lower Right ( 524364.375,-2504202.375) (117d14'13.66"E, 22d38'40.72"S)
Center ( 401700.375,-2396244.375) (116d 2'59.68"E, 21d40'0.36"S)
Band 1 Block=4304x1 Type=Byte, ColorInterp=Gray
NoData Value=10
Band 2 Block=4304x1 Type=Byte, ColorInterp=Undefined
NoData Value=10
Converted to Erdas Imagine Image:
% rm out.*
% gdal_translate -of hfa geor:scott/tiger,,gdal_import,raster,t.raster.rasterid=177 out.img
% gdalinfo out.img
Driver: HFA/Erdas Imagine Images (.img)
Files: out.img
out.img.aux.xml
Size is 4304, 3788
Coordinate System is:
PROJCS["WGS_1984_UTM_Zone_50N",
GEOGCS["GCS_WGS_1984",
DATUM["WGS_1984",
SPHEROID["WGS_1984",6378137,298.257223563]],
PRIMEM["Greenwich",0],
UNIT["Degree",0.017453292519943295]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",0],
PARAMETER["central_meridian",117],
PARAMETER["scale_factor",0.9996],
PARAMETER["false_easting",500000],
PARAMETER["false_northing",0],
UNIT["Meter",1]]
Origin = (279036.375000000000000,-2288286.375000000000000)
Pixel Size = (57.000000000000000,-57.000000000000000)
Corner Coordinates:
Upper Left ( 279036.375,-2288286.375) (114d52'43.45"E, 20d40'51.52"S)
Lower Left ( 279036.375,-2504202.375) (114d51'0.13"E, 22d37'49.46"S)
Upper Right ( 524364.375,-2288286.375) (117d14'2.25"E, 20d41'37.96"S)
Lower Right ( 524364.375,-2504202.375) (117d14'13.66"E, 22d38'40.72"S)
Center ( 401700.375,-2396244.375) (116d 2'59.68"E, 21d40'0.36"S)
Band 1 Block=64x64 Type=Byte, ColorInterp=Undefined
Description = Layer_1
NoData Value=10
Metadata:
LAYER_TYPE=athematic
Band 2 Block=64x64 Type=Byte, ColorInterp=Undefined
Description = Layer_2
NoData Value=20
Metadata:
LAYER_TYPE=athematic
I don't know if we should consider it a bug or a feature of Geotiff driver but it is a "users be aware" issue.
comment:9 by , 13 years ago
On r20266 I implemented full support for NODATA Values and Value Ranges.
More information about that GeoRaster's concept is available here:
http://download.oracle.com/docs/cd/E11882_01/appdev.112/e11827/geor_intro.htm#BABJCCBA
The way it is working now is that if there are just one regular NoData values on a GeoRaster Layer then this value will be treated as a regular GDALRasterBand NoData value.
If there are more than one value or a combination of values and value ranges then the drivel will act as if there was no NoData value unless the Configuration Option or environment variable GEOR_FILTER_NODATA_VALUES exist.
If the GEOR_FILTER_NODATA_VALUES is present and set to any value other than "NO" then the driver will apply the filtering process as it is described by Oracle's documentation. It will look into all the pixel from each Band and it will compare against the 'Object Layer's NoData and 'SubLayer's NoData, if it is the case then it will replace the pixel by the first NoData value on the list.
I ran a test with QGIS loading a GeoRaster with NoData Values and Value Ranges and the results are looking pretty good.
Next step if to add that to the documentation.
But first, please let me know if that solution works for you.
comment:10 by , 13 years ago
Oops, I made a mistake on my last comment.
Correcting me:
If there are several NoData values and/or NoData value ranges the driver will assume by default that it suppose to do the filtering, replacing values as it was explained.
In that case, the only thing that will prevent the filtering process is the environment variable GEOR_FILTER_NODATA_VALUES set to NO. That could also be set at GDAL command lines utilities with "--configure GEOR_FILTER_NODATA_VALUES=NO" or programatically.
The logic behind it is that a visualization tool like QGIS and Mapserver will represent the image as expected by the user without any change in the software configuration.
comment:11 by , 13 years ago
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
This problem has passed unnoticeable because the data is still consistent and can be read by GDAL and even Oracle's GeoRasterViewer.
Here is the section of the GeoRaster metadata, where the "rasterInfo" is showing the NODATA entry at the end:
<rasterInfo> <cellRepresentation>UNDEFINED</cellRepresentation> <cellDepth>16BIT_S</cellDepth> <totalDimensions>2</totalDimensions> <dimensionSize type="ROW"> <size>512</size> </dimensionSize> <dimensionSize type="COLUMN"> <size>512</size> </dimensionSize> <ULTCoordinate> <row>0</row> <column>0</column> </ULTCoordinate> <blocking> <type>REGULAR</type> <totalRowBlocks>2</totalRowBlocks> <totalColumnBlocks>2</totalColumnBlocks> <rowBlockSize>256</rowBlockSize> <columnBlockSize>256</columnBlockSize> </blocking> <interleaving>BSQ</interleaving> <pyramid> <type>NONE</type> </pyramid> <compression> <type>NONE</type> </compression> '''<NODATA>-9999</NODATA>''' </rasterInfo>And the problem shows when running the function validateGeoRaster() from SDO_GEOR
Frank,
I don't think the GDAL's MiniXML allow us to force the order of new XML Elements under a certain node. Does it? Can you see how can we do that?