Opened 13 years ago

Closed 13 years ago

Last modified 13 years ago

#4204 closed defect (fixed)

Duplication in Metadata when translating from NetCDF to NetCDF

Reported by: hsumanto Owned by: warmerdam
Priority: normal Milestone:
Component: default Version: unspecified
Severity: normal Keywords: netcdf, gdal_translate
Cc: Kyle Shannon

Description

Let's say we need to perform the translation below to change nodata to certain value: gdal_translate -of netCDF -a_nodata 0 L71092086_08620110129_B50-netCDF-small.nc L71092086_08620110129_B50-netCDF-small-withnodata.nc

If you look at the gdalinfo of the new NetCDF file, you will notice that some metadata/attributes such as Conventions, grid_mapping, long_name, etc have been duplicated as shown below. 

Driver: netCDF/Network Common Data Format
Files: L71092086_08620110129_B50-netCDF-small.nc
       L71092086_08620110129_B50-netCDF-small.nc.aux.xml
Size is 333, 333
Coordinate System is:
PROJCS["WGS 84 / UTM zone 55N",
    GEOGCS["WGS 84",
    ...
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AUTHORITY["EPSG","32655"]]
Origin = (281385.000000000000000,-4040685.000000000000000)
Pixel Size = (30.030030030030030,-30.030030030030030)
Metadata:
  NC_GLOBAL#Conventions=CF-1.0
  Band1#grid_mapping=transverse_mercator
  Band1#long_name=GDAL Band Number 1
  transverse_mercator#Northernmost_Northing=-4.04068e+06
  transverse_mercator#Southernmost_Northing=-4.05068e+06
  transverse_mercator#Easternmost_Easting=291385
  transverse_mercator#Westernmost_Easting=281385
  transverse_mercator#spatial_ref=PROJCS["WGS 84 / UTM zone 55N",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433],AUTHORITY["EPSG","4326"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",147],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AUTHORITY["EPSG","32655"]]
  transverse_mercator#GeoTransform=281385 30.03003003003003 0 -4040685 0 -30.03003003003003
  transverse_mercator#grid_mapping_name=transverse_mercator
  transverse_mercator#latitude_of_projection_origin=0.000000e+00
  transverse_mercator#longitude_of_central_meridian=1.470000e+02
  transverse_mercator#scale_factor_at_central_meridian=9.996000e-01
  transverse_mercator#false_easting=5.000000e+05
  transverse_mercator#false_northing=0.000000e+00
  AREA_OR_POINT=Point
Corner Coordinates:
Upper Left  (  281385.000,-4040685.000) (144d33'34.28"E, 36d29'11.56"S)
Lower Left  (  281385.000,-4050685.000) (144d33'24.08"E, 36d34'35.81"S)
Upper Right (  291385.000,-4040685.000) (144d40'15.85"E, 36d29'19.59"S)
Lower Right (  291385.000,-4050685.000) (144d40' 6.12"E, 36d34'43.86"S)
Center      (  286385.000,-4045685.000) (144d36'50.08"E, 36d31'57.75"S)
Band 1 Block=333x1 Type=Byte, ColorInterp=Gray
  NoData Value=0
  Metadata:
    NETCDF_VARNAME=Band1

Driver: netCDF/Network Common Data Format
Files: L71092086_08620110129_B50-netCDF-small-withnodata.nc
       L71092086_08620110129_B50-netCDF-small-withnodata.nc.aux.xml
Size is 333, 333
Coordinate System is:
PROJCS["WGS 84 / UTM zone 55N",
    GEOGCS["WGS 84",
    ...
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AUTHORITY["EPSG","32655"]]
Origin = (281385.000000000000000,-4040685.000000000000000)
Pixel Size = (30.030030030030030,-30.030030030030030)
Metadata:
  NC_GLOBAL#Conventions=CF-1.0
  NC_GLOBAL#NC_GLOBAL-Conventions=CF-1.0
  NC_GLOBAL#Band1-grid_mapping=transverse_mercator
  NC_GLOBAL#Band1-long_name=GDAL Band Number 1
  NC_GLOBAL#transverse_mercator-Northernmost_Northing=-4.04068e+06
  NC_GLOBAL#transverse_mercator-Southernmost_Northing=-4.05068e+06
  NC_GLOBAL#transverse_mercator-Easternmost_Easting=291385
  NC_GLOBAL#transverse_mercator-Westernmost_Easting=281385
  NC_GLOBAL#transverse_mercator-spatial_ref=PROJCS[WGS 84 / UTM zone 55N,GEOGCS[WGS 84,DATUM[WGS_1984,SPHEROID[WGS 84,6378137,298.257223563,AUTHORITY[EPSG,7030]],AUTHORITY[EPSG,6326]],PRIMEM[Greenwich,0],UNIT[degree,0.0174532925199433],AUTHORITY[EPSG,4326]],PROJECTION[Transverse_Mercator],PARAMETER[latitude_of_origin,0],PARAMETER[central_meridian,147],PARAMETER[scale_factor,0.9996],PARAMETER[false_easting,500000],PARAMETER[false_northing,0],UNIT[metre,1,AUTHORITY[EPSG,9001]],AUTHORITY[EPSG,32655]]
  NC_GLOBAL#transverse_mercator-GeoTransform=281385 30.03003003003003 0 -4040685 0 -30.03003003003003
  NC_GLOBAL#transverse_mercator-grid_mapping_name=transverse_mercator
  NC_GLOBAL#transverse_mercator-latitude_of_projection_origin=0.000000e+00
  NC_GLOBAL#transverse_mercator-longitude_of_central_meridian=1.470000e+02
  NC_GLOBAL#transverse_mercator-scale_factor_at_central_meridian=9.996000e-01
  NC_GLOBAL#transverse_mercator-false_easting=5.000000e+05
  NC_GLOBAL#transverse_mercator-false_northing=0.000000e+00
  NC_GLOBAL#AREA_OR_POINT=Point
  Band1#grid_mapping=transverse_mercator
  Band1#long_name=GDAL Band Number 1
  Band1#NETCDF_VARNAME=Band1
  transverse_mercator#Northernmost_Northing=-4.04068e+06
  transverse_mercator#Southernmost_Northing=-4.05068e+06
  transverse_mercator#Easternmost_Easting=291385
  transverse_mercator#Westernmost_Easting=281385
  transverse_mercator#spatial_ref=PROJCS["WGS 84 / UTM zone 55N",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433],AUTHORITY["EPSG","4326"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",147],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AUTHORITY["EPSG","32655"]]
  transverse_mercator#GeoTransform=281385 30.03003003003003 0 -4040685 0 -30.03003003003003
  transverse_mercator#grid_mapping_name=transverse_mercator
  transverse_mercator#latitude_of_projection_origin=0.000000e+00
  transverse_mercator#longitude_of_central_meridian=1.470000e+02
  transverse_mercator#scale_factor_at_central_meridian=9.996000e-01
  transverse_mercator#false_easting=5.000000e+05
  transverse_mercator#false_northing=0.000000e+00
  AREA_OR_POINT=Point
Corner Coordinates:
Upper Left  (  281385.000,-4040685.000) (144d33'34.28"E, 36d29'11.56"S)
Lower Left  (  281385.000,-4050685.000) (144d33'24.08"E, 36d34'35.81"S)
Upper Right (  291385.000,-4040685.000) (144d40'15.85"E, 36d29'19.59"S)
Lower Right (  291385.000,-4050685.000) (144d40' 6.12"E, 36d34'43.86"S)
Center      (  286385.000,-4045685.000) (144d36'50.08"E, 36d31'57.75"S)
Band 1 Block=333x1 Type=Byte, ColorInterp=Gray
  NoData Value=0
  Metadata:
    NETCDF_VARNAME=Band1

Attachments (1)

L71092086_08620110129_B50-netCDF-small.nc​ (109.6 KB ) - added by hsumanto 13 years ago.
netCDF file

Download all attachments as: .zip

Change History (8)

by hsumanto, 13 years ago

Attachment: L71092086_08620110129_B50-netCDF-small.nc​ added

netCDF file

comment:1 by Even Rouault, 13 years ago

Cc: Kyle Shannon added

comment:2 by etiennesky, 13 years ago

I have submitted patches that should fix this issue in ticket #2129

comment:3 by etiennesky, 13 years ago

With my fixes, here is the metadata for the first file: 

Metadata:
  NC_GLOBAL#Conventions=CF-1.0
  Band1#grid_mapping=transverse_mercator
  Band1#long_name=GDAL Band Number 1
  transverse_mercator#Northernmost_Northing=-4040685
  transverse_mercator#Southernmost_Northing=-4050685
  transverse_mercator#Easternmost_Easting=291385
  transverse_mercator#Westernmost_Easting=281385
  transverse_mercator#spatial_ref=PROJCS["WGS 84 / UTM zone 55N",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433],AUTHORITY["EPSG","4326"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",147],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AUTHORITY["EPSG","32655"]]
  transverse_mercator#GeoTransform=281385 30.03 0 -4.04068e+06 0 -30.03 
  transverse_mercator#grid_mapping_name=transverse_mercator
  transverse_mercator#latitude_of_projection_origin=0.000000
  transverse_mercator#longitude_of_central_meridian=147.000000
  transverse_mercator#scale_factor_at_central_meridian=0.999600
  transverse_mercator#false_easting=500000.000000
  transverse_mercator#false_northing=0.000000
Corner Coordinates:
Upper Left  (  281385.000,-4040680.000) (144d33'34.28"E, 36d29'11.40"S)
Lower Left  (  281385.000,-4050679.990) (144d33'24.08"E, 36d34'35.65"S)
Upper Right (  291384.990,-4040680.000) (144d40'15.86"E, 36d29'19.43"S)
Lower Right (  291384.990,-4050679.990) (144d40' 6.12"E, 36d34'43.70"S)
Center      (  286384.995,-4045679.995) (144d36'50.09"E, 36d31'57.59"S)
Band 1 Block=333x1 Type=Byte, ColorInterp=Undefined
  NoData Value=-127
  Metadata:
    NETCDF_VARNAME=Band1
    grid_mapping=transverse_mercator
    long_name=GDAL Band Number 1

and for the second: 

Metadata:
  NC_GLOBAL#Conventions=CF-1.0
  Band1#grid_mapping=transverse_mercator
  Band1#long_name=GDAL Band Number 1
  Band1#NETCDF_VARNAME=Band1
  transverse_mercator#Northernmost_Northing=-4040685
  transverse_mercator#Southernmost_Northing=-4050685
  transverse_mercator#Easternmost_Easting=291385
  transverse_mercator#Westernmost_Easting=281385
  transverse_mercator#spatial_ref=PROJCS["WGS 84 / UTM zone 55N",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433],AUTHORITY["EPSG","4326"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",147],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AUTHORITY["EPSG","32655"]]
  transverse_mercator#GeoTransform=281385 30.03 0 -4.04068e+06 0 -30.03 
  transverse_mercator#grid_mapping_name=transverse_mercator
  transverse_mercator#GDAL-Bottom_Up=1
  transverse_mercator#latitude_of_projection_origin=0.000000
  transverse_mercator#longitude_of_central_meridian=147.000000
  transverse_mercator#scale_factor_at_central_meridian=0.999600
  transverse_mercator#false_easting=500000.000000
  transverse_mercator#false_northing=0.000000
Corner Coordinates:
Upper Left  (  281385.000,-4040680.000) (144d33'34.28"E, 36d29'11.40"S)
Lower Left  (  281385.000,-4050679.990) (144d33'24.08"E, 36d34'35.65"S)
Upper Right (  291384.990,-4040680.000) (144d40'15.86"E, 36d29'19.43"S)
Lower Right (  291384.990,-4050679.990) (144d40' 6.12"E, 36d34'43.70"S)
Center      (  286384.995,-4045679.995) (144d36'50.09"E, 36d31'57.59"S)
Band 1 Block=333x1 Type=Byte, ColorInterp=Undefined
  NoData Value=-127
  Metadata:
    NETCDF_VARNAME=Band1
    grid_mapping=transverse_mercator
    long_name=GDAL Band Number 1

Is this ok with you?

comment:4 by hsumanto, 13 years ago

I have tried mod-2-meta.txt patch and it fixes the duplication of metadata and it also fixes the issue where Northernmost_Northing and Southernmost_Northing parameters also seem to loose precision in ticket #4200.

So I am OK with that patch.

This is the metadata for the first one. 

Metadata:
  NC_GLOBAL#Conventions=CF-1.0
  Band1#grid_mapping=transverse_mercator
  Band1#long_name=GDAL Band Number 1
  transverse_mercator#Northernmost_Northing=-4040685
  ...
  transverse_mercator#false_northing=0.000000
  AREA_OR_POINT=Point

This is the metadata for the second one: 

Metadata:
  NC_GLOBAL#Conventions=CF-1.0
  NC_GLOBAL#GDAL_AREA_OR_POINT=Point
  Band1#grid_mapping=transverse_mercator
  Band1#long_name=GDAL Band Number 1
  Band1#NETCDF_VARNAME=Band1
  transverse_mercator#Northernmost_Northing=-4040685
  ...
  transverse_mercator#false_northing=0.000000
  AREA_OR_POINT=Point

So now the only new attributes being introduced for gdal_translate from netCDF to netCDF is only (the diff result between the above gdalinfo): 

  NC_GLOBAL#GDAL_AREA_OR_POINT=Point
  Band1#NETCDF_VARNAME=Band1

comment:5 by hsumanto, 13 years ago

Thank you etiennesky. When will this patch be committed to trunk?

comment:6 by etourigny, 13 years ago

Resolution: → fixed
Status: new → closed

Fixed in trunk (r23094), along with other metadata fixes (#4211).

hsumanto: if you can please test the fix on the original geotiff file attached in #4200 (not the one attached here).

comment:7 by hsumanto, 13 years ago

I have built and tested on the gdal trunk. It fixes the duplication in metadata/attributes issue for that dataset too.

This is the metadata for the first one (geotiff -> netcdf): 

Metadata:
  Band1#grid_mapping=transverse_mercator
  Band1#long_name=GDAL Band Number 1
  NC_GLOBAL#Conventions=CF-1.2
  NC_GLOBAL#GDAL=GDAL 1.9dev, released 2011/01/18
  NC_GLOBAL#GDAL_AREA_OR_POINT=Point
  transverse_mercator#Easternmost_Easting=291385
  transverse_mercator#false_easting=500000.000000
  transverse_mercator#false_northing=0.000000
  transverse_mercator#GeoTransform=281385 30.03003003003003 0 -4040685 0 -30.03003003003003
  transverse_mercator#grid_mapping_name=transverse_mercator
  transverse_mercator#latitude_of_projection_origin=0.000000
  transverse_mercator#longitude_of_central_meridian=147.000000
  transverse_mercator#Northernmost_Northing=-4040685
  transverse_mercator#scale_factor_at_central_meridian=0.999600
  transverse_mercator#Southernmost_Northing=-4050685
  transverse_mercator#spatial_ref=PROJCS["WGS 84 / UTM zone 55N",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433],AUTHORITY["EPSG","4326"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",147],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AUTHORITY["EPSG","32655"]]
  transverse_mercator#Westernmost_Easting=281385

This is the metadata for the second one (netcdf -> netcdf): 

Metadata:
  Band1#grid_mapping=transverse_mercator
  Band1#long_name=GDAL Band Number 1
  NC_GLOBAL#Conventions=CF-1.2
  NC_GLOBAL#GDAL=GDAL 1.9dev, released 2011/01/18
  NC_GLOBAL#GDAL_AREA_OR_POINT=Point
  transverse_mercator#Easternmost_Easting=291385
  transverse_mercator#false_easting=500000.000000
  transverse_mercator#false_northing=0.000000
  transverse_mercator#GeoTransform=281385 30.03003003003003 0 -4040685 0 -30.03003003003003
  transverse_mercator#grid_mapping_name=transverse_mercator
  transverse_mercator#latitude_of_projection_origin=0.000000
  transverse_mercator#longitude_of_central_meridian=147.000000
  transverse_mercator#Northernmost_Northing=-4040685
  transverse_mercator#scale_factor_at_central_meridian=0.999600
  transverse_mercator#Southernmost_Northing=-4050685
  transverse_mercator#spatial_ref=PROJCS["WGS 84 / UTM zone 55N",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433],AUTHORITY["EPSG","4326"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",147],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AUTHORITY["EPSG","32655"]]
  transverse_mercator#Westernmost_Easting=281385

The only thing I noticed when testing the gdal in trunk, the gdalinfo metadata section has changed the order of transverse_mercator attributes.

Note: See TracTickets for help on using tickets.