Opened 11 years ago

Closed 3 years ago

#4378 closed defect (wontfix)

Find and fix errors in ESRI WKT morphing

Reported by: etourigny Owned by: etourigny
Priority: normal Milestone: closed_because_of_github_migration
Component: OGR_SRS Version: unspecified
Severity: normal Keywords: esri wkt morph
Cc: warmerdam, Kyle Shannon


ESRI WKT morphing magic needs some improvement and testing with actual data. Many ESRI EKT .prj files have been collected in #4345 .

Added simple autotest to osr/ and support data in r23510.

For each input ESRI WKT, the test morphs from and to ESRI and checks for matching SRS (with IsSame()) and also for matching WKT (non fatal as almost all fail this test due to precision differences).

Attaching first results, there are quite a few errors, mostly due to different or missing datums.

Attachments (6)

esri_gcs_srs.txt (8.7 KB ) - added by etourigny 11 years ago.
esri_pcs_srs.txt (41.9 KB ) - added by etourigny 11 years ago.
log file with errors in PCS defifnitions (4.3 KB ) - added by etourigny 11 years ago.
esri_datum_override.csv (514 bytes ) - added by etourigny 11 years ago.
esri_gcs.csv (147.7 KB ) - added by etourigny 11 years ago.
gdal_datum.csv (182.1 KB ) - added by etourigny 11 years ago.

Download all attachments as: .zip

Change History (18)

by etourigny, 11 years ago

Attachment: esri_gcs_srs.txt added

by etourigny, 11 years ago

Attachment: esri_pcs_srs.txt added

log file with errors in PCS defifnitions

comment:1 by etourigny, 11 years ago

Status: newassigned

comment:2 by etourigny, 11 years ago

Test files in osr/data have been obtained from ESRI, transformed to a CSV file (semi-colon separator) and gzipped for space. The file data/esri_extra.csv is to add any other definitions.

comment:3 by Even Rouault, 11 years ago

data/esri_extra.csv has not been commited.

And I get 'fail' tests. Perhaps due to the fact that you have local changes in your ogr/ogr_srs_esri.cpp

  TEST: osr_esri_21 ... WARNING: Failed 738 WKT tests, see file tmp/esri_gcs_wkt.txt
    line 824: ERROR: Failed 26 SRS tests, see file tmp/esri_gcs_srs.txt
  TEST: osr_esri_22 ... WARNING: Failed 4268 WKT tests, see file tmp/esri_pcs_wkt.txt
    line 824: ERROR: Failed 48 SRS tests, see file tmp/esri_pcs_srs.txt
  TEST: osr_esri_23 ... fail
    line 767: input file data/esri_extra.csv does not exist

Test Script: osr_esri
Succeeded: 20
Failed:    3 (0 blew exceptions)
Skipped:   0
Expected fail:0

And the _wtk.txt files are always empty :

-rw-r--r-- 1 even even  8926 2011-12-10 18:18 esri_gcs_srs.txt
-rw-r--r-- 1 even even     0 2011-12-10 18:18 esri_gcs_wkt.txt
-rw-r--r-- 1 even even 42953 2011-12-10 18:18 esri_pcs_srs.txt
-rw-r--r-- 1 even even     0 2011-12-10 18:18 esri_pcs_wkt.txt

Seems to be due to ofile_wkt file handle being only opened and closed, but never written

comment:4 by etourigny, 11 years ago

Fixed in r23511 .

Status returned is now 'expected_fail' when at least one SRS does not match. Note that these errors have not been fixed yet!

comment:5 by etourigny, 11 years ago

r23520 : manage special cases of PROJECTION parameters that have multiple mappings. This fixes many apparent errors due to multiple mappings of PROJECTION names, namely: Transverse_Mercator/Gauss_Kruger , Equidistant_Cylindrical/Plate_Carree , Hotine_Oblique_Mercator_Azimuth_Natural_Origin/Hotine_Oblique_Mercator_Azimuth_Center.


ERROR: SRS not matching for # 53001

r23521: remap node if new value is not empty OGR_SRSNode::applyRemapper now only applies the remapped value if it is not "". This fixes many bugs where the DATUM value was set to 'D_' after morphFromESRI and threre is no corresponding ESRI_DATUM_NAME entry in gdal_datum.csv. Ideally these known datums should be added to gdal_datum.csv for completeness.


ERROR: SRS not matching for # 2989

comment:6 by etourigny, 11 years ago

Cc: warmerdam added

r23522: fix ESRI_DATUM_NAME for D_MOLDREF99 and D_Philippine_Reference_System_1992.

2 Entries in gdal_datum.csv were erroneous.

D_Phillipine_Reference_System_1992 was a typo ( should be D_Philippine_Reference_System_1992).

According to Moldref 99 uses D_MOLDREF99 and not D_International_1967:


To Frank: do any changes in *.csv files have to be made elsewhere so they don't get overwritten by a future update of the EPSG database?

Also: There were a number of missing ESRI_DATUM_NAME entries that this test showed (masked by fix in r23521), should I also add them to gdal_datum.csv ?

comment:7 by etourigny, 11 years ago

r23524: add test OGC->ESRI->OGC from EPSG codes and cleanup ESRI->OGC->ESRI tests

The new test fails for 20 EPSG codes in gcs.csv and 109 codes in pcs.csv , due to differences in datum names when 1 ESRI datum name maps to many datums in gdal_datum.csv .

Perhaps gdal_datum.csv should be revised and only allow 1 instance per ESRI Datum name?

For Example, D_Bogota maps to 2 named datums:

in tmp/epsg_gcs_srs.txt:

ERROR: SRS not matching for # 4802
GEOGCS["Bogota 1975 (Bogota)",DATUM["Bogota_1975_Bogota",SPHEROID["International 1924",6378388,297]],PRIMEM["Bogota",-74.08091666666667],UNIT["degree",0.0174532925199433]]
GEOGCS["Bogota 1975 (Bogota)",DATUM["Bogota_1975",SPHEROID["International_1924",6378388,297]],PRIMEM["Bogota",-74.08091666666667],UNIT["Degree",0.017453292519943295]]

in gdal_datum.csv:

6218,Bogota 1975,geodetic,"Fundamental point: Bogota observatory. Latitude: 4°35'56.570""N, longitude: 74°04'51.300""W (of Greenwich).",1975,7022,8901,3686,Topographic mapping.,"Replaces 1951 adjustment. Replaced by MAGNA-SIRGAS (datum code 6685).","Instituto Geografico Agustin Codazzi (IGAC) special publication no. 1, 4th edition (1975) ""Geodesia: Resultados Definitvos de Parte de las Redes Geodesicas Establecidas en el Pais"".",OGP,"2010/03/30",2000.200 2005.060 2007.060 2008.045 2009.106,0,"D_Bogota"
6802,"Bogota 1975 (Bogota)",geodetic,"Fundamental point: Bogota observatory. Latitude: 4°35'56.570""N, longitude: 0°E (of Bogota).",1975,7022,8904,3229,Topographic mapping.,,,OGP,"2008/06/24",2000.200 2003.361 2008.045,0,"D_Bogota"

comment:8 by Even Rouault, 11 years ago

Not sure if Frank follows closely this ticket, but yes indeed gdal_datum.csv is refreshed during the process described in

comment:9 by etourigny, 11 years ago

r23541 : manage special cases in osr_esri_test_ogc_esri_ogc (missing rectified_grid_angle or scale_factor)

This fixes a few apparent errors, which are due to differences in GDAL vs. ESRI projection parameters.

r23542 : add optional fixing of TOWGS84, DATUM and GEOGCS

This fixes problem described above, when ESRI uses the same datum name for 2 different EPSG named datums (e.g. D_Bogota => Bogota_1975_Bogota and Bogota_1975). For now functionality is not active by default, so created a new test with GDAL_FIX_ESRI_WKT=DATUM which throws no errors.

This patch also addresses issues in bug #4345).

comment:10 by etourigny, 11 years ago

r23558 : add tests for EPSG codes

Reveals quite a few missing ESRI_DATUM_NAME in gdal_datum.csv .

I generated a new gdal_datum.csv using a slightly modified and file esri_gcs.csv derived from gcs.htm .

When that new gdal_datum.csv is used, there are less errors (104 for test osr_esri_21 and 58 for test osr_esri_23)

Attaching the modified files for reference, gdal_datum.csv has less missing ESRI datums. Ideally more should be added to esri_datum_override.csv to fill all missing data (58 from osr_esri_23).

by etourigny, 11 years ago

Attachment: added

by etourigny, 11 years ago

Attachment: esri_datum_override.csv added

by etourigny, 11 years ago

Attachment: esri_gcs.csv added

by etourigny, 11 years ago

Attachment: gdal_datum.csv added

comment:11 by Kyle Shannon, 8 years ago

Cc: Kyle Shannon added

comment:12 by Even Rouault, 3 years ago

Milestone: closed_because_of_github_migration
Resolution: wontfix
Status: assignedclosed

This ticket has been automatically closed because Trac is no longer used for GDAL bug tracking, since the project has migrated to GitHub. If you believe this ticket is still valid, you may file it to if it is not already reported there.

Note: See TracTickets for help on using tickets.