Changes between Version 5 and Version 6 of CsMapRfc7


Ignore:
Timestamp:
Feb 18, 2014, 12:04:57 PM (10 years ago)
Author:
Norm Olsen
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • CsMapRfc7

    v5 v6  
    5353 3. writing a program to manipulate data (such as a dictionary source file) using the modified tablular data.
    5454
    55 The result is a modification to, for example, a dictionary file which can be examined and tested. Should test results indicate, the program/table is adjusted and the modifictaion process is repeated until the modified data file (typiocally a dictionary source file) satifies the necessary requirements.  In this manner, the modifications to the necessary data files are accomplished with the lowest possible probability of error, typographical or otherwise.  These various programs and reulsting tables are all version controlled in the folder named ConsoleUtilities.
     55The result is a modification to, for example, a dictionary file which can be examined and tested. Should test results indicate, the program/table is adjusted and the modification process is repeated until the modified data file (typically a dictionary source file) satisfies the necessary requirements.  In this manner, the modifications to the necessary data files are accomplished with the lowest possible probability of error, typographical or otherwise.  These various programs and resulting tables are all version controlled in the folder named ConsoleUtilities.
    5656
    57 Using the abovev described technique, the following chores must be accomplished:
     57Using the above described technique, the following chores must be accomplished:
    5858
    5959 * A NSRS2007 version of most all HARN CRS systems must be generated.
    … …  
    6262 * NameMapper entries for all new systems must be generated.
    6363 * Entries for the EPSG names and EPSG code must be added to the NameMapper and properly associated with the new CS-MAP NSRS CRS definition.
    64  * Extries for the ESRI names and ESRI codes must be added to the NameMapper and properly associated with the new CS-MAP NSRS CRS definitions and the EPSG Names and codes.
     64 * Entries for the ESRI names and ESRI codes must be added to the NameMapper and properly associated with the new CS-MAP NSRS CRS definitions and the EPSG Names and codes.
    6565 * To perform that immediately above, programs which manipulate ESRI projection files (i.e. the WKT files with the .prj extension) need to written to properly extract ESRI names and code.
    6666 * Programs which will properly match EPSG, ESRI, and CS-MAP definition names and codes need to be written.
    6767
    68 Having accomplished all of the above, NSRS2007 and NSRS 2011 will have been implemented, but with one major issue remaining.
     68Having accomplished all of the above, NSRS2007 and NSRS2011 will have been implemented, but with one major issue remaining.
    6969
    7070=== The HPGN Problem ===
    7171
    72 As indicated above, conversion from NAD83 to NSRS2007 first requires a conversion to HARN (aka NAD83/96, HPGN, NAD83/91, etc.).  Conversion from NAD83 to HARN is accomplished through the use of grid interpolation files with names in the form of ''??hpgn.l?s'' where the first two characters are generally replaced with a two characetr state code. For example, two data files named ''cohpgn.las'' and ''cohpgn.los'' define the shift required to convert from NAD83 to HARN in the state of Colorado.  The .las file carries the latitude shift values, while the .los file carries the longitude shift values. The cmplicating factor here is that these data file naturally overlap and the shifts carries in the overlapping files are not the same in the regions of overlap.  Thus, for geography covered by the regions of overlap there are indeed two (or more) officially designated shift values.
     72As indicated above, conversion from NAD83 to NSRS2007 first requires a conversion to HARN (aka NAD83/96, HPGN, NAD83/91, etc.).  Conversion from NAD83 to HARN is accomplished through the use of grid interpolation files with names in the form of ''??hpgn.l?s'' where the first two characters are generally replaced with a two character state code. For example, two data files named ''cohpgn.las'' and ''cohpgn.los'' define the shift required to convert from NAD83 to HARN in the state of Colorado.  The .las file carries the latitude shift values, while the .los file carries the longitude shift values. The complicating factor here is that these data file sets naturally overlap and the shifts carried in the overlapping files are not the same in the regions of overlap.  Thus, for geography covered by the regions of overlap there are indeed two (or more) officially designated shift values.
    7373
    74 This is why in release 13.0 of CS-MAP, the original HARN datum was replaced by some 40+ distinct datums named HARN/??.  Each of these new datums essentially provided the end user with the means to address a very specific set of HPGN data files.  This capability was greeted with much appreciation for those working geography in the regions of HPGN file overlaps.
     74This is why in release 13.0 of CS-MAP, the original HARN datum was replaced by some 40+ distinct datums named HARN/??.  Each of these new datums essentially provided the end user with the means to address a very specific set of HPGN data files.  This capability was greeted with much appreciation for those working geography in the regions of HPGN file overlap.
    7575
    76 With the introduction of NSRS2007, however, we now have yet another problem.  At the datum shoft calculation level, we only have geographic coordinates to work with.  If such geographic coordinates came from the inverse projection conversion of a state plane CRS is unknown at this level.  Thus, the conversion of, say, NAD83 to NSRS2007 presents the problem of which specific set of HPGN data files should be used for any given coordinate. There is no clear, unambiguous solution to this problem.
     76With the introduction of NSRS2007, however, we now have yet another problem.  At the datum shift calculation level, we only have geographic coordinates to work with.  If such geographic coordinates came from the inverse projection conversion of a state plane CRS is unknown at this level.  Thus, the conversion of, say, NAD83 to NSRS2007 presents the problem of which specific set of HPGN data files should be used for any given coordinate. There is no clear, unambiguous, solution to this problem.
    7777
    7878=== The 48hpgn.l?s File ===
    7979
    80 It is proposed that this problem be solved by the generation of a new ''totally unofficial'' set of HPGN files which models the NAD83 <--> HARN datum shift for all 48 conterminous states.  This file to be produced using the followoing concepts:
    81  * All HPGN daum shift files have a standard grid cell size of 15 minutes of latitude/longitude.
    82  * Fairly definitive state boundaries are available from teh EPSG database; certainly definitive enough to predict in which state any 15 minute Lat/long coordinate resides in.
    83  * We then establish a HPGN grid file set named 48hpgn.las and 48hpgn.los which covers the entirity of the 48 conterminous states and has a 15 minute grid cell size.
    84  * For each data point in the 48HPGN dta file set:
     80It is proposed that this problem be addressed by the generation of a new ''totally unofficial'' set of HPGN files which model the NAD83 <--> HARN datum shift for all 48 conterminous states.  This file to be produced using the following concepts:
     81 * All HPGN datum shift files have a standard grid cell size of 15 minutes of latitude/longitude.
     82 * Fairly definitive state boundaries are available from the EPSG database; certainly definitive enough to predict in which state any 15 minute Lat/long coordinate resides in.
     83 * We then establish a HPGN grid file set named 48hpgn.las and 48hpgn.los which covers the entirety of the 48 conterminous states and has a 15 minute grid cell size.
     84 * For each data point in the 48HPGN data file set:
    8585  * Determine the state in which the point lies
    8686  * Determine the specific HPGN data file set from which values are to be extracted,
    8787  * Extract the appropriate data values and insert into the 48HPGN data file set being created.
    8888
    89 This process will succedd in generating valid coverage for most all of the 48 state geography.  Problems occur where there are substantial waterways between states which are not included in the state boundary polygons; CHesapeak Bay, and the Great Lakes for example. These situation need to be detected and than manually dealt with.
     89This process will succeed in generating valid coverage for most all of the 48 state geography.  Problems occur where there are substantial waterways between states which are not included in the state boundary polygons; Chesapeake Bay, and the Great Lakes for example. These situations need to be detected and then specifically dealt with.
    9090
    9191=== Additional Complications ===
    … …  
    9696  * the nehpgn.l?s data file set covers VT, NH, MA, and CT; i.e. New England (sans Maine)
    9797  * the mdhpgn.l?s data file set convers both Maryland and Deleware.
    98   * the hihpgn.l?s files are for Hawaii, but don't overlapo anything, so there is no problem.
    99  2> There are three states for which there are two HPGN data file sets:
    100   * The cshpgn.l?s and cnhpgn.l?s files cover southern and northern California repectively.
     98 2> There are four situations for which there are two HPGN data file sets for a given political entity:
     99  * The cshpgn.l?s and cnhpgn.l?s files cover southern and northern California respectively.
    101100  * the emhpgn.l?s and wmhpgn.l?s files cover eastern and western Montana respectively.
    102101  * the ethpgn.l?s and wthpgn.l?s files cover eastern and western Texas respectively.
    103   * the eshpgn.l?s and wshpgn.l?s files cover eastern and western Ammerican Samoa repectively.  These files oonly overlap with themselves, and the region of overlap is all Pacific Ocean, so these file do not present a problem.
     102  * the eshpgn.l?s and wshpgn.l?s files cover eastern and western American Samoa respectively.  These files only overlap with themselves, and the region of overlap is all Pacific Ocean, so these files do not present a specific problem.
    104103
    105 Of course, in the case of multiple HPGN data file sets covering a single state the files overlap, and yes the grid shift values in the regions of overlap are not the same.  It is now known, for example, in the region of overlap in Montana, the differences between the two files is as much as 17 centimeters.
     104Of course, in the case of multiple HPGN data file sets covering a single state the files overlap, and (of course) the grid shift values in the regions of overlap are not the same.  It is now known, for example, in the region of overlap in Montana, the differences between the two files is as much as 17 centimeters at a certain point.
    106105
    107106=== Result Evaluation ===
    108107
    109 In regions internal to a specific state, the 48HPGN file solution produces results basically identical with the previous releases.  The exact same interpolation code is used on the same data values.  However, when one approaches the the boundary of any specific HPGN region (i.e. thegeography where HPGN data file overlap) the results can be as much as 10 centimeters different.
     108In regions internal to a specific state, the 48HPGN file solution produces results basically identical with the previous releases.  The exact same interpolation code is used on the same data values.  However, when one approaches the the boundary of any specific HPGN region (i.e. the geography where HPGN data file sets overlap) the results can be as much as 10 centimeters different.
    110109
    111 The best example of this condition is the four corneres area of the southwestern United States, a point of geography where four state borders come together and there are four different sets of HPGN data files which cover the intersection point.  Thus there are four separate values of the shift at that point.  The Colorado and Uath numbers, while not identical, are very close.  The Arizona and New Mexico numbers are very close.  Howevere, there is a 10 centimeter difference between the two sets of numbers.  Who is to say which is correct?
     110The best example of this condition is the four corners area of the southwestern United States, a point of geography where four state borders come intersect and there are four different sets of HPGN data files which cover the intersection point.  Thus there are four separate values of the shift at that point.  The Colorado and Utah numbers, while not identical, are very close.  The Arizona and New Mexico numbers are very close.  However, there is a 10 centimeter difference between the two sets of numbers.  Who is to say which is correct?
    112111
    113112== Testing ==
    114113
    115 Testing of the horizontal shifting at the geodetic coordinate level (i.e. lat/longs) will be accomplished using data points generated by the NGS supplied executable geocon.exe.  This program accepts input data, and produces output data, in what is known as Blue Book format.  In order to facilitate the generation of test data in the form used by the CS-MAP ConsoleTestCpp program, a TcsBlueBook object has been developed and program which produce test point data in this format, and the eventually convert the Blue Book data produced by geocon.exe into the TEST.DAT form have also been written.
     114Testing of the horizontal shifting at the geodetic coordinate level (i.e. lat/longs) will be accomplished using data points generated by the NGS supplied executable geocon.exe.  This program accepts input data, and produces output data, in what is known as Blue Book format.  In order to facilitate the generation of test data in the form used by the CS-MAP ConsoleTestCpp program, a TcsBlueBook object has been developed and program which produces test point data in this format, and eventually converts the Blue Book data produced by geocon.exe into the TEST.DAT form have also been written.
    116115
    117 Generally, the results match at the +/- 2 millimeter level.  Precision is difficult to maintain in this testing environment as the Blue Book format only supports five didgits of precision at the arc-second level.  Thus, verifying results at that low precision value get into trying to make sure ASCII to real and real to ASCII conversions, along with conversions from/to degrees, minutes, and seconds are now all part of the verification issue.
     116Generally, the results match at the +/- 2 millimeter level.  Precision is difficult to maintain in this testing environment as the Blue Book format only supports five digits of precision at the arc-second level.  Thus, verifying results at that low precision value involves getting into trying to make sure ASCII to real and real to ASCII conversions, along with conversions from/to degrees, minutes, and seconds are consistent across the various platforms.  Note that the geocon.exe source code is available, but the program is written in FORTRAN, so the code is of little value other than analysis.
    118117
    119 Tests generated for inclusion in the standard TEST.DAT file used by ConsoleTestCpp do not test vertical components (a weakness which deserves attention).  Thus, testing the vertical component of the NSRS2007/NSRS2011 implementation needs to be done manually.
     118Tests generated for inclusion in the standard TEST.DAT file used by ConsoleTestCpp do not test vertical components (a weakness which deserves attention).  Thus, testing the vertical component of the NSRS2007/NSRS2011 implementation needs to be done separately.
    120119
    121120== Implications ==
    … …  
    123122The implementation as is currently being tested is maintain in the CS-MAP Subversion repository as a ''sandbox'' named NSRS2007.
    124123
    125 This implementation leaves all of the precious HARN/?? datum shifts in place.  It introduces a new datum named NAD83/HARN.  The transformations referneced to this new datum are the only transformations which reference the new 48HPGN data set. End users which have the need to have a specific set of HPGN grid shift files can achieve this by simply placing an entry for the superceeding HPGN data file set ahead of the 48HPGN data file set reference in the Geodetic Transformation Dictionary.  Thus, a user working in the Four Corners area of the southwest, can specificy that, for example, the Arizona datum shift values are to prevail can do so  by entering a reference to azhpgn,l?s ahead of the reference to 48hpgn.l?s.  Editing the Geodtic Transformation Dictionary is recognized as not being an ideal solution, but given the nature of the beast, it is the best available short of adding the possibility of user interaction required when certain coordinate conversions are selected.
     124This implementation leaves all of the precious HARN/?? datum shifts in place.  It introduces a new datum named NAD83/HARN.  The transformations referenced to this new datum are the only transformations which reference the new 48HPGN data set. End users which have the need to have a specific set of HPGN grid shift files can achieve this by simply placing an entry for the superseding HPGN data file set ahead of the 48HPGN data file set reference in the Geodetic Transformation Dictionary.  Thus, a user working in the Four Corners area of the southwest, can specify that, for example, the Arizona datum shift values are to prevail can do so  by entering a reference to azhpgn.l?s ahead of the reference to 48hpgn.l?s.  Editing the Geodetic Transformation Dictionary is recognized as not being an ideal solution, but given the nature of the beast, it is the best available short of adding the possibility of user interaction required when certain coordinate conversions are selected.
    126125
    127126== Funding and Resources ==