CS-Map RFC 6 - Separate "user-edited definitions" from "system definitions" in CSD files

This page contains an change request (RFC) for the CS-Map Open Source project. More CS-Map RFCs can be found on the RFCs page.

Status

RFC Template Version	(1.0)
Submission Date
Last Modified	(Andre) Timestamp
Author	(Andre)
RFC Status	(draft)
Implementation Status
Proposed Milestone
Assigned PSC guide(s)
Voting History

Overview

CS-MAP’s “geodetic database” is a set of 6 binary catalog (*.CSD) files. These are created out of a set of *.ASC files by the CS_comp.exe compiler tool. By default, all definitions entries generated by this tool are write-protected. However, custom definitions can be added to the files. That is, "system definitions" and "customized definitions" are contained in the same file(s).

This RFC is to add capabilities to CS-MAP, so that custom(ized) definitions are no longer being written into the files produced by CS_comp.exe but are rather kept separated if required by the application hosting CSMAP.lib.

Motivation

Mixing customized definitions with system definitions in 1 file nowadays leads to several problems:

• Sharing customized CS information typically requires sharing entire CSD files.
• Backing up customized definitions requires backing up all CSD files.
• Writing clean installer scripts is more difficult.
• File system privileges are more difficult to administer. Users required to work with customized definitions have to be given write access to files which typically should be accessible by system administrators only.

Applications would benefit from the change proposed by this RFC in the following way:

• Sharing customized CS information becomes way easier. One would only have to share all or a subset of the *.CSD files from the user defined path. No additional tool required that would extract / import customized definitions.
• Backing-up user-defined dictionaries becomes simpler.
• Writing clean installers (scripts) is less difficult. If customized definitions were separated from the system definitions, installer scripts could remove all previously installed files without having to include additional logic to handle modified *.CSD files.
• Applications can more easily be upgraded to a new version of CS-MAP and dictionaries.

Proposed Solution

CS-MAP has its very own “definition protection schema” which can be controlled by application hosting CS-MAP. In principle, all definitions found in a *.CSD file are either...:

• Completely unprotected (must be enabled by the application): That is, even definitions created by the CS_comp.exe tool can be changed.
• Only system definitions are protected (default): Only definitions created by the CS_comp.exe tool are protected
• Custom definitions become protected: System definitions are protected. In addition, customized definitions become protected after a given amount of days where no modifications were made.

The proposed solution is to keep all aforementioned behavior but allow the hosting application to set a directory where all customized definitions should be stored in. In short, the following implementation would have to be done:

• Provide new CS_usrdir() API. This would have to exist in addition to the CS_altdir() API. If set, CS-MAP will store all custom definitions in that directory. New files will be created by CS-MAP, if not yet existing.
• Change all relevant IO API for all types of supported definitions, e.g. [CS_*def()], [CS_*rd()], [CS_*del], … to include the files, if found, from the directory set by [CS_usrdir()]. Note, that all that these methods, while implemented in separate C-Modules, all provide the same file IO functionality. In fact, most of the code is identical. To ease porting all that code to the new behavior, this RFC also aims for implementing a C++ module which would be internal to CS-MAP. That C++ module would centralize all relevant IO routines in templated functions and would expose its functionality through an “extern-C” wrapper module to CS-MAP. CS-MAP already contains other CPP files, too.
• Remove all code that might keep the COORDSYS.CSD, DATUM.CSD and ELIPSOID.CSD files open after 1st usage. There should be one common behavior in how CS-MAP uses its *.CSD files – open, read/write, close. That schema has also been implemented for all new dictionary types.

Implications

Behavioral changes:

None for current clients. Storing custom(ized) definitions in separate *.CSD files must be explicitly enabled via a call to the new [CS_usrdir()]. For clients wishing to use the new capabilities, all changes will be transparent. That is, all API like CS_*def() will start reading definition information from the user-provided dictionary path, and if nothing was found, will continue with the default set of *.CSD files.

API additions:

CS_usrdir()

API removal:

Remove the following static variables from CS-MAP and thus the option for an API client to specify how CS-MAP would keep files open. As aforementioned, CS-MAP would now always close any *.CSD file right after having used it.

csThread short cs_CsStrmFlg = FALSE;
csThread short cs_DtStrmFlg = FALSE;
csThread short cs_ElStrmFlg = FALSE;
 
csThread csFILE* cs_CsStream = 0;
csThread csFILE* cs_DtStream = 0;
csThread csFILE* cs_ElStream = 0;

Addendum

With this RFC being implemented for all types of definitions currently known by CS-MAP (CS, datum, ellipsoid, ..., category), the implementation in CS_category.c will have to be completed. Along with the integration of the new functionality proposed by this RFC, the following work will have to be undertaken in the CS_category module:

• Shield in-memory list from external modifications. That is, make sure, no API client can directly modify the pointers / structures used with CS-MAP's CS_category.c module. Note: This requires updating existing API clients, because pointers returned by CS-MAP will now have to be free'd by the caller.
• Implement missing API for category management.

In particular...

• ...provide API directly in CSMAP to Add, Modify and Remove categories.
• ...implement API to Add, Modify, Remove CS names from categories

Additionally, adapt the implementation in the CS_category.c so it allows for storing category modifications in a file which is separated from the "system" category information file. As before, if no external directly has been set via CS_usrdir(), categories can be modified in any way. That is, there's no distinction being made with what's considered a system definition and a user-defined category.

If, however, a directory has been explicitly defined through CS_usrdir(), (system) categories cannot be edited in the main file that has been read from the directory specified through CS_altdr(). Instead, the information from the main file and the user's additional file will be merged. In detail, the following operations are forbidden or permitted, respectively:

• system categories cannot be removed
• CS names cannot be removed from system categories but new CS names can be added - even to system categories. That information, however, will be written into the separated file
• new categories can be added and freely edited in terms of renaming the categories and also adding, modifying and removing (valid) CS names
• newly added categories can be removed

For example, new CS names can be added to the "Lat Long" category. But pre-defined names like LL84 or LL83 cannot be removed from there.

Test Plan

The following set of unit tests will be implemented:

• Read
• Add / Update
• Delete

Funding/Resources

Supplied by Autodesk.