S57 driver fails to apply update to lnam_refs correctly

[boatbeacon]

We are using the GDAL OGR S57 driver to extract data from the UKHO ENC data sets. We came across a bug with the extracted data ( a lighthouse with lights not attached) and reported it to the UKHO. They carried out a thorough investigation and narrowed it down to a bug in the OGR S-57 handling of updates. Specifically OGR S57 was applying the update correctly to remove the light objects (4 of them) and add 4 new light objects but it was failing to update the lnam_refs field of the lighthouse (a LNDMRK object) with the lnams of the new light objects. I have attached the feedback from the IC-ENC team below. I have the GB301478 data set available for reproducing the bug with, please contact me for a copy.

Feedback from IC-ENC:- Base cell GB301478: Creates the LNDMARK object and sets up the master-slave association with the four LIGHTS objects you noted GB301478 update2: Creates 4 more LIGHTS objects, deletes the original slave objects then overwrites the feature-feature pointers with the new LIGHTS created in the update. The reason that you cannot find the slave objects and have 4 co-located lights is because the OGR tools are wrongly processing update 2. They are reporting the original slave objects rather than the new ones. The data below shows a summary of the actions for update 2 followed by an XML encoding of the ISO8211 of update2 where you can see the modification of the feature-feature links.

R1[154b-1795] DSID DSSI 
R2[214b-1949] FRID Add  100/48775  LIGHTS FOID [540/620599488/5]  ATTF FSPT 
R3[210b-2163] FRID Add  100/48776  LIGHTS FOID [540/2147483647/6]  ATTF FSPT 
R4[210b-2373] FRID Add  100/48777  LIGHTS FOID [540/1546091932/7]  ATTF FSPT 
R5[210b-2583] FRID Add  100/48774  LIGHTS FOID [540/2147483647/4]  ATTF FSPT 
R6[55b-2793] FRID Del  100/2443  LIGHTS 
R7[55b-2848] FRID Del  100/2444  LIGHTS 
R8[55b-2903] FRID Del  100/2445  LIGHTS 
R9[55b-2958] FRID Del  100/2446  LIGHTS 
R10[137b-3013] FRID Mod  100/2429  LNDMRK FOID [540/2147483647/2508]  FFPC Mod  FFPT
 
<FRID>
    <RCNM>100</RCNM>
    <RCID>2429</RCID>
    <PRIM>1</PRIM>
    <GRUP>2</GRUP>
    <OBJL>74 [LNDMRK] </OBJL>
    <RVER>2</RVER>
    <RUIN>3</RUIN>
</FRID>
<FOID>
    <AGEN>540</AGEN>
    <FIDN>2147483647</FIDN>
    <FIDS>2508</FIDS>
</FOID>
<FFPC>
    <FFUI>3</FFUI>
    <FFIX>1</FFIX>
    <NFPT>4</NFPT>
</FFPC>
<FFPT>
    <LNAM>1C02D88B92C00400</LNAM>
    <RIND>2</RIND>
    <COMT></COMT>
    <LNAM>1C02C098FD240500</LNAM>
    <RIND>2</RIND>
    <COMT></COMT>
    <LNAM>1C02023C9FA50600</LNAM>
    <RIND>2</RIND>
    <COMT></COMT>
    <LNAM>1C029C7D275C0700</LNAM>
    <RIND>2</RIND>
    <COMT></COMT>
</FFPT>

[boatbeacon]

Data set to reproduce the problem

[boatbeacon]

Additional info about the object ID's

The unique S57 object id (lnam) for the lighthouse is 021CC3AEF58409CC and it is classified as a LNDMRK. The Lighthouse claims the following 4 attached objects (lnam_refs). (4:021C082CC0350827,021C0EEA97440828,021CFA43ED4C0829,021C94CCDBD6082A)

Here is the command I used to find what objects the lighthouse had attached

"ogrinfo -al GB301478/2/0/GB301478.000 |grep 021C082CC0350827"

None of these objects appear in the ogrinfo summary of the chart data for GB301478.

There are 4 unattached lights at the same position in GB301478 but their IDs are:-

021CC0928BD80004 (white light) and 021CA59F3C020005, 021CA59F3C020006, 021CA59F3C020007 (red lights)

[warmerdam]

Note that the attachment just gives a contact email to request the actual data.

[boatbeacon]

Replying to warmerdam:

Note that the attachment just gives a contact email to request the actual data.

The UKHO data is proprietary and I can't make it publicly available to everyone but I can provide it for test and debug purposes to anyone looking at the problem - please contact me at snbennett at gmail.com or steve at pocketmariner.com to request the source data.

(N.B. I couldn't find a way of removing the attachment after realising I can't make the data public).

[warmerdam]

It appears that at a minimum the FFPT part of the following update record is not being applied:

Record 10 (113 bytes)
    Field 0001: ISO 8211 Record Identifier
    Field FRID: Feature record identifier field
        RCNM = 100
        RCID = 2429
        PRIM = 1
        GRUP = 2
        OBJL = 74
        RVER = 2
        RUIN = 3
    Field FOID: Feature object identifier field
        AGEN = 540
        FIDN = 3283023236
        FIDS = 2508
    Field FFPC: Feature record to feature object pointer control field
        FFUI = 3
        FFIX = 1
        NFPT = 4
    Field FFPT: Feature record to feature object pointer field
        LNAM = 0x1C02D88B92C00400       FOID AGEN = 540,FIDN = 3230829528,FIDS = 4
        RIND = 2
        COMT = `'
        LNAM = 0x1C02C098FD240500       FOID AGEN = 540,FIDN = 620599488,FIDS = 5
        RIND = 2
        COMT = `'
        LNAM = 0x1C02023C9FA50600       FOID AGEN = 540,FIDN = 2778676226,FIDS = 6
        RIND = 2
        COMT = `'
        LNAM = 0x1C029C7D275C0700       FOID AGEN = 540,FIDN = 1546091932,FIDS = 7
        RIND = 2
        COMT = `'

Digging further...

[warmerdam]

FYI, the FFPC field is also not applied, an omission that was already recognised in S57Reader::ApplyRecordUpdate().

[warmerdam]

I have implemented preliminary support for feature to feature pointer updates in trunk (r25740). It works for this file for "update" updates (replacing the same number of pointers). It is untested for insert or delete operations.

Please add a note here if you find it works or introduces other problems.

It could be back ported to 1.9 but it seems sort of high risk.

[boatbeacon]

I can confirm it works for the defect we came across. Many thanks to warmerdam. I have tested with the original data and eyeballed the lighthouse and it now has its light correctly attached :)