S57 Driver fails to apply updates to Polygons and breaks them

[boatbeacon]

We are getting Polygon assembly failures when reading US NOAA S57 data with ogr2ogr This results in depth areas amongst other polygon objects not drawing correctly. The failure only occurs when the ENC updates (e.g. .001,002 files etc.) are applied - the base cell converts correctly on its own.

Example command to reproduce the problem:- ogr2ogr -f KML output3.kml US3CA14M/US3CA14M.000 DEPARE -where "lnam='02261193EB3A0A3B'"

I have attached two kml files, output0.kml which is the kml produced when only the base cell (.000) is converted. output4.kml is the broken polygon output when all 4 updates have been applied. I have also attached a gzipped tar of the US3CA14M S57 ENC Cell.

The error/warning output from the command is:-

[esteve@ip-10-212-126-141 ENC_ROOT]$ ogr2ogr --version
GDAL 1.10.0, released 2013/04/24
[esteve@ip-10-212-126-141 ENC_ROOT]$ ogr2ogr -f KML output3.kml US3CA14M/US3CA14M.000 DEPARE -where "lnam='02261193EB3A0A3B'"
Warning 1: Can't find RCNM=130,RCID=2557 for delete.

Warning 1: Can't find RCNM=130,RCID=2558 for delete.

Warning 1: Can't find RCNM=130,RCID=2563 for delete.

Warning 1: Can't find RCNM=130,RCID=2572 for delete.

Warning 1: Can't find RCNM=100,RCID=1217 for delete.

Warning 1: Can't find RCNM=110,RCID=1406 for delete.

Warning 1: Couldn't find spatial record 2688.
Feature OBJL=DEPARE, RCID=0 may have corrupt ormissing geometry.
Warning 1: Couldn't find spatial record 2585.
Feature OBJL=DEPARE, RCID=0 may have corrupt ormissing geometry.
Warning 1: Couldn't find spatial record 2523.
Feature OBJL=DEPARE, RCID=0 may have corrupt ormissing geometry.
Warning 1: Couldn't find spatial record 2654.
Feature OBJL=DEPARE, RCID=0 may have corrupt ormissing geometry.
Warning 1: Couldn't find spatial record 2986.
Feature OBJL=DEPARE, RCID=0 may have corrupt ormissing geometry.
Warning 1: Couldn't find spatial record 3109.
Feature OBJL=DEPARE, RCID=0 may have corrupt ormissing geometry.
Warning 1: Couldn't find spatial record 3004.
Feature OBJL=DEPARE, RCID=0 may have corrupt ormissing geometry.
Warning 1: Couldn't find spatial record 2820.
Feature OBJL=DEPARE, RCID=0 may have corrupt ormissing geometry.
Warning 1: Couldn't find spatial record 2843.
Feature OBJL=DEPARE, RCID=0 may have corrupt ormissing geometry.
Warning 1: Polygon assembly has failed for feature FIDN=294906682,FIDS=2619.
Geometry may be missing or incomplete.
Warning 1: Couldn't find spatial record 2585.
Feature OBJL=DEPARE, RCID=0 may have corrupt ormissing geometry.
Warning 1: Couldn't find spatial record 2688.
Feature OBJL=DEPARE, RCID=0 may have corrupt ormissing geometry.
Warning 1: Polygon assembly has failed for feature FIDN=295168262,FIDS=2602.
Geometry may be missing or incomplete.

[boatbeacon]

Corrrect conversion using .000 data file only

[boatbeacon]

Corrupted polygon when S57 updates applied

[boatbeacon]

US NOAA S57 Cell to test bug with

[Even Rouault]

I've investigated a bit and I believe indeed that the update files are buggy. The object lnam = '02261193EB3A0A3B' in DEPARE (RCID = 326) is modified in update .002. It is completely reset :

Record 28 (64 bytes)
    Field 0001: ISO/IEC 8211 Record Identifier
    Field FRID: Feature record identifier field
        RCNM = 100
        RCID = 326
        PRIM = 3
        GRUP = 1
        OBJL = 42
        RVER = 2
        RUIN = 3
    Field FOID: Feature object identifier field
        AGEN = 550
        FIDN = 294906682
        FIDS = 2619
    Field FSPC: Feature record to spatial record pointer control field
        FSUI = 2
        FSIX = 1
        NSPT = 423

and then recreated :

Record 29 (3493 bytes)
    Field 0001: ISO/IEC 8211 Record Identifier
    Field FRID: Feature record identifier field
        RCNM = 100
        RCID = 326
        PRIM = 3
        GRUP = 1
        OBJL = 42
        RVER = 3
        RUIN = 3
    Field FOID: Feature object identifier field
        AGEN = 550
        FIDN = 294906682
        FIDS = 2619
    Field FSPC: Feature record to spatial record pointer control field
        FSUI = 1
        FSIX = 1
        NSPT = 425
    Field FSPT: Feature record to spatial record pointer field
        NAME = 0x82060E0000 VRID RCNM = 130,RCID = 3590
        ORNT = 1
        USAG = 1
        MASK = 2
        [...]

In the above array there is a reference to

        NAME = 0x82800A0000 VRID RCNM = 130,RCID = 2688
        ORNT = 1
        USAG = 1
        MASK = 2

but that edge (as well as 2585, etc...) are not found in any of the .00? files.

Perhaps the S57 driver could handle better the situation (not sure how !), but I believe there's an issue with the data files themselves. Are there other S57 readers that can display correctly this polygon when the update files are present (and do they display it identically or differently from when there's only the .000 file ?)

[boatbeacon]

I have tried the same S57 Cell using the OpenCPN app and it displays the Depth Areas correctly (DEPARE) though I can't vouch for whether its S57 reader uses and applies the update files. I would imagine it very unlikely that the S57 update files are corrupt or broken as these are the official NOAA ENC Chart updates released to US mariners.

[warmerdam]

fyi +goatbar (Kurt)

[Even Rouault]

I see that OpenCPN uses a forked version of S57 reader... I've discovered that it takes updates into account only if you copy them manually to the ~/.opencpn/SENC directory (and remove the generated .S57 and .BMP files). Once you do that, you get segfaults in OpenCPN renderer when you zoom into the area that causes problems, probably because the reconstructed polygon is invalid...

[Even Rouault]

Another finding : GlobalMapper displays the problematic area incorrectly when the update files are present. Well, it also uses internally the GDAL S57 reader (an old version)....

[Even Rouault]

SeeMyENC 2.0 ( ​http://www.sevencs.com/download ) which does not seem to use the GDAL S57 reader refuses to open the .001 update (and consequently all following updates) due to errors in it.

[boatbeacon]

For Information: The ENC file attached to this bug report is only one example of polygons being broken via updates applied using the S57 driver. We have run all the latest official NOAA ENC S57 files for the state of California through the command and Polygon assembly fails after updates are applied for 123 polygons. The Official NOAA ENC data is used by commercial shipping in SOLAS approved ECDIS Chart plotters. If the NOAA data update files were corrupt or invalid then there would be questions in Congress let alone groundings and collisions at sea. It is highly unlikely that the update files are invalid.

[Even Rouault]

Replying to boatbeacon:

It is highly unlikely that the update files are invalid.

There are perhaps valid, but there's something I must miss in my analysis then. I fail to see what unfortunately. The .002 update file references edges to delete (e.g. 2557) or insert (e.g 2688) that are not found in any of the .000, .001 or .002 files. Some lights from specialists of the S57 format would be welcome.

[boatbeacon]

Hi Rouault

Could you let me know what tool you use to read the update files in human readable form so that I might take a look at them too , to see if I can make any sense out of them or add some more clues?

[Even Rouault]

Could you let me know what tool you use to read the update files in human readable form so that I might take a look at them too , to see if I can make any sense out of them or add some more clues?

I used the 8211view utility that is an internal utility of GDAL not compiled by default. If you can compile GDAL from sources, after having built the GDAL lib, go to the "frmts/iso8211" directory and type "make 8211view" (on Linux) or "nmake /f makefile.vc 8211view.exe" (on Windows)

[Even Rouault]

Removing obsolete milestone

[Even Rouault]

This ticket has been automatically closed because Trac is no longer used for GDAL bug tracking, since the project has migrated to GitHub. If you believe this ticket is still valid, you may file it to ​https://github.com/OSGeo/gdal/issues if it is not already reported there.