Opened 21 years ago
Last modified 21 years ago
#233 closed defect (fixed)
missing attributes in OGR NTF meridian-2 => shapefile
| Reported by: | Owned by: | warmerdam | |
|---|---|---|---|
| Priority: | high | Milestone: | |
| Component: | default | Version: | unspecified |
| Severity: | critical | Keywords: | |
| Cc: |
Description
I'm converting OS Meridian-2 NTF files to ESRI shapefile but I think some of the text
attributes are going missing, particularly proper names for lines (roads).
I've appended an excerpt from an NTF file. It looks as though proper names (PN) are
detected for POINTs (see MERIDIAN_POINT listing below) but not for LINEs (see
MERIDIAN_LINE listing below). My reading of the user guide (http://
www.ordnancesurvey.co.uk/downloads/vector/meridian/Meridian_guide_3_1.pdf)
suggests that proper names can occur for lines as well as points. Grepping for the proper
name strings in the results of converting excerpted or full NTF files doesn't find names
that appear in the NTF (eg, "CASTLEBANK").
One possibility is that OGR is parsing an older version of Meridian, implied by old user
guide at OGR WWW site, and inclusion of 2 in Meridian-2 name??? I've submitted this
message to GDAL bugzilla in case this is right.
Another possibility is that I'm doing something wrong, or don't understand the output (inc
which case I'll delete the buzilla report).
thanks
Ian
=== EXCERPT FROM NTF FILE ===
01OS-EDINA-Digimap glacsianm 2002102100000130200V \0%
02Meridian_02.01 DEFAULT_02.00 19920515 000000001%
00Meridian_02.00 20000901 00000000000%
40FC004I4 FEATURE CODE\NUMERIC FEATURE CODE\0%
40OD013A13 OSODR\ORDNANCE SURVEY OSCAR DATA REFERENCE\0%
40JN A* JUNCTION_NAME\Name Of Road Junction\0%
40LL005I5 LINE_LENGTH\Length Of Line Feature\0%
40NP002I2 NUM_PARENTS\Number Of Parent OSODRs\0%
40PO013A13 PARENT_OSODR\Parent OSODR\0%
40RT001A1 ROUNDABOUT\Roundabout Indicator\0%
40RN A* ROAD_NUM\DoT Route Number\0%
40SN A* SETTLEMENT_NAME\Settlement Name\0%
40TR001A1 TRUNK_ROAD\Trunk Road Indicator\0%
40LC006I6 LEFT COUNTY\LEFT COUNTY INDICATOR\0%
40RC006I6 RIGHT COUNTY\RIGHT COUNTY INDICATOR\0%
40LD006I6 LEFT DISTRICT\LEFT DISTRICT INDICATOR\0%
40RD006I6 RIGHT DISTRICT\RIGHT DISTRICT INDICATOR\0%
40NM A* ADMIN NAME\ADMINISTRATIVE AREA NAME\0%
40PI006I6 GLOBAL ID\ADMIN AREA GLOBAL IDENTIFIER\0%
40DA013A13 DLUA ID\DLUA IDENTIFIER\0%
40PN A* PROPER NAME\DEFINITIVE NAME\0%
40RI013A13 RAIL ID\RAILWAY IDENTIFIER\0%
40SI013A13 STATION ID\STATION IDENTIFIER\0%
40TX A* TEXT\INDEPENDENT TEXT\0%
40FA013A13 FOREST ID\FOREST IDENTIFIER\0%
40WA013A13 WATER AREA ID\AREA WATER IDENTIFIER\0%
40WI013A13 WATER LINK ID\LINEAR WATER IDENTIFIER\0%
40HT008I8 HEIGHT\DERIVED HEIGHT IN METRES\0%
...FEAT_CODE listing deleted...
...90* lines deleted...
23000002000002010000020%
21000002200040570506322 0571606292 0563606276 0548806279 0%
14000002ODO3FPHP6MY4LYVFC3004LL00262PNCASTLEBANK STREET\
NP03POO1FPHP6MY4LYVPO1%
00O1YPGCAMVCFXXPOO1FPHNB5VQVYV0%
23000003000003010000030%
21000003200020589007483 0585207499 0%
14000003ODO3FPF6U5NE9FVFC3004LL00042NP01POO1FPF6U5NE9FV0%
23000004000004010000040%
21000004200020658406907 0657006883 0%
14000004ODO3FPFKL5UJHFVFC3004LL00028NP01POO1FPFKL5UJHFV0%
23000005000005010000050%
21000005200030796005614 0794805636 0771205639 0%
14000005ODO3FP5DVMYG0YVFC3004LL00266PNST VINCENT TERRACE\
NP01POO1FP5DVMYG0YV0%
23000006000006010000060%
21000006200020876706271 0878406273 0%
14000006ODO3FP12BMVC9FVFC3004LL00058NP03POO1FP12BMVC9FVPOO1FP0
YBMVACFVPO1%
00O1FP16XMV9TYV0%
=== OGRINFO MERIDIAN_LINE OUTPUT ===
Layer name: MERIDIAN_LINE
Geometry: Line String
Feature Count: 5
Extent: (255488.000000, 665614.000000) - (258784.000000, 667499.000000)
Layer SRS WKT:
PROJCS["OSGB 1936 / British National Grid",
GEOGCS["OSGB 1936",
DATUM["OSGB_1936",
SPHEROID["Airy 1830",6377563.396,299.3249646]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",49],
PARAMETER["central_meridian",-2],
PARAMETER["scale_factor",0.999601272],
PARAMETER["false_easting",400000],
PARAMETER["false_northing",-100000],
UNIT["metre",1]]
LINE_ID: Integer (6.0)
FEAT_CODE: String (4.0)
GEOM_ID: Integer (6.0)
OSMDR: String (13.0)
ROAD_NUM: String (80.0)
TRUNK_ROAD: String (1.0)
RAIL_ID: String (13.0)
LEFT_COUNT: Integer (6.0)
RIGHT_COUN: Integer (6.0)
LEFT_DISTR: Integer (6.0)
RIGHT_DIST: Integer (6.0)
TILE_REF: String (10.0)
OGRFeature(MERIDIAN_LINE):0
LINE_ID (Integer) = 2
FEAT_CODE (String) = 3004
GEOM_ID (Integer) = 2
OSMDR (String) = (null)
ROAD_NUM (String) = (null)
TRUNK_ROAD (String) = (null)
RAIL_ID (String) = (null)
LEFT_COUNT (Integer) = (null)
RIGHT_COUN (Integer) = (null)
LEFT_DISTR (Integer) = (null)
RIGHT_DIST (Integer) = (null)
TILE_REF (String) = NS56
LINESTRING (255705 666322,255716 666292,255636 666276,255488 666279)
OGRFeature(MERIDIAN_LINE):1
LINE_ID (Integer) = 3
FEAT_CODE (String) = 3004
GEOM_ID (Integer) = 3
OSMDR (String) = (null)
ROAD_NUM (String) = (null)
TRUNK_ROAD (String) = (null)
RAIL_ID (String) = (null)
LEFT_COUNT (Integer) = (null)
RIGHT_COUN (Integer) = (null)
LEFT_DISTR (Integer) = (null)
RIGHT_DIST (Integer) = (null)
TILE_REF (String) = NS56
LINESTRING (255890 667483,255852 667499)
... features 2-4 deleted...
=== OGRINFO MERIDIAN_LINE OUTPUT ===
Layer name: MERIDIAN_POINT
Geometry: Point
Feature Count: 0
Extent: (0.000000, 0.000000) - (0.000000, 0.000000)
Layer SRS WKT:
PROJCS["OSGB 1936 / British National Grid",
GEOGCS["OSGB 1936",
DATUM["OSGB_1936",
SPHEROID["Airy 1830",6377563.396,299.3249646]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",49],
PARAMETER["central_meridian",-2],
PARAMETER["scale_factor",0.999601272],
PARAMETER["false_easting",400000],
PARAMETER["false_northing",-100000],
UNIT["metre",1]]
POINT_ID: Integer (6.0)
GEOM_ID: Integer (6.0)
FEAT_CODE: String (4.0)
PROPER_NAM: String (80.0)
OSMDR: String (13.0)
JUNCTION_N: String (80.0)
ROUNDABOUT: String (1.0)
STATION_ID: String (13.0)
GLOBAL_ID: Integer (6.0)
ADMIN_NAME: String (80.0)
DA_DLUA_ID: String (13.0)
TILE_REF: String (10.0)
=== OGRINFO MERIDIAN_NODE OUTPUT ===
Layer name: MERIDIAN_NODE
Geometry: None
Feature Count: 0
Layer SRS WKT:
PROJCS["OSGB 1936 / British National Grid",
GEOGCS["OSGB 1936",
DATUM["OSGB_1936",
SPHEROID["Airy 1830",6377563.396,299.3249646]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",49],
PARAMETER["central_meridian",-2],
PARAMETER["scale_factor",0.999601272],
PARAMETER["false_easting",400000],
PARAMETER["false_northing",-100000],
UNIT["metre",1]]
NODE_ID: Integer (6.0)
GEOM_ID_OF: Integer (6.0)
NUM_LINKS: Integer (4.0)
TILE_REF: String (10.0)
=== OGRINFO MERIDIAN_TEXT OUTPUT ===
Layer name: MERIDIAN_TEXT
Geometry: Point
Feature Count: 0
Extent: (0.000000, 0.000000) - (0.000000, 0.000000)
Layer SRS WKT:
PROJCS["OSGB 1936 / British National Grid",
GEOGCS["OSGB 1936",
DATUM["OSGB_1936",
SPHEROID["Airy 1830",6377563.396,299.3249646]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433]],
PROJECTION["Transverse_Mercator"],
PARAMETER["latitude_of_origin",49],
PARAMETER["central_meridian",-2],
PARAMETER["scale_factor",0.999601272],
PARAMETER["false_easting",400000],
PARAMETER["false_northing",-100000],
UNIT["metre",1]]
TEXT_ID: Integer (6.0)
FEAT_CODE: String (4.0)
FONT: Integer (4.0)
TEXT_HT: Real (5.1)
DIG_POSTN: Integer (1.0)
ORIENT: Real (5.1)
TEXT: String (80.0)
TEXT_HT_GR: Real (10.3)
TILE_REF: String (10.0)
Change History (6)
comment:2 by , 21 years ago
Can you attach a sample NTF file, or email it directly to me?
The NTF file I excerpted in my posts is at:
http://www.dcs.gla.ac.uk/equator/ntf/meridian-ns56-20020101.ntf
The problem is (I assume) that I create a fixed schema for "known" Ordnance
Survey products, and the fixed schema for Meridian only includes the proper
name on the point layer. It should be easy to fix this with a sample though
I have to wonder what else is missing from my fixed schemas.
I had a look to see if I could work out where the schemas lived with a view to
messing with them, but I didn't spend long on it.
By the way, my sample Meridian data start with:
01ORDNANCE SURVEY 1996040300000130200V \0%
02Meridian_01.95 DEFAULT_02.00 19920515 000000001%
00Meridian_01.00 19950901 00000000000%
The excerpt suggests my data is Meridian-2 (as does the cover of the user guide)
so maybe there's been a change in the fixed schemas.
The other thing you could try is forcing the reader into "generic" mode
where it actually inspects what attributes occur on all features before
setting up the schema for the layers. This can be done by setting the
environment variable "OGR_NTF_OPTIONS" to "FORCE_GENERIC=ON" before running
ogr2ogr or ogrinfo.
Interesting. The PNs now appear in the LINE.dbf which is promising, though I'm
yet to check programmatic access to them. It's promising, though.
Also worth noting that the file size of POINT.dbf has also gone up, and that
there's now a NODE.shp (which doesn't seem to get generated with the
schema-based translation.
Another difference is that I can now load the LINE data into postgis, which I
couldn't do with MERIDIAN_LINE (returns "fseek(444800) failed on DBF file").
Also, probably trivially, without the option set I get 3 instances of
ERROR 6: Can't create fields of type IntegerList on shapefile layers.
and 1 of
ERROR 6: Can't create fields of type RealList on shapefile layers.
With the option I get 2 instances of the first message and none of the second.
thanks
Ian
PS Let me know if you want me to post any of this to bugzilla
comment:3 by , 21 years ago
I have committed support to CVS for MERIDIAN 2 product. Amoung other things I now get proper names on lines. Please try this out and let me know how it works. The source is available from CVS. If it is impractical for you to get the code from CVS let me know, and I will prepare a quick source distribution.
comment:4 by , 21 years ago
compiles fine - I'm too lazy to sort it out properly so, under OSX, I configure
--without-python and manually add -lcrypto -lssl in GDALmake.opt
processing MERIDIAN2 still gives extra list warnings as for MERIDIAN
I checked the results by loading the shapefiles into postgis (shp2pgsql) and
then dumping the tables (pg_dump). I checked the column names, number of rows
and spot-checked some values. The columns are named differently (GENERIC uses OS
abbreviations) and also ordered differently so I didn't do a full diff (lazy!).
+ MERIDIAN2_LINE is accepted by shp2pgsql (MERIDIAN_LINE wasn't)
+ GENERIC_ and MERIDIAN2_{LINE,POINT,TEXT} have same number of rows and values
+ cvs version (MERIDIAN2) doesn't give NODE.shp but (1.1.7) GENERIC does
+ for TEXT, GENERIC adds geom_id (duplicates gid?)
+ for POINT (some may be mapped to each other)
+ GENERIC has np po po_list fa
+ MERIDIAN2 has parent_oso proper_nam_10 height
+ for LINE (again, some may actually be mapped)
+ GENERIC has ll np po po_list
+ MERIDIAN2 has parent_oso trunk_road
This is fine for my current purposes. Thanks for the prompt followup.
thanks
Ian
comment:5 by , 21 years ago
Ian, I investigated why a MERIDIAN2_NODE.shp isn't produced. It turns out that the OGR node features read by the non-generic meridian reader have the GEOM_ID of the point geometry as an attribute, but they don't actually attach the geometry itself because it is not stored with the node in the file. The generic reader actually loads the entire file into memory and looks up stuff by index whereas the non-generic version(s) try read features one group at a time. If having the node geometries is key for you, just use the generic option. The warnings that look like this: warmerda@gdal[717]% fg ogr2ogr wrk meridian2.ntf ERROR 6: Can't create fields of type IntegerList on shapefile layers. ERROR 6: Can't create fields of type IntegerList on shapefile layers. Are because Shapefile format doesn't have a list field type. The result is that the lists like GEOM_ID_OF_LINK on the MERIDIAN2_NODE are lost. The GEOM_ID_OF_LINK is the list of geometry ids of the lines that terminate at the node. If these are important to you, you will need to either code more directly to OGR or pick another working format.
comment:6 by , 21 years ago
The detailed notes were for completeness - MERIDIAN2 and GENERIC are both fine for my purposes. thanks again for the prompt followup
Note:
See TracTickets
for help on using tickets.
