VMAP0/Eurnasia -> SHAPE produces no results (OGDI driver)

[Markus Neteler]

Frank,
[moving this to Bugzilla]

On Mon, May 12, 2003 at 03:21:27PM -0400, Frank Warmerdam wrote:
> Markus Neteler wrote:
> > Frank,
> >
> > after playing a bit with OGDI in ogr, I have update the
> > related HTML page. Find diffs attached, maybe you like them
> > and add to the repository.
>
> Markus,
>
> I have updated the drv_ogdi.html according to your suggestions
> with the exception of the stuff about .pft, .lft, .aft and .tft
> extensions which I assume is specific to some specific kinds of
> VPF data.
 
It is not yet submitted to CVS, right?
 
> Are the examples tested?
 
Yes and no: All vmap0 maps have
  Feature Count: 0
except four maps.
This is really strange as the data set is some 750MB large.
See attached test report for all maps in VMAP0/Eurasia.
Somewhere I must be missing something.
 
> Note that there are many feature layers that will appear in the ogrinfo
> list with VPF data, but that might contain no features.

Results:
- 72 maps are reported to contain 0 features
-  4 maps contain this number of features
     Feature Count: 1424 - libref@libref(*)_line
     Feature Count: 28   - tileref@tileref(*)_area
     Feature Count: 107  - libreft@libref(*)_text
     Feature Count: 28   - tilereft@tileref(*)_text

From libref@libref(*)_line I was able to generate a SHAPE file.
...what are the other 750 MB about :-)

Maybe you have another recommendation for me.

Thanks in advance

 Markus

[neteler@…]

ogrinfo results (VMAP0/Eurnasia - OGDI driver, Redhat Linux)

[warmerdam]

Markus,

I am sorry for dragging my feet on this so long. 

I have finally found my eurnasia VMAP CD and it seems to work fine for me.  But
I originally ran into similar problems with VITD data, and the problem was the
file naming.  When I mounted the CD all the files were in upper case. But if
the files are renamed to be in lower case with no trailing dot (this can be
done with a script, or possibly appropriate mount options) things do work.  My
eurnasia tree looks something like this:

warmerda@gdal2200[4]% ls
fgdc_dat.txt  readme1.txt  view  vmaplv0
warmerda@gdal2200[5]% ls vmaplv0/
dht  eurnasia  lat  rference
warmerda@gdal2200[6]% ls vmaplv0/eurnasia/
bnd  dq   dqx   grt    ind  libref       phys  tileref  util
cat  dqt  elev  hydro  lht  lineage.doc  pop   trans    veg

[neteler@…]

Frank,

thanks for looking into this. I have downloaded the file from
NIMA directly, and the tree looks like

levi:v0eur[320.64] ls
fgdc_dat.txt  readme1.txt  view  vmaplv0
levi:v0eur[321.65] ls vmaplv0/eurnasia/
bnd   dq    dqx.  grt.   ind   libref  pop tileref  util
cat.  dqt.  elev  hydro  lht.  lineage.doc  phys   trans    veg

So these dots like 'dqt.' are causing problems? OK, after fixing
them to (with a script)

ls vmaplv0/eurnasia/
bnd  dq   dqx   hydro  lht  lineage.doc phys tileref  util
cat  dqt  elev  grt    ind  libref  pop trans   veg

I still have the same problem:
ogrinfo  gltp:/vrf/levi0/ssi/BIO/vectordata/vmap0_europe/v0eur/vmaplv0/eurnasia
'polbnda@bnd(*)_area'
ERROR 4: OGDI Driver doesn't support update.
Had to open data source read-only.
INFO: Open of
`gltp:/vrf/levi0/ssi/BIO/vectordata/vmap0_europe/v0eur/vmaplv0/eurnasia'
using driver `OGDI' successful.

Layer name: polbnda@bnd(*)_area
Geometry: Polygon
Feature Count: 0
Layer SRS WKT:
GEOGCS["NAD83",
    DATUM["North_American_Datum_1983",
        SPHEROID["GRS 1980",6378137,298.257222101,
            AUTHORITY["EPSG","7019"]],
        TOWGS84[0,0,0,0,0,0,0],
        AUTHORITY["EPSG","6269"]],
    PRIMEM["Greenwich",0,
        AUTHORITY["EPSG","8901"]],
    UNIT["degree",0.0174532925199433,
        AUTHORITY["EPSG","9108"]],
    AXIS["Lat",NORTH],
    AXIS["Long",EAST],
    AUTHORITY["EPSG","4269"]]
id: Integer (10.0)
f_code: String (5.0)
nam: String (64.0)
na2: String (2.0)
na3: String (1.0)
tile_id: Integer (5.0)
fac_id: Integer (10.0)

-> Features: 0

ogr2ogr polbnda.shp 
gltp:/vrf/levi0/ssi/BIO/vectordata/vmap0_europe/v0eur/vmaplv0/eurnasia
'polbnda@bnd(*)_area'
-rw-r--r--    1 neteler  ssi           100 May 22 18:58 polbnda.shx
-rw-r--r--    1 neteler  ssi           100 May 22 18:58 polbnda.shp
-rw-r--r--    1 neteler  ssi           333 May 22 18:58 polbnda.prj
-rw-r--r--    1 neteler  ssi           257 May 22 18:58 polbnda.dbf

l /levi0/ssi/BIO/vectordata/vmap0_europe/v0eur/vmaplv0/eurnasia/bnd/polbnd*
-r-xr-xr-x    1 neteler  ssi         46784 Jul 31  2000
/levi0/ssi/BIO/vectordata/vmap0_europe/v0eur/vmaplv0/eurnasia/bnd/polbnda.afx
-r-xr-xr-x    1 neteler  ssi        226749 Jul 31  2000
/levi0/ssi/BIO/vectordata/vmap0_europe/v0eur/vmaplv0/eurnasia/bnd/polbnda.aft
-r-xr-xr-x    1 neteler  ssi         49611 Jul 31  2000
/levi0/ssi/BIO/vectordata/vmap0_europe/v0eur/vmaplv0/eurnasia/bnd/polbndl.lft
-r-xr-xr-x    1 neteler  ssi        128624 Jul 31  2000
/levi0/ssi/BIO/vectordata/vmap0_europe/v0eur/vmaplv0/eurnasia/bnd/polbndp.pfx
-r-xr-xr-x    1 neteler  ssi        402444 Jul 31  2000
/levi0/ssi/BIO/vectordata/vmap0_europe/v0eur/vmaplv0/eurnasia/bnd/polbndp.pft
-r-xr-xr-x    1 neteler  ssi          1727 Sep 11  2000
/levi0/ssi/BIO/vectordata/vmap0_europe/v0eur/vmaplv0/eurnasia/bnd/polbnda.doc
-r-xr-xr-x    1 neteler  ssi          1224 Sep 11  2000
/levi0/ssi/BIO/vectordata/vmap0_europe/v0eur/vmaplv0/eurnasia/bnd/polbndp.doc

ogdi_info -u
gltp:/vrf/levi0/ssi/BIO/vectordata/vmap0_europe/v0eur/vmaplv0/eurnasia -l
'polbnda@bnd(*)_line'
Projection = `+proj=longlat +datum=nad83'
Bounds
north = 89.999900
south = 30.000000
east = 179.999900
west = -60.000000
[...]

Another test:
find . -name '*.aft' -ls | grep hydr
4849703 2360 -r-xr-xr-x   1 neteler  ssi       2411706 Aug  4  2000
./hydro/inwatera.aft

ogr2ogr inwatera.shp 
gltp:/vrf/levi0/ssi/BIO/vectordata/vmap0_europe/v0eur/vmaplv0/eurnasia
'inwatera@hydro(*)_area'
-> generates
-rw-r--r--    1 neteler  ssi           100 May 22 19:03 inwatera.shx
-rw-r--r--    1 neteler  ssi           100 May 22 19:03 inwatera.shp
-rw-r--r--    1 neteler  ssi           333 May 22 19:03 inwatera.prj
-rw-r--r--    1 neteler  ssi           225 May 22 19:03 inwatera.dbf

final test:
find . -name 'w*.?ft' -ls | grep hydr
4849724 4860 -r-xr-xr-x   1 neteler  ssi       4961588 Aug  4  2000
./hydro/watrcrsl.lft

ogr2ogr watrcrsl.shp
gltp:/vrf/levi0/ssi/BIO/vectordata/vmap0_europe/v0eur/vmaplv0/eurnasia
'watrcrsl@hydro(*)_line'
-rw-r--r--    1 neteler  ssi           100 May 22 19:05 watrcrsl.shx
-rw-r--r--    1 neteler  ssi           100 May 22 19:05 watrcrsl.shp
-rw-r--r--    1 neteler  ssi           333 May 22 19:05 watrcrsl.prj
-rw-r--r--    1 neteler  ssi           225 May 22 19:05 watrcrsl.dbf

There must be another trick...

Best regards

 Markus

[warmerdam]

Markus,

This is that I get for a similar construct:

warmerda@gdal2200[188]% ogrinfo -ro
gltp:/vrf/home/warmerda/CDROM/v0eur/vmaplv0/eurnasia 'polbnda@bnd(*)_area' | less
INFO: Open of `gltp:/vrf/home/warmerda/CDROM/v0eur/vmaplv0/eurnasia'
using driver `OGDI' successful.

Layer name: polbnda@bnd(*)_area
Geometry: Polygon
Feature Count: 5847
Extent: (-31.265751, 30.000000) - (179.999900, 81.857361)
Layer SRS WKT:
GEOGCS["NAD83",
    DATUM["North_American_Datum_1983",
        SPHEROID["GRS 1980",6378137,298.257222101,
            AUTHORITY["EPSG","7019"]],
        TOWGS84[0,0,0,0,0,0,0],
        AUTHORITY["EPSG","6269"]],
    PRIMEM["Greenwich",0,
        AUTHORITY["EPSG","8901"]],
    UNIT["degree",0.0174532925199433,
        AUTHORITY["EPSG","9108"]],
    AXIS["Lat",NORTH],
    AXIS["Long",EAST],
    AUTHORITY["EPSG","4269"]]
id: Integer (10.0)
f_code: String (5.0)
nam: String (64.0)
na2: String (2.0)
na3: String (1.0)
tile_id: Integer (5.0)
fac_id: Integer (10.0)
OGRFeature(polbnda@bnd(*)_area):0
  id (Integer) = 1
  f_code (String) =  FA001
  nam (String) =  VIDOY#FAEROE ISLANDS
  na2 (String) =  FO
  na3 (String) =  E
  tile_id (Integer) = 1
  fac_id (Integer) = 18
  POLYGON ((-6.52663898 62.31544495,-6.52933311 62.32108307,-6.54263878 62.33297
348,-6.55186129 62.34711075,-6.54486132 62.36074829,-6.54672241 62.36811066,-6.5
5230570 62.37208176,-6.58330536 62.37900162,-6.59083319 62.38347244,-6.59144449 
62.38852692,-6.57733345 62.39219284,-6.56777763 62.40074921,-6.55266666 62.39866
638,-6.53966665 62.39455414,-6.53438902 62.39288712,-6.50997210 62.37163925,-6.5
1072216 62.35652924,-6.49133348 62.34038925,-6.48169422 62.33816528,-6.47061110 
62.32905579,-6.47391653 62.31880569,-6.47238874 62.30755615,-6.46466684 62.30580
521,-6.45400000 62.30974960,-6.44374990 62.32677841,-6.39958334 62.30791855,-6.4
2336130 62.29410934,-6.42877769 62.28802872,-6.42791653 62.28602600,-6.42508316 
62.27947235,-6.51197243 62.29238892,-6.52663898 62.31544495))

[warmerdam]

OK, I am not sure where to go from here Markus.  Are you working on Linux?  
My best guess is still that it is something to do with file naming conventions
(to which the OGDI VRF driver is pretty sensitive), but you seem to have
everything in good order.  

Can you provide ssh access to the system with the data and the problems?

[neteler@…]

Frank,

[yes, all Linux - Redhat 7.3 with updates]
after setting up an account on grass.itc.it, installing
ODGI and GDAL there, extracting VMAP0, renaming the files/dirs
as recommended, it WORKS on that machine. Magic!

I assume following reasons:
- I made a mistake somewhere
- there is an old lib somewhere (but I used ldd everywhere)
- OGDI driver does not work over NFS (the vmap0 data are on
  another machine connected over network. While today I tried
  all on a single machine)

May I ask you to add to the OGR/OGDI html page:
- NFS support untested
- the trick with file names (trailing dot, capital letters when using CDROM)

and, if possible, update the OGDI makefiles (see OGDI bugtracker).

Well, it was a long way, thanks for your patience and assistance.

Best regards

 Markus