Different results in QGIS using 2.0 OGR when more than one geometry field

[mj10777]

Sometime around December/January I noticed that geometries in tables that had more than 1 geometry no longer showed up when adding a new vector layer. Also in existing qgis projects, the properties of these Layers were listed as 'Unknown geometry'.

I updated the gdal code this morning to check was still occurring.

After looking at the qgis code in

• src/providers/ogr/qgsogrprovider.cpp

these 2 different problems are caused in the functions

• subLayers() - adding a new layer
• getOgrGeomType() - reading the geometry property

1) subLayers()

if the result of OGR_FD_GetGeomType is wkbUnknown, the a loop through fields is done.

This now returns the geometry-type of the first geometry, so the loop is not called.

Thus only the first geometry is shown and the others are ignored

2) getOgrGeomType() (which is called from loadFields())

OGR_FD_GetGeomType returns wkbUnknown

Why OGR_FD_GetGeomType different results is not clear to me.

To resolve this I added a check to the gdal-version (GDAL_VERSION_NUM < 2000000) and in the else

1) subLayers()

int fieldCount=OGR_FD_GetGeomFieldCount(fdef);
for(int j=0; j<fieldCount; j++ )
{
 QString theLayerFieldName = FROM8(OGR_GFld_GetNameRef(OGR_FD_GetGeomFieldDefn(OGR_L_GetLayerDefn(layer), j)));
 OGRwkbGeometryType layerGeomType=OGR_GFld_GetType(OGR_FD_GetGeomFieldDefn(OGR_L_GetLayerDefn(layer), j))        
 QgsDebugMsg( QString( "layerGeomType = %1" ).arg( layerGeomType ) );
 if ( layerGeomType != wkbUnknown )
 {
  int theLayerFeatureCount = OGR_L_GetFeatureCount( layer, j );
  QString geom = ogrWkbGeometryTypeName( layerGeomType );
 mSubLayerList << QString( "%1:%2(%3):%4:%5" ).arg( layer_number++ ).arg( theLayerName ).arg( theLayerFieldName ).arg( theLayerFeatureCount == -1 ? tr( "Unknown" ) : QString::number( theLayerFeatureCount ) ).arg( geom );
 }
 else
 { // Loop code
 }
}

This resolved the issue when adding a new vector-layer, the difference being that the layername in now always 'table_name(field_name)' as apposed to before where tables that had only one geometry used 'table_name' as the layer-name.

2) getOgrGeomType()

    if (GDAL_VERSION_NUM < 2000000)
     geomType = OGR_FD_GetGeomType( fdef );
    else
     geomType = OGR_GFld_GetType(OGR_FD_GetGeomFieldDefn(OGR_L_GetLayerDefn(ogrLayer), 0));

This resolved the issue of the geometry property and the geometries now show correctly.

---

As far as I can see the functions used existed in gdal 1.11, so there may be no need for version checking, but only a replacement of a few code lines.

If a conformation could be given that these conclusions make sense, then maybe a copy of this ticket could be sent to whoever maintains the qgis-ogr code, so that corrections can be made before it becomes noticeable.

I should also note, that the qgis I use is the source version of '2014_02_22', since starting with '2014_05_01' the screen flickers so much while updating that I find to hard on the eyes to use.

[Even Rouault]

I'm a bit confused by the above, but normally OGR_FD_GetGeomType( fdef ) and OGR_GFld_GetType(OGR_FD_GetGeomFieldDefn(OGR_L_GetLayerDefn(ogrLayer), 0)) should return the same result in GDAL >= 1.11. Which which driver did you notice the problem ? Could you attach a dataset that exhibits the problem ? (The fact that QGIS will only handle one geometry field is really an enhancement on QGIS side)

[mj10777]

I have created an archive with a sample database, screenshots and the qgis projects (using the default ubuntu version 2.8.1 - Wien)

wget http://www.mj10777.de/public/download/stadtteile_listen/20150328.gdal_5903.tar.bz2

the Screenshots also include the results after installing gdal 1.11.2 compiled from the original tarball (where everything works correctly using qgis version 2.8.1 - Wien).

There is a readme_images.txt in the archive describing the Screenshots, the spatialite database and the goals.

While doingso I noticed that 'ExecuteSQL' fails when using a layer-name using the 'table_name(field_name)' syntax.

The archive is 23.5 MB in size and was to big to attach.

[Even Rouault]

With trunk, I don't reproduce the difference between OGR_FD_GetGeomType( fdef ) and OGR_GFld_GetType(OGR_FD_GetGeomFieldDefn(OGR_L_GetLayerDefn(ogrLayer), 0)) with the equivalent Python code (didn't try in C, but should behave the same)

>>> from osgeo import ogr
>>> ds = ogr.Open('gdal_5903/Friedrichswerder.db')
>>> lyr = ds.GetLayerByName('album_coverages')
>>> lyr.GetGeomType()
3

>>> from osgeo import ogr
>>> ds = ogr.Open('gdal_5903/Friedrichswerder.db')
>>> lyr = ds.GetLayerByName('album_coverages')
>>> lyr.GetLayerDefn().GetGeomFieldDefn(0).GetType()
3

Could you try to identify a simple reproducible test case ?

Regarding ExecuteSQL() thais fails when using a layer-name using the 'table_name(field_name)' syntax, this expected if you use the default dialect, that will the native SQLite one in that case, since SQLite only knows table name syntax. If you specify OGRSQL as a dialect it should work (by quoting with double quote characters).

[mj10777]

I will try.

At the moment I am building 2 version of qgis 210 in different application directories. One that will use gdal 2.0 and the other that will use 1.11.2, so that a comparison will be easier.

At the moment I am assuming that on these conditions 1.11.2 is returning 0 (wkbUnknown), wheras 2.0 is returning 3 (wkbPolygon). When 0, the qgis code in subLayers() goes through the fields, adding them. Since 2.0 return 3, this is not done - causing the problem.

Once the building of qgis with 1.11.2 is completed, I will add code that will log out what is be done where and from whom.

A major problem is that I cannot compile the present qgis version.

Looking at the present master code (290), I also see that there is a slite change, where now wkbFlatten(layerGeomType) is used instead of 'if ( layerGeomType != wkbUnknown )' as in the 210 code. this may also be a cause.

The build has just completed, so I will now try to work out the exact cause and try to figure out a common code for bothe gdal 1.112 and 2.0.

[mj10777]

After building in log messages and comparing the result running with 1.11.2 and 2.0, the last assumtion can be scraped.

The cause of the problem is that:

gdal 1.11.2 returns a different amount of layers than gdal 2.0.

---

gdal 1.11.2

each geometry field is treated as 1 layer

if 1 table has more than 1 geometry, each geometry is treated as a separate layer

QGIS 2.1.0-master with gdal 1.11.2
subLayers gdal[1110200] layer_count[9] OGR_DS_GetLayerByName[album_coverages(album_geom)] layerGeomType[3]
subLayers gdal[1110200] layer_loop[0] [layerGeomType != wkbUnknown] ogrWkbGeometryTypeName[Polygon] gType[3] LayerFeatureCount[0] GeometryfieldCount[5]
subLayers gdal[1110200] layer_count[9] OGR_DS_GetLayerByName[album_coverages(album_points)] layerGeomType[2147483652]
subLayers gdal[1110200] layer_loop[1] [layerGeomType != wkbUnknown] ogrWkbGeometryTypeName[MultiPoint25D] gType[2147483652] LayerFeatureCount[1] GeometryfieldCount[5]
subLayers gdal[1110200] layer_count[9] OGR_DS_GetLayerByName[album_coverages(album_linestrings)] layerGeomType[2147483653]
subLayers gdal[1110200] layer_loop[2] [layerGeomType != wkbUnknown] ogrWkbGeometryTypeName[MultiLineString25D] gType[2147483653] LayerFeatureCount[2] GeometryfieldCount[5]
subLayers gdal[1110200] layer_count[9] OGR_DS_GetLayerByName[album_coverages(album_geomcollection)] layerGeomType[2147483655]
subLayers gdal[1110200] layer_loop[3] [layerGeomType != wkbUnknown] ogrWkbGeometryTypeName[Unknown WKB: 2147483655] gType[2147483655] LayerFeatureCount[3] GeometryfieldCount[5]
subLayers gdal[1110200] layer_count[9] OGR_DS_GetLayerByName[earth_album_data] layerGeomType[2147483652]
subLayers gdal[1110200] layer_loop[4] [layerGeomType != wkbUnknown] ogrWkbGeometryTypeName[MultiPoint25D] gType[2147483652] LayerFeatureCount[4] GeometryfieldCount[1]
..
subLayers gdal[1110200] layer_count[9] OGR_DS_GetLayerByName[friedrichswerder_album_data] layerGeomType[2147483652]
subLayers gdal[1110200] layer_loop[8] [layerGeomType != wkbUnknown] ogrWkbGeometryTypeName[MultiPoint25D] gType[2147483652] LayerFeatureCount[8] GeometryfieldCount[1]
..
getOgrGeomType gdal[1110200] layer_count[9] OGR_DS_GetLayerByName[friedrichswerder_album_data] layerGeomType[2147483652]
getOgrGeomType gdal[1110200] layer_loop[8] [layerGeomType != wkbUnknown] ogrWkbGeometryTypeName[MultiPoint25D] gType[2147483652] LayerFeatureCount[8] GeometryfieldCount[1]

---

gdal 2.0

each table is treated a 1 layer

QGIS 2.1.0-master with gdal 2.0.0-dev
subLayers gdal[2000000] layer_count[6] OGR_DS_GetLayerByName[album_coverages] layerGeomType[3]
subLayers gdal[2000000] layer_loop[0] [layerGeomType != wkbUnknown] ogrWkbGeometryTypeName[Polygon] gType[3] LayerFeatureCount[0] GeometryfieldCount[5]
subLayers gdal[2000000] layer_count[6] OGR_DS_GetLayerByName[earth_album_data] layerGeomType[2147483652]
subLayers gdal[2000000] layer_loop[1] [layerGeomType != wkbUnknown] ogrWkbGeometryTypeName[MultiPoint25D] gType[2147483652] LayerFeatureCount[1] GeometryfieldCount[1]
..
subLayers gdal[2000000] layer_count[6] OGR_DS_GetLayerByName[friedrichswerder_album_data] layerGeomType[2147483652]
subLayers gdal[2000000] layer_loop[5] [layerGeomType != wkbUnknown] ogrWkbGeometryTypeName[MultiPoint25D] gType[2147483652] LayerFeatureCount[5] GeometryfieldCount[5]

The way the qgis code in subLayers() is now written, only the first geometry of a table will be found

---

Note:

layerCount is the result of OGR_DS_GetLayerCount(OGRDataSourceH)

GeometryfieldCount result of OGR_FD_GetGeomFieldCount(OGRFeatureDefnH)

is the amount of geometry rows in the table and NOT as I first assumed, the amount table columns.

[Even Rouault]

From your latests comments, I don't see any GDAL problem, so I'm closing that ticket. In GDAL 2.0, RFC 41 has been implemented for Spatialite too, but this is per design. QGIS definitely needs changes to accomodate properly layers with multiple geometry fields.

[mj10777]

I have worked out a code in qgis that works for both gdal 1.* and 2.0 and have submitted a bug reportat:

​https://hub.qgis.org/issues/12479