Filter encoding: simple filters using propertyislike not applied properly

[assefa]

This was reported by Bart,

"I am tracking down why a simple PropertyIsLike filter is not being sent to the database.

The reason is that in FLTValidForPropertyIsLikeFilter there is a check on the psLeftNode and psRightNode being null. Ofcourse this is not the case, since the leftNode contains the propertyname and the rightnode contains the value. So the function returns false.

if (psFilterNode->psLeftNode == NULL && psFilterNode->psRightNode == NULL)

return 1;

return 0;

This code was not in 5.0.0, do you know why this was introduced?"

response from Assefa:

But obviously the function is incorrect. The previous code (5.0) should work but is limited. Here is what the intention is:

• if it is a MapServer flat file layer
  • for thr filter encoding to be considered 'simple" : only one propertyislike is permitted if we want to use the filter/filteritem parameters of the layer. The code in 5.0 works for that

• if it is a DB layer: we can have unlimited number of propertyislike filters since the final expression is would end up being something like: (attribute_name like 'value')

Revering back to the 5.0 code would be a start but I need to check for the layer type to make sure that both cases described are supported properly.

[assefa]

patch to allow propertyislike filters to be treated as other binary operations using operand =~

msautotests to be committed too after the 5.6 release:

1.1-simple 1 propertyislike ​http://127.0.0.1/cgi-bin/mapserv.exe?map=f:/projects/mapserver-buildkit-2008/mapserver/msautotest/wxs/wfs_filter_mine.map&&SERVICE=WFS&VERSION=1.0.0&REQUEST=GetFeature&TYPENAME=popplace&FILTER=<Filter><PropertyIsLike+wildCard='*'+singleChar='.'+escape='!'><PropertyName>NAME</PropertyName><Literal>Syd*</Literal></PropertyIsLike></Filter>

​http://127.0.0.1/cgi-bin/mapserv.exe?map=f:/projects/mapserver-buildkit-2008/mapserver/msautotest/wxs/wfs_filter_mine.map&&SERVICE=WFS&VERSION=1.0.0&REQUEST=GetFeature&TYPENAME=popplace&FILTER=<Filter><PropertyIsLike+wildCard='*'+singleChar='.'+escape='!'><PropertyName>NAME</PropertyName><Literal>Lun*</Literal></PropertyIsLike></Filter>

1.2- 2 propertyislike with an OR

​http://127.0.0.1/cgi-bin/mapserv.exe?map=f:/projects/mapserver-buildkit-2008/mapserver/msautotest/wxs/wfs_filter_mine.map&&SERVICE=WFS&VERSION=1.0.0&REQUEST=GetFeature&TYPENAME=popplace&FILTER=<Filter><OR><PropertyIsLike+wildCard='*'+singleChar='.'+escape='!'><PropertyName>NAME</PropertyName><Literal>Syd*</Literal></PropertyIsLike><PropertyIsLike+wildCard='*'+singleChar='.'+escape='!'><PropertyName>NAME</PropertyName><Literal>Lun*</Literal></PropertyIsLike></OR></Filter>

1.3- 2 propertyislike with an AND

​http://127.0.0.1/cgi-bin/mapserv.exe?map=f:/projects/mapserver-buildkit-2008/mapserver/msautotest/wxs/wfs_filter_mine.map&&SERVICE=WFS&VERSION=1.0.0&REQUEST=GetFeature&TYPENAME=popplace&FILTER=<Filter><AND><PropertyIsLike+wildCard='*'+singleChar='.'+escape='!'><PropertyName>NAME</PropertyName><Literal>Syd*</Literal></PropertyIsLike><PropertyIsLike+wildCard='*'+singleChar='.'+escape='!'><PropertyName>POP_RANGE</PropertyName><Literal>3</Literal></PropertyIsLike></AND></Filter>

1.4 - 1 propertyislike and 1 propertyisequal with an AND ​http://127.0.0.1/cgi-bin/mapserv.exe?map=f:/projects/mapserver-buildkit-2008/mapserver/msautotest/wxs/wfs_filter_mine.map&&SERVICE=WFS&VERSION=1.0.0&REQUEST=GetFeature&TYPENAME=popplace&FILTER=<Filter><AND><PropertyIsLike+wildCard='*'+singleChar='.'+escape='!'><PropertyName>NAME</PropertyName><Literal>Syd*</Literal></PropertyIsLike><PropertyIsEqualTo><PropertyName>POP_RANGE</PropertyName><Literal>4</Literal></PropertyIsEqualTo></AND></Filter>

1.5 with sld and filter

​http://127.0.0.1/cgi-bin/mapserv.exe?map=f:/projects/mapserver-buildkit-2008/mapserver/msautotest/wxs/wms_sld.map&SERVICE=WMS&VERSION=1.1.0&REQUEST=GetMap&SRS=EPSG:4326&BBOX=-68.125,42,-58.375,48.5&FORMAT=image/png&WIDTH=300&HEIGHT=200&STYLES=&LAYERS=road&sld_body=%3CStyledLayerDescriptor+version%3D%221.0.0%22%3E%3CNamedLayer%3E%3CName%3Eroad%3C%2FName%3E%3CUserStyle%3E%3CTitle%3Exxx%3C%2FTitle%3E%3CFeatureTypeStyle%3E%3CRule%3E%3CFilter%3E%3CPropertyIsLessThan%3E%3CPropertyName%3EROAD_ID%3C%2FPropertyName%3E%3CLiteral%3E900%3C%2FLiteral%3E%3C%2FPropertyIsLessThan%3E%3C%2FFilter%3E%3CLineSymbolizer%3E%3CGeometry%3E%3CPropertyName%3Ecenter-line%3C%2FPropertyName%3E%3C%2FGeometry%3E%3CStroke%3E%3CCssParameter+name%3D%22stroke%22%3E%230000ff%3C%2FCssParameter%3E%3CCssParameter+name%3D%22stroke-width%22%3E2.0%3C%2FCssParameter%3E%3C%2FStroke%3E%3C%2FLineSymbolizer%3E%3C%2FRule%3E%3C%2FFeatureTypeStyle%3E%3C%2FUserStyle%3E%3C%2FNamedLayer%3E%3C%2FStyledLayerDescriptor%3E

​http://127.0.0.1/cgi-bin/mapserv.exe?map=f:/projects/mapserver-buildkit-2008/mapserver/msautotest/wxs/wms_sld.map&SERVICE=WMS&VERSION=1.1.0&REQUEST=GetMap&SRS=EPSG:4326&BBOX=-68.125,42,-58.375,48.5&FORMAT=image/png&WIDTH=300&HEIGHT=200&STYLES=&LAYERS=road&sld_body=%3CStyledLayerDescriptor+version%3D%221.0.0%22%3E%3CNamedLayer%3E%3CName%3Eroad%3C%2FName%3E%3CUserStyle%3E%3CTitle%3Exxx%3C%2FTitle%3E%3CFeatureTypeStyle%3E%3CRule%3E%3CFilter%3E%3CAnd%3E%3CPropertyIsGreaterThan%3E%3CPropertyName%3EROAD_ID%3C%2FPropertyName%3E%3CLiteral%3E900%3C%2FLiteral%3E%3C%2FPropertyIsGreaterThan%3E%3CPropertyIsLike+wildCard%3D'*'+singleChar%3D'.'+escape%3D'!'%3E%3CPropertyName%3ENAME_E%3C%2FPropertyName%3E%3CLiteral%3ETrans*%3C%2FLiteral%3E%3C%2FPropertyIsLike%3E%3C%2FAnd%3E%3C%2FFilter%3E%3CLineSymbolizer%3E%3CGeometry%3E%3CPropertyName%3Ecenter-line%3C%2FPropertyName%3E%3C%2FGeometry%3E%3CStroke%3E%3CCssParameter+name%3D%22stroke%22%3E%230000ff%3C%2FCssParameter%3E%3CCssParameter+name%3D%22stroke-width%22%3E2.0%3C%2FCssParameter%3E%3C%2FStroke%3E%3C%2FLineSymbolizer%3E%3C%2FRule%3E%3C%2FFeatureTypeStyle%3E%3C%2FUserStyle%3E%3C%2FNamedLayer%3E%3C%2FStyledLayerDescriptor%3E

[assefa]

committed in trunk (r9584)

[assefa]

msautotest added r9586.

[bartvde]

Tested, works fine.

[dionw]

Code from changeset:9584 which trims slashes causes a crash when the pszFinalAttributeValue is later free()d. This happens because the pointer is increased by one or two bytes and so doesn't match the original pointer returned at allocation time.

B.T.W. I've used the very useful valgrind utility ​http://valgrind.org/docs/manual/quick-start.html#quick-start.intro to find locate the problem.

Example SLD_BODY which triggers a crash when used in a GetMap:

<sld:StyledLayerDescriptor xmlns:sld="http://www.opengis.net/sld" version="1.0.0"
 xsi:schemaLocation="http://www.opengis.net/sld http://schemas.opengis.net/sld/1.0.0
 /StyledLayerDescriptor.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xmlns:ogc="http://www.opengis.net/ogc" xmlns:gml="http://www.opengis.net/gml">
<sld:NamedLayer><sld:Name>AAA207</sld:Name><sld:UserStyle>
<sld:Name>default</sld:Name><sld:FeatureTypeStyle><sld:Rule>
<sld:Name>default</sld:Name><ogc:Filter xmlns:ogc="http://www.opengis.net/ogc">
<ogc:PropertyIsLike wildCard="*" singleChar="." escape="!">
<ogc:PropertyName>DISTRICTNA</ogc:PropertyName><ogc:Literal>*Eindhoven*</ogc:Literal>
</ogc:PropertyIsLike></ogc:Filter><sld:PolygonSymbolizer><sld:Fill><sld:CssParameter
name="fill">#FF0000</sld:CssParameter></sld:Fill></sld:PolygonSymbolizer></sld:Rule>
</sld:FeatureTypeStyle></sld:UserStyle></sld:NamedLayer></sld:StyledLayerDescriptor>

I'll add a patch which moves the substring to do the trimming instead.

[dionw]

Patch to fix trimming of pszFinalAttributeValue string

[assefa]

Thanks a lot. I applied the patch in trunk -r9763.

[sdlime]

There's been a bit of discussion on the mapserver-users list on this one. Two questions: 1) should this be backported to the 5.6 branch? and 2) can the ticket then be closed?

Steve

[assefa]

The changed involved using the =~ operator for the propertyislike (replacing the regex expression and the filteritem). It touched several part of the code and I was unwilling at the time to do modifications/tests at 2 places (​http://trac.osgeo.org/mapserver/changeset/9584). Closing the ticket