Filter parsing error

[rblazek]

Mapserver fails with parsing error if certain 'key'(?) words are used in filter set on layer by layerObj.setFilter( ) in python mapscript.

For example, setFilter("(point.id = 1)") causes in mapObj_OWSDispatch error:

msTokenizeExpression(): Expression parser error. Parsing time value failed. msParseTime(): Regular expression error. Unrecognized date or time format ().

It works correctly if the same filter is used in mapfile. Filter without parenthesis "point.id = 1" or without table name "(id = 1)" work also with setFilter().

Similar problem appears with certain layers (I was not able to identify a condition) if EXPRESSION is used in layer class and in a filter set by setFilter() appears 'PolygonFromText' - the real problem is the part of the string 'Text'. It fails with:

msTokenizeExpression(): Expression parser error. Parsing fromText function failed.

Work around in both cases is using "true and ( <filter> )" as an expression.

Mapserver 6.0.1 built from source.

[sdlime]

Hmmm... In this case you're actually setting a PostGIS filter not a MapServer expression. The ()'s have meaning to MapServer. In the mapfile the lines:

FILTER 'point.id = 1'
FILTER (point.id = 1)

have completely different meanings. The first one is just a string and is passed to the underlying database. The second is interpreted by MapServer as an expression and is tokenized.

I'm not so sure this is a bug but rather a documentation of syntax issue. What happens if you do:

setFilter("'(point.id = 1)'")

[rblazek]

Thanks, your suggestion with additional single quotes works.

Do you say that mapserver decides how to use the filter according to filter syntax? The documentation says something else:

"For OGR and shapefiles the string is simply a mapserver regular expression. For spatial databases the string is a SQL WHERE clause that is valid with respect to the underlying database."

Note: the problem appeared after upgrade to version 6.0, in 5.x line it was working without additional single quotes.

[sdlime]

This was missed in the migration guide... I'm going to move this to the documentation component. Sorry about that.

Steve

[rblazek]

Unfortunately with additional single quotes it only works with some layers, with other layers it adds the filter to SQL including the quotes. I was not able to discover what makes the difference.

[sdlime]

Can you include specific examples? -Steve

[rblazek]

The trick with single quotes does not work (obviously?) if single quotes also appear in the filter itself:

setFilter("'(point.id = 1)'")

-> sql query created by mapserver is ok:

select "id",encode(ST_AsBinary(ST_Force_2D("geo"),'NDR'),'hex') as geom,"id" from point where geo && GeomFromText('POLYGON((528236 4943148,528236 5271852,806764 5271852,806764 4943148,528236 4943148))',3064) and ((point.id = 1))

but

setFilter("'(point.label = 'pokus')'"

-> additional quotes are used in query:

select "id",encode(ST_AsBinary(ST_Force_2D("geo"),'NDR'),'hex') as geom,"id" from point where geo && GeomFromText('POLYGON((528236 4943148,528236 5271852,806764 5271852,806764 4943148,528236 4943148))',3064) and ('(point.label = 'pokus')')

escaping the quotes in filter

setFilter("'(point.label = \'pokus\')'"

does not help.

Wasn't it better in 5.x version - OGR layer -> mapserver parser, PostGIS -> database?

Can you describe exactly how mapserver decides if FILTER is mapserver expression or it should be passed to database backend in 6.x version?

[havatv]

I move this one back to where it came from, since there still seems to an issue that is not related to documentation here. I suggest this is adressed before the ticket is moved back to documentation (if necessary). If it is moved back to documentation, it would be nice to get some instructions - expressions seem to be complex things.