setFilter cuts strings after ', " and alike

[bear@…]

If the setFilter function encounters a ' character (or ", possibly others, too),
it ends the string right there and cuts off the remaining characters. For example

$layer->setFilter('shape.id = vertex.shape');

works, but the expression

$layer->setFilter('shape.id = vertex.shape AND shape.type="arc"');

sets the filter to 

shape.id = vertex.shape AND shape.type=

If the same filters are hard-coded in the .map file, they work fine. I think
there is a problem with the string value parser, this might appear at other
set<something> functions as well.

[dmorissette]

Julien, can you please try to reproduce/investigate this when you have a chance.

[bear@…]

I have located the culprit - the following code from mapfile.c gives the result

shape.id = vertex.shape and shape.layer='2'
shape.id = vertex.shape and shape.layer=


int loadExpressionString(expressionObj *exp, char *value)
{
  msyystate = 2; msyystring = value;

  freeExpression(exp); // we're totally replacing the old expression so free
then init to start over
  // initExpression(exp);
  if((exp->type = getSymbol(3, MS_STRING,MS_EXPRESSION,MS_REGEX)) == -1) return(-1);
  exp->string = strdup(msyytext);
  printf("%s<BR>\n%s<BR>\n", value, exp->string);

  return(0);
}

[bear@…]

*** Bug 340 has been marked as a duplicate of this bug. ***

[dmorissette]

In bug 340 it was reported that the expression string may be truncated at a '_' 
too.

[dmorissette]

Steve, I think you're the one who knows the parser best, so I'll reassign this 
to you.

[sdlime]

The example expressions (that don't work) are not valid MapServer expressions
(strings). (I'm guessing then that these are MySQL expressions.) Technically you
should have one other enclosing set of quotes so that the lexer recognizes
everything between them as one string. Without them the lexer will do goofy
things. Admittedly that's counter intuitive.

So, I changed the loadExpressionString code to check for only logical and
regular expressions. If those patterns don't match then the entire input string
is considered as a string expression. I had made this change a bit earlier
(which is why the first example works) but neglected to remove the test for
string expressions so embedded quotes would still be a pain. This should fix it
if someone could confirm I'd appreciate it.

Steve

[sdlime]

I did find one potential problem with the fix. Previously legal strings with
quotes wouldn't work. So to fix we check the length of the input string against
the parsed string. If they differ by exactly 2 characters then we use the parsed
string assuming that the enclosing quotes have been removed. Otherwise we use
the entire string.

Steve

[sgillies@…]

Steve,

Your fix, calling getSymbol() against the MS_REGEX,MS_EXPRESSION pair,
is raising MS_SYMERR when our expression strings are like

'foo' or '"foo"'

This error is going unnoticed to all but Python MapScript, which is
extra sensitive to errors.

[sgillies@…]

Fixed.

loadExpressionString handles the "false positive" from getSymbol in the case
of string (MS_STRING) arguments by clearing the error stack before 
loadExpression or setFilter, setExpression, setText (in mapscript.i) see
any errors.