Opened 12 years ago

Last modified 12 years ago

#4112 reopened defect

SLD with FILTER doesn't apply properly to POSTGIS LAYERS

Reported by: guillaume Owned by: aboudreault
Priority: high Milestone:
Component: WMS Server Version: 6.0
Severity: major Keywords: SLD, POSTGIS
Cc: havatv, pgoulborn

Description

Using MapServer 6 and SLD gives incorrect results for POSTGIS LAYERS (and maybe others)

  • SLD filter is used to build a FILTER into the LAYER block, but no EXPRESSION is written into the CLASSes blocks. As a result, all records gets first CLASS representation on the final map.

MapServer5 is OK MapServer6 is OK with a SHAPEFILE data (same SLD used). In this case, no FILTER is written and the EXPRESSION statements are correctly put into CLASSes definitions.

Attachments (3)

mapserv5.png (6.0 KB ) - added by guillaume 12 years ago.
result with MS5
mapserv6.png (3.5 KB ) - added by guillaume 12 years ago.
result with MS6
test.sld (1.1 KB ) - added by guillaume 12 years ago.
sld file used

Download all attachments as: .zip

Change History (23)

by guillaume, 12 years ago

Attachment: mapserv5.png added

result with MS5

by guillaume, 12 years ago

Attachment: mapserv6.png added

result with MS6

by guillaume, 12 years ago

Attachment: test.sld added

sld file used

comment:1 by havatv, 12 years ago

Cc: havatv added

I have observed the same problems. SLD that worked for PostGIS layers in 5 does not work anymore in 6.

comment:2 by pgoulborn, 12 years ago

Cc: pgoulborn added

We're being bitten by this issue too, to the point where we're having to stick with 5.6 until it gets resolved. One part of each of the filters for a layer is being written into the FILTER and is being taken out of/not written into the CLASS EXPRESSIONS when the layer is using POSTGIS.

In our case the symptoms are that when we pass a custom SLD to a LAYER backed by PostGIS all features are styled with the first style defined for the layer. Using the same SLD with Mapserver 5.6 on the same server works as expected.

Casting a very inexpert eye over the source it seems as though there is code to create the FILTER for layers fronting database data only (source:trunk/mapserver/mapogcsld.c@12904#L431) and I would guess that's where the issue arises.

Is there any way to expedite the investigation/resolution of the problem, through a bounty or directly funding development?

comment:3 by assefa, 12 years ago

Owner: changed from assefay@… to msogcbugs

comment:4 by dmorissette, 12 years ago

See also ticket #4152 which seems to be a report of the same issue.

comment:5 by guillaume, 12 years ago

Resolution: fixed
Status: newclosed

I've found where the issue was. On lines 433-440 in mapogcsld.c (file from version 6.0.1, as it has changed a bit in trunk already) :

    if (!bFailedExpression)
    {
        snprintf(szTmp, sizeof(szTmp), "%s", "))");
        pszBuffer =msStringConcatenate(pszBuffer, szTmp);
        msLoadExpressionString(&lp->filter, pszBuffer);
        /* '''HERE IT IS. SIMPLY UNCOMMENT THESE TWO LINES'''
            for (k=0;k<lp->numclasses;k++)
                freeExpression(&lp->class[k]->expression);
        */
    }

Actually I'm always reluctant to remove/comment something which seems to have been done by purpose, but it definitely fixes the issue.

comment:6 by aboudreault, 12 years ago

Resolution: fixed
Status: closedreopened

Thanks guillaume for your comment. I started looking into this as well and have a potential fix in mind that I will commit in the next day or two once we have done more testing on it. I was working with the ticket 4152 test case and I found a completely different issue, that fixed the ticket properly. I'll need to review all this and ensure we have a solution that fixes both tickets.

comment:7 by aboudreault, 12 years ago

Owner: changed from msogcbugs to aboudreault
Status: reopenednew

comment:8 by aboudreault, 12 years ago

I've committed a fix for BRANCH 6.0 in r13014. Basically, there was 2 bugs since the sld can passes through a different part of the code.

  • Bug 1: When the SLD does not contain any AND/OR operator in the rules, the FILTER is properly generated but the expression are not written in the mapfile... the style is invalid. After a few tests, the Guillaume patch is ok.
  • Bug 2: When the SLD does contain AND/OR operators, the filter is incorrectly generated. Only the first rule is used.

My changeset fixes both issues. However, nothing seems to work in trunk and I'll need to debug this. The trunk code changed and the behavior is different and broken.

comment:9 by pgoulborn, 12 years ago

I've got a build of r13014 and it's looking good so far - the cases where we hit the error before are now returning identical images to 5.6.6

comment:10 by pgoulborn, 12 years ago

Hmm... I've just hit an issue with FILTER in WFS GetFeature requests - I can't say whether this is a regression since 6.0.1 release but the same filters work with 5.6.6.

From the logs it looks as though a temporary map file should be created with a FILTER for the LAYER - but no FILTER is present.

comment:11 by aboudreault, 12 years ago

pgoulborn, could you create a simple test case, or at least provide me the SLD + a mapserver URL example you are using. This might be a new issue, not necessary related to this one. If so, we'll create another ticket.

comment:12 by pgoulborn, 12 years ago

You're right, it doesn't appear to be related and I've tested against 6.0.1 and get the same result. I'm a bit tied up at the moment so I'm going to go back to 5.6 for now, but I'll open a new ticket once I have more information or something you can test against.

As far as I'm concerned, the problem in this ticket is resolved in 6.0 r13014

comment:13 by pgoulborn, 12 years ago

I finally had some more time to look at the not-related-to-this-case FILTER issue and the problem was with me and not Mapserver, I'm posting here in case anyone else encounters the same issue.

I was using an incorrect filter: <Intersect> rather than <Intersects>. This works with 5.6 but not 6.0, I was being lazy and picked an example of the filter from http://mapserver.org/ogc/filter_encoding.html#tests which, despite elsewhere on the page talking about "Intersects", uses "Intersect" in the actual test. I guess that page is out of date - and the demo server is as well (GetCapabilities says 5.6.5)?

in reply to:  13 comment:14 by havatv, 12 years ago

Replying to pgoulborn:

I was using an incorrect filter: <Intersect> rather than <Intersects>. This works with 5.6 but not 6.0, I was being lazy and picked an example of the filter from http://mapserver.org/ogc/filter_encoding.html#tests which, despite elsewhere on the page talking about "Intersects", uses "Intersect" in the actual test. I guess that page is out of date - and the demo server is as well (GetCapabilities says 5.6.5)?

I have changed Intersect to Intersects for the filter encoding page (r13077).

comment:15 by aboudreault, 12 years ago

Resolution: fixed
Status: newclosed

After another fix related to SLD r13021, Initial issue is now fixed in trunk in r13085.

comment:16 by guillaume, 12 years ago

Resolution: fixed
Status: closedreopened

Hi, If that issue is closed, we have still a problem with FILTER. When a FILTER clause exists on a LAYER and you apply an SLD file (without any filter) to it, the FILTER CLAUSE is not removed and thus only the objects that pass the filter are displayed.

in reply to:  16 ; comment:17 by havatv, 12 years ago

Replying to guillaume:

Hi, If that issue is closed, we have still a problem with FILTER. When a FILTER clause exists on a LAYER and you apply an SLD file (without any filter) to it, the FILTER CLAUSE is not removed and thus only the objects that pass the filter are displayed.

I don't understand why you would want layer filters to be removed in the presence external styling / SLD. I would definitely want my layer filters to be enforced, as they have nothing to do with the styling (they only define which features I want to be make available in the layer).

in reply to:  17 comment:18 by guillaume, 12 years ago

Replying to havatv:

Replying to guillaume:

Hi, If that issue is closed, we have still a problem with FILTER. When a FILTER clause exists on a LAYER and you apply an SLD file (without any filter) to it, the FILTER CLAUSE is not removed and thus only the objects that pass the filter are displayed.

I don't understand why you would want layer filters to be removed in the presence external styling / SLD. I would definitely want my layer filters to be enforced, as they have nothing to do with the styling (they only define which features I want to be make available in the layer).

As seen previously, MapServer applySLD adds a FILTER clause to any POSTGIS LAYER when an SLD filter must be applied. So after applying an SLD containing a filter in it, you end up with a LAYER containing a FILTER. If you want to remove that filter, to display all the data like it was before SLD application, you can't, because even if you apply an SLD file without any filter in it, your layer's FILTER is not removed from the mapfile.

comment:19 by dmorissette, 12 years ago

You talk about restoring state after applying a SLD, so I presume you are not using the straight CGI, but instead are using some kind of long-running process (FastCGI or MapScript)? Sounds like this is a different issue from this ticket's original problem and should get its own ticket with more details on your setup and/or what you are doing exactly.

in reply to:  19 comment:20 by guillaume, 12 years ago

Replying to dmorissette:

You talk about restoring state after applying a SLD, so I presume you are not using the straight CGI, but instead are using some kind of long-running process (FastCGI or MapScript)? Sounds like this is a different issue from this ticket's original problem and should get its own ticket with more details on your setup and/or what you are doing exactly.

Hi Daniel, Actually I'm just saving map files once modified with applySLD method. And it's then impossible to remove the FILTER clause. But as havatv said, a preset FILTER shouldn't be removed either. That's why I guess this "add filter when applying SLD on DB layers" optimization is not a good idea. If the FILTER must be protected when applying an SLD file, why could it be set during the same operation ? This FILTER thing as lead to many bogus behaviors, like the one described sooner in this report. SLD is about styling, and should only impact the CLASS/STYLES blocks, not the LAYER level. If any optimization can be done using filter, it should be done deeper in mapserver core, when mapserver reads the layer definition and sees CLASSES are using expressions. My 2 cents.

Guillaume

Note: See TracTickets for help on using tickets.