occasional crashing in new_mapObj() under heavy load

[richf]

Once with the following stack trace, with a locally compiled mapserver 5.0.0 plus patches for bug 2525 and bug 2497:

_IO_fread (/lib/libc.so.6)
yy_get_next_buffer (maplexer.c:3522)
msyylex (maplexer.c:3353)
loadLayer (mapfile.c:2572)
loadMapInternal (mapfile.c:4353)
msLoadMap (mapfile.c:4578)
new_mapObj (mapscript/java/mapscript_wrap.c:1612)
Java_edu_umn_gis_mapscript_mapscriptJNI_new_1mapObj (mapscript/java/mapscript_wrap.c:19633)

Once with the following strack trace, with a locally compiled branch-5-0 (svn r7432), along with local patches both for bug 2497 and for bug 2533:

_IO_fread (/lib/libc.so.6)
yy_get_next_buffer (maplexer.c:3526)
msyylex (maplexer.c:3357)
loadClass (mapfile.c:2102)
loadLayer (mapfile.c:2577)
loadMapInternal (mapfile.c:4348)
msLoadMap (mapfile.c:4573)
new_mapObj (mapscript/java/mapscript_wrap.c:1612)
Java_edu_umn_gis_mapscript_mapscriptJNI_new_1mapObj (mapscript/java/mapscript_wrap.c:19783)

Both instances were on debian 3.1 (sarge), locally compiled with gcc 3.3.5, using locally compiled versions of GD 2.0.35 and AGG 2.5, and debian packages for everything else. The mapserver configuration is as follows:

MapServer version 5.0.0 OUTPUT=GIF OUTPUT=PNG OUTPUT=WBMP OUTPUT=SVG SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE SUPPORTS=THREADS INPUT=POSTGIS INPUT=SHAPEFILE

The Java mapscript code was compiled with SWIG 1.3.24 and Sun Java 1.5.0_06. It runs with the same version of Java and with tomcat 5.5.17. It queries its data from a PostgreSQL (8.1) / PostGIS (1.1.2) database, uses projections, uses the AGG renderer, includes labels with TrueType fonts, and generates PNG output.

This was originally discussed in this mailing list thread: ​http://www.nabble.com/msProcessProjection%28%29%3A-Projection-library-error.-to15815968.html#a16112206

which led to this bug: ​http://trac.osgeo.org/mapserver/ticket/2533

but I believe this is a separate issue.

Tests so far indicate that a suitable workaround is to surround the java call to new mapObj(String filename) in a synchronized() {} block, without a significant impact on performance.

[sdlime]

Is there anything special about the mapfile? Perhaps we're missing a lock when parsing the mapfile.

Steve

[richf]

I can't think of anything special about the mapfiles. This is a bit of a simplification, and not every layer includes exactly all of the directives in any LAYER below, but the listing below gives an overall feel for the types of directives that I'm using and what the layers look like.

• Rich

MAP
    NAME
    OUTPUTFORMAT
        NAME
        DRIVER
        IMAGEMODE
        FORMATOPTION
    END

    EXTENT
    ANGLE

    PROJECTION
    END

    SIZE
    IMAGECOLOR
    FONTSET
    SYMBOLSET

    LAYER
        NAME
        STATUS
        TYPE
        CONNECTIONTYPE
        CONNECTION
        PROCESSING
        DATA
        CLASS
            NAME
            STYLE
                COLOR
                WIDTH
            END
        END
        PROJECTION
        END
    END
        
    LAYER
        NAME
        STATUS
        TYPE
        CLASSITEM
        CONNECTIONTYPE
        CONNECTION
        PROCESSING
        DATA
        CLASS
            NAME
            EXPRESSION
            STYLE
                COLOR
                SIZE
                SYMBOL
            END
        END
        PROJECTION
        END
    END
        
    LAYER
        NAME
        STATUS
        TYPE
        CONNECTIONTYPE
        CONNECTION
        PROCESSING
        DATA
        CLASS
            NAME
            LABEL
                COLOR
                OUTLINECOLOR
                TYPE
                FONT
                SIZE
                ANGLE
                MINDISTANCE
                POSITION
                BUFFER
            END
        END
        PROJECTION
        END
    END
        
    LAYER
        NAME
        STATUS
        TYPE
        CLASSITEM
        LABELITEM
        CONNECTIONTYPE
        CONNECTION
        PROCESSING
        DATA
        CLASS
            NAME
            EXPRESSION
            STYLE
                COLOR
                SYMBOL
            END
            LABEL
                COLOR
                OUTLINECOLOR
                TYPE
                FONT
                SIZE
                MINDISTANCE
                POSITION
                BUFFER
            END
        END
        PROJECTION
        END
    END
        
END

[sdlime]

Rich: Have you run your tests on the 5.2 beta's by chance?

Steve

[richf]

re: my status

no, i haven't yet had a chance to run anything against either 5.0.3 or 5.2beta, but i'm taking a look at the state of things and considering what to do right now.

[richf]

Sorry I didn't get a chance to test any of this before the 5.2.0 release. I've started running some stress tests with the 5.2.0 release, and so far so good, no signs of this yet. But it was pretty rare, so I can't yet conclude anything definite. I will report more after my tests have run for a longer period of time.

[richf]

I just took a closer look at my code. In the original bug filing, I said:

Tests so far indicate that a suitable workaround is to surround the java call to new mapObj(String filename) in a synchronized() {} block, without a significant impact on performance.

And I have been doing that workaround ever since. So my currently running tests may be of mixed value to you. A better case, if indeed I am not currently getting any crashes, might be to remove the synchronized block and see if I can cause the crashes again. Unfortunately, I don't currently have the opportunity to do this. However, if you think that it might be useful, once my stress tests are done running with the existing codebase (I'd like to let them run for at least about a week or so), I could make that tweak. Let me know.

[sdlime]

Curious if 5.2 or 5.4 for suffer on your system still. I'm moving out to a future release given the ambiguity...

Steve

[richf]

Sorry for taking a while to get back to you. I have not been doing any active development with mapserver for a while, and I only sporadically check the email address to which these bug updates go to.

We have a pair of production machines running mapserver 5.2.0 on Debian GNU/Linux 4.0 (etch). We are not currently seeing this problem, but as I mentioned in the comment on 08/01/08 15:46:53, I don't think that means much, since the new mapObj(String filename) call is wrapped in a synchronized() {} block. To definitively answer your question, I would have to recompile our code without the synchronized() {} block and run stress tests. And I unfortunately still do not have the resources to do this, nor do I expect to at any time in the forseeable future. Sorry.

[sdlime]

No worries, marking as WONTFIX for now...

Steve