Opened 14 years ago
Closed 13 years ago
#3207 closed defect (fixed)
MapServer going into an endless loop!
Reported by: | paalkr | Owned by: | tbonfort |
---|---|---|---|
Priority: | normal | Milestone: | 6.0 release |
Component: | AGG | Version: | 5.4 |
Severity: | major | Keywords: | |
Cc: | sdlime, dmorissette, pramsey, aboudreault, basic999 |
Description
Hi!
I have discovered a situation where MapServer seems to enter an endless loop and uses all the CPU power available on a server. I have managed to track down the specific feature in our postgis database that cases MapServer to hang. A database dump containing only this feature and a working mapfile are attached.
The really strange thing is that this feature does not always fail to draw, the situation does only occur with some specific BBOX's in a WMS GetMap request. A working and a non working request are posted below. The only difference between the requests is that the bbox in the working one is extended with 1000 meters to the east.
I have tested this on MapServer 5.4.2 on Linux and Windows, with the same result. I also tried it on an old MapServer 5.0.0 (windows) that still lays around, and the error was NOT present on 5.0.0, where both the requests below did work.
Regards, Pål Kristensen The Norwegian Mapping Authority
Attachments (6)
Change History (38)
by , 14 years ago
Attachment: | n2000_arealdekkeflate.zip added |
---|
comment:1 by , 14 years ago
Component: | MapServer C Library → WMS Server |
---|---|
Owner: | changed from | to
comment:2 by , 14 years ago
Cc: | added |
---|
comment:3 by , 14 years ago
Owner: | changed from | to
---|
follow-ups: 8 10 comment:4 by , 14 years ago
I am unable to reproduce. However, in order to get the map file running I had to change the OUTPUTFORMAT from AGG/JPEG to GD/JPEG. If you change that, does the bug go away for you?
follow-up: 7 comment:5 by , 14 years ago
Even stranger, the box you provide in your WMS URL does not intersect with the extents of the feature you have provided:
norway=# select st_extent(st_transform(st_envelope(geometri),32632)) from arealdekke st_extent ------------------------------------------------------------------------- BOX(601028.766902945 6775184.59170812,621989.102556984 6808336.6956772) (1 row)
comment:6 by , 14 years ago
Cc: | added |
---|
comment:7 by , 14 years ago
Hi!
I have noticed the same thing, but I have now tested this on 5.4.0-beta1 as well with the same result. I can not understand why this specific feature should influence on the request at all. There is a transformation of the data going, on and I thought that maybe there were some trouble related to that.
I will try with the GD renderer tomorrow and report back.
Pål
Replying to pramsey:
Even stranger, the box you provide in your WMS URL does not intersect with the extents of the feature you have provided:
follow-up: 9 comment:8 by , 14 years ago
Replying to pramsey:
I am unable to reproduce. However, in order to get the map file running I had to change the OUTPUTFORMAT from AGG/JPEG to GD/JPEG. If you change that, does the bug go away for you?
Hmm, strange. Did you get any error messages regarding the mapfile? Syntax errors? What exactly happened when you tried the AGG outputformat provided in the mapfile? Didn't PNGs (8 or 24 bits) work either? I'm asking out of curiosity ;)
I got the example running on a server just before zipping it down. I will try GD tomorrow and report back.
Pål
comment:9 by , 14 years ago
Replying to paalkr:
Hmm, strange. Did you get any error messages regarding the mapfile? Syntax errors? What exactly happened when you tried the AGG outputformat provided in the mapfile? Didn't PNGs (8 or 24 bits) work either? I'm asking out of curiosity ;)
My installation just doesn't have AGG is all. I changed the map file enough to get it running (altered postgis connection params, removed the schema reference from the data statement) and when it ran it returned a blank image. Which seemed odd until I did that extent check.
comment:10 by , 14 years ago
Replying to pramsey:
I am unable to reproduce. However, in order to get the map file running I had to change the OUTPUTFORMAT from AGG/JPEG to GD/JPEG. If you change that, does the bug go away for you?
Hi!
Could not wait until tomorrow, so I tried to change to the GD rendered :) Changing all the outputformats like shown below, did actually eliminate the problem.
OUTPUTFORMAT NAME "png8bit" DRIVER "GD/PNG" MIMETYPE "image/png" EXTENSION "png" FORMATOPTION "QUANTIZE_FORCE=ON" FORMATOPTION "QUANTIZE_COLORS=256" FORMATOPTION "QUANTIZE_NEW=ON" FORMATOPTION INTERLACE=OFF END OUTPUTFORMAT NAME "png24bit" DRIVER "GD/PNG" MIMETYPE "image/png; mode=24bit" EXTENSION "png" IMAGEMODE "RGB" FORMATOPTION INTERLACE=OFF END OUTPUTFORMAT NAME "jpeg24" DRIVER "GD/JPEG" MIMETYPE "image/jpeg" EXTENSION "jpg" IMAGEMODE "RGB" FORMATOPTION "QUALITY=85" #FORMATOPTION INTERLACE=OFF END
Because the issue seems related to the AGG rendering engine I got to think about the hatch symbol, and by turning off the hatch it even worked with the original AGG outputformats.
CLASS NAME "Myr" EXPRESSION "Myr" STYLE #BAKGRUNNSFARGE PÅ MYRA COLOR 254 254 254 OUTLINECOLOR 130 200 240 WIDTH 1 END # STYLE #BLÅ LINJER I MYRA # SYMBOL "Hatch" # ANGLE 0 # SIZE 4 # WIDTH 1 # COLOR 130 200 240 # END END
The hatch symbol is defined like this (like provided in the zip archive).
SYMBOL Name 'Hatch' TYPE HATCH END
If you guys need more information or test data to sort out the issue, I will gladly help out. Thanks for looking into this.
Pål
follow-up: 12 comment:11 by , 14 years ago
Cc: | added |
---|
When it works, it returns a blank image, I assume? (Since the extents are not around the feature...)
comment:12 by , 14 years ago
Replying to pramsey:
When it works, it returns a blank image, I assume? (Since the extents are not around the feature...)
Thats correct. But if you turn on maximum logging (DEBUG 5) on the layer you will see that the SQL sent to the database actually fetches the feature, for some reason. I can't understand why because, as you say, the bbox does not intersect with the envelope of the feature. Could it be that the transformation of the bbox from epsg:32632 to epsg:2633 extends the bbox in some way and thats why the feature is fetched?
I have attached a dump (logs.zip on the ticket) from my log file captured when executing the non working request (that actually works when commenting out the hatch symbol). I have also attached a log file when the error occurs (hatch symbol not commented out, and the AGG renderer used).
Pål
comment:13 by , 14 years ago
Actually, re-looking at the request and the SQL log, and running the queries on the test table you send, it appears that request does legitimately intersect the feature.
norway=# select "objtype","ogc_fid" from arealdekkeflate_8 where st_intersects(geometri,GeomFromText('POLYGON((191735.126816272 6798737.21925225,191735.126816272 6892710.66142604,285708.844634827 6892710.66142604,285708.844634827 6798737.21925225,191735.126816272 6798737.21925225))',32633)); objtype | ogc_fid ----------------------------------+--------- Myr | 3319 (1 row) norway=# select astext(transform('SRID=32632;LINESTRING(513024 6792928,599680 6879584)',32633)); astext --------------------------------------------------------------------------------- LINESTRING(191550.844991555 6806509.50909476,285893.088281053 6884880.93915102) (1 row)
I'll see if I can compile with AGG and get an infinite loop going in my machine.
comment:15 by , 14 years ago
The profile hopefully gives enough context for an AGG-master to track down the infinite loop.
comment:16 by , 14 years ago
Owner: | changed from | to
---|
I've placed a test case that uses shape files instead of PostGIS online at http://dl.dropbox.com/u/1184727/norway.zip for easier testing by the AGG wizards.
comment:17 by , 14 years ago
Cc: | added; removed |
---|
comment:18 by , 14 years ago
Any news on this issue? The error appears quite frequent and does cause quite some trouble for us. Our MapServer cluster handles approx 500.000 WMS request pr day at the moment, and almost 1-2% of the request fails because of this. I use a vector fill as a workaround at the moment, but that is not suitable for hatch with an angle.
Pål
comment:19 by , 14 years ago
Sorry, that old link went stale, here's a new one to the test case. http://dl.dropbox.com/u/1184727/norway.zip Have you considered contracting with www.mapgears.com or www.terriscope.fr on this? The problem is in AGG not PostGIS, so it's outside my scope of expertise.
comment:20 by , 14 years ago
Component: | WMS Server → AGG |
---|---|
Status: | new → assigned |
I'll have a look when I have a moment.
can you try with an angle!=0 on your hatch pattern, I seem to recall there was a singularity with that specific angle.
comment:21 by , 14 years ago
Hi!
I tried with several different angles, including 0, and they all fails the same way.
Regards, Pål
comment:23 by , 14 years ago
I confirm the bug. The following line in mapagg.cpp is entering in a endless loop by calling this function: ras1.add_path(pattern) (in renderPathSolidClipped function). I noticed that it's only happening when the polygon have an invalid exterior ring (probably not closed ?).
comment:24 by , 14 years ago
Cc: | added |
---|
comment:25 by , 14 years ago
Cc: | added |
---|
comment:26 by , 14 years ago
I did a small test in python to check the feature that produce the endless loop, here's the output:
OGR DataSet opened MULTIPOLYGON -> IsValid: True MULTIPOLYGON -> IsValid: True Warning 1: Self-intersection at or near point 214246 154524 MULTIPOLYGON -> IsValid: False MULTIPOLYGON -> IsValid: True ...
In MapServer, it's effectively the shape who is invalid that fail to draw with a hatch.
comment:27 by , 14 years ago
Hi Alan: Is your test before or after the conversion of the shape from map to image coordinates?
Steve
comment:28 by , 14 years ago
Steve, I don't think there is any conversion since my test doesn't draw anything. It simply uses the OGR python binding to check the features. I guess that once in mapserver, if the feature is not valid with this python script, it wont be valid before and after the conversion? I'm going to attach the python script.
by , 14 years ago
Attachment: | check_features.py added |
---|
python script that check the validity of features
comment:29 by , 14 years ago
It seems that the invalid feature is not the one that cause the endless loop. I misinterpreted where I was in MapServer. sorry for confusion. I've identified the real feature that cause the bug, will test it alone, export it in WKT and attach it in the ticket.
comment:30 by , 14 years ago
Here's a simple mapfile with one WKT feature to reproduce the bug: http://dl.mapgears.com/3207.map
comment:31 by , 14 years ago
attaching a patch on trunk that solves the infinite loop for Alan's testcase
there's a chain of problems here:
- shape->line[0].numpoints is < 3 , so the line is marked as discarded ( shape->line[0].numpoints is set to 0 ) : don't know why this is happening, as the first line in the wkt contains more than 3 points
- the bounds of the full shape is not initialized correctly, as shape->line[0].numpoints == 0. the hatch crreation function is therefore passed an unitialized width and height for the pattern to create, which causes an "infinite" loop (not sure this would be infinite actually, just very (very) long)
two areas to investigate:
- why is the wkt parser returning a shape with a line with less than 3 points
- need to make the hatch creator more robust (there's no need to create a pattern that's larger than the final image)
comment:32 by , 13 years ago
Milestone: | → 6.0 release |
---|---|
Resolution: | → fixed |
Status: | assigned → closed |
this does not affect mapserver 6.0 (the hatching pipeline has changed)
PG_DUMP