ST_AsSVG crashed DB backend with large geometries.

[mgl.gis]

Hello, this is a follow-up from a question I posted on the users list.

I've found that ST_AsSVG will cause the database connection to lock-up, with the following message in the pg_log:

** glibc detected *** postgres: postgres testdb [local] SELECT: > realloc(): invalid next size: 0x000000000220f410 ***

I've attached a sample geometry in WKT/WKB format, but the geometry itself seems to be arbitrary, as I can cause this to happen with a variety of geometries that I merge together using st_union. The attached sample is just a bunch of lines and points that I selected from a couple layers I have.

If there is anything I can do to help debug this, let me know.

[mgl.gis]

Sorry, forgot to mention, I'm using PostgreSQL 8.3.6 & PostGIS 1.3.5:

postgis_full_version

---

POSTGIS='1.3.5' GEOS='3.0.3-CAPI-1.4.2' PROJ='Rel. 4.6.1, 21 August 2008' USE_STATS

[mcayland]

<i>(No comment was entered for this change.)</i>

[colivier]

Fix as r3813 on 1.4 branche It will probably be not fix on 1.3 one as it implied a complete rewrite of export function for clean memory geometry collection handle

Join to this comment a regression test used to check that output is still the same before and after this commit

[mgl.gis]

Is it possible for me to patch my own copy of the current release with this, or is it dependant on other intermediate changes?

[robe]

Olivier,

I assume this change broke my torture test. Can you try this on your machine to rule out an error in my compilation. I testd this on the latest 1.4 SVN and don't recall it ever crashing before. The below now crashes my server. I flipped the change back from fixed to testing.

SELECT ST_AsSVG(foo1.the_geom)

FROM ((SELECT ST_Buffer(ST_SetSRID(ST_Point(i,j),4326), j)

As the_geom

FROM generate_series(-10,50,10) As i

CROSS JOIN generate_series(40,70, 20) As j)) As foo1 CROSS

JOIN ((SELECT ST_SetSRID(ST_Point(i,j),4326) As the_geom

FROM generate_series(-10,50,15) As i

CROSS JOIN generate_series(40,70, 15) j)) As foo2 LIMIT 3;

[pramsey]

<i>(No comment was entered for this change.)</i>

[colivier]

Mike for 1.3.5 version, you could try with the whole lwgeom_svg.c from current trunk. You will at least need to change all new *_release(geom) into pfree_* syntax.

Another option is to upgrade to new 1.4 (coming soon)

HTH,

[robe]

Olivier - even after the last set of SVN changes this still crashes on torture test on my OpenSUSE 11. Given that all the regression tests pass though even on OpenSUSE 11 and doesn't crash on any other machine I have tested and no one else has run into this - I'm willing to accept this as a fault of something funky on my VM.

[mcayland]

I'm still nervous about this, since I can't see a reason for this crash to happen. A couple of things:

i) I see different output between 1.3 and trunk for Regina's comment #5

The trunk output contains extra 'L' characters - Olivier, can you comment on this?

ii) Relation to architecture / word size

Mike/Regina?, which platform and word size are you using? I'm using 64-bit and can't reproduce the crash here. Can you both add to this bug report the output of:

SELECT version(), postgis_full_version();

I'm wondering if it just shows up on 32-bit machines due to the fact that 64-bit gives you extra padding for alignment, and so any miscalculations have a little more squeeze room.

Temporarily flicked back to testing.

ATB,

Mark.

[robe]

Mark, Here is my version, postgis_full_version output PostgreSQL 8.3.1 on i686-pc-linux-gnu, compiled by GCC gcc (SUSE Linux) 4.3.1 20080507 (prerelease) [gcc-4_3-branch revision 135036]

POSTGIS='1.4.0SVN' GEOS='3.1.0-CAPI-1.5.0' PROJ='Rel. 4.6.0, 21 Dec 2007' USE_STATS

Thanks, Regina

[mgl.gis]

Sorry...I've been on the road a fair bit for the past couple weeks. I'll probably be able to test this sometime in the next few days. I already posted my postgis_full_version() above...here's my version():

version

---

PostgreSQL 8.3.6 on x86_64-redhat-linux-gnu, compiled by GCC gcc (GCC) 4.3.2

20081105 (Red Hat 4.3.2-7)

[pramsey]

Actually, this crashes for me on OS/X on the latest checkout. POSTGIS='1.4.0SVN' GEOS='3.1.0-CAPI-1.5.0' PROJ='Rel. 4.6.0, 21 Dec 2007' (procs from 1.4 USE_GEOS=1 USE_PROJ=1 USE_STATS=1 need upgrade)

[mcayland]

Paul,

I see that you have 'need upgrade' on your status line.

I vaguely recall that Olivier changed the signature on the ST_AsSVG() function, so you need to make sure that you drop the database and create a new one from postgis.sql to make sure the new signatures take effect.

Regina, have you tried doing this on your setup too?

HTH,

Mark.

[pramsey]

I take it back, apparently I was testing against an old version of the library. When I went to get you a stack trace against a pristine install, everything worked fine. No problems here as of r3915.

[robe]

Mark,

I use script that drops the database and rebuilds it isntalling the generated postgis.sql script and then runs the torturetest script. I'll verify to make sure it is running the right install script and get back to you on this.

[robe]

I verified I'm using the right script and still crashes, but jsut on this box. The build date shows today as well.

[mcayland]

Regina very kindly allowed me access to her machine to obtain a backtrace, included below:

Program received signal SIGABRT, Aborted. 0xffffe430 in kernel_vsyscall () (gdb) bt #0 0xffffe430 in kernel_vsyscall () #1 0xb7aa0900 in raise () from /lib/libc.so.6 #2 0xb7aa2238 in abort () from /lib/libc.so.6 #3 0xb7adc10d in ?? () from /lib/libc.so.6 #4 0xb7b594c8 in fortify_fail () from /lib/libc.so.6 #5 0xb7b57500 in chk_fail () from /lib/libc.so.6 #6 0xb7b56b88 in ?? () from /lib/libc.so.6 #7 0xb7ae0f23 in overflow () from /lib/libc.so.6 #8 0xb7aba3f6 in printf_fp () from /lib/libc.so.6 #9 0xb7ab42bc in vfprintf () from /lib/libc.so.6 #10 0xb7b56c37 in vsprintf_chk () from /lib/libc.so.6 #11 0xb7b56b7d in sprintf_chk () from /lib/libc.so.6 #12 0xb5300da0 in pointArray_svg_abs (pa=0x84ccab0,

output=0x84ccaca '50 -60 L 48.847116824193833 -48.294580679032315

45.432771950677221 -37.038994058094637 39.888176738152737 -26.665786018823894 32.426406871192889 -17.573593128807186 23.334213981176184 -10.111823261847'..., close_ring=0 '\0', precision=15)

at /usr/include/bits/stdio2.h:34

#13 0xb5300f69 in assvg_polygon_buf (poly=0x84cca60,

output=0x84ccac8 'M 50 -60 L 48.847116824193833 -48.294580679032315

45.432771950677221 -37.038994058094637 39.888176738152737 -26.665786018823894 32.426406871192889 -17.573593128807186 23.334213981176184 -10.1118232618'..., relative=0 '\0', precision=15)

at lwgeom_svg.c:253

#14 0xb5301673 in geometry_to_svg (geom=0x84cc65c '�', relative=<value optimized out>, precision=15) at lwgeom_svg.c:269 #15 0xb5301a48 in assvg_geometry (fcinfo=0xbf8c8414) at lwgeom_svg.c:74 #16 0x081760b7 in ?? () #17 0x08175cc5 in ExecProject? () #18 0x081869ff in ExecNestLoop? () #19 0x081750f0 in ExecProcNode? () #20 0x081888b0 in ExecLimit? () #21 0x08174fd0 in ExecProcNode? () #22 0x0817335b in ExecutorRun? () #23 0x08213253 in ?? () #24 0x08214619 in PortalRun? () #25 0x0820f4df in ?? () #26 0x08210ccf in PostgresMain? () #27 0x081e371d in ?? () #28 0x081e473f in PostmasterMain? () #29 0x081992a6 in main ()

So it looks as if it's being triggered by the FORTIFY_SOURCE option which is explained here:  http://fedoraproject.org/wiki/Security/Features#Compile_Time_Buffer_Checks_.28FORTIFY_SOURCE.29.

Interestingly enough, if I re-build PostGIS with -O0 rather than -O2 then the problem goes away.

So I'm wondering if the way in which the output buffer pointers are being used in pointArray_svg_abs is confusing the object size checking? Then again, it could be a genuine bug. Olivier, can you check the buffer size calculation for polygons again?

ATB,

Mark.

[colivier]

Mark, Regina

Thanks for all theses informations ! I do check again buffer size calculation for polygons

Olivier

[colivier]

Mark, Regina

Could you test again with r3921 ? i think this one will be good !

Olivier

[robe]

Doesn't crash for me anymore so I guess we are good.

[colivier]

For information this was related to precision handle change (significant digits versus precision digits). A static buffer is used before trim trailing zero and wasn't large enough with full precision datas

The others export functions was also concerned and fixed (GML, GeoJson?, KML) in r3921

GCC with FORTIFY_SOURCE was a so good think to help find this issue ! Greats thanks to Regina and Mark for your help and patience on this one !

Olivier

[mgl.gis]

 http://postgis.googlecode.com/issues/attachment?aid=4876954533785043674&name=biggeom_ewkb.txt

[mgl.gis]

 http://postgis.googlecode.com/issues/attachment?aid=-4146996751248203603&name=biggeom_ewkt.txt

[mgl.gis]

 http://postgis.googlecode.com/issues/attachment?aid=2039002366346178974&name=svg_units