Buffer overflow in pgsql2shp

[gabrimonfa]

Postgis 2.1.7 compiled from source

Database tested Postgresql 8.3 with postgis 1.3.3 Postgresql 9.2.10 with postgis 2.0

I'm on Linux Intel 64bit

When I use pgsql2shp to convert a table to shapefile it crashes with

*** buffer overflow detected ***: /tmp/postgis-2.1.7/loader/.libs/pgsql2shp terminated

I've produced a small test case in form of a plain postgresql dump that produce a table named test in schema public I'm dumping the table to shp with the following command

pgsql2shp -f pippo -h host -u user -g the_geom db public.test

The table is empty.

The problem may arise due the warnings issued to the user that many field names have been truncated, since if I remove one field the crash does not happen.

gdb full backtrace provided

[gabrimonfa]

small test case

[gabrimonfa]

gdb session

[strk]

Can you try at reproducing by using a self-contained query as the table ? So there's no data to download…

Also a valgrind report would be useful

[gabrimonfa]

Here it is

./pgsql2shp -f test -h host -u user db "SELECT null::integer as id, \
null::public.geometry as the_geom, \
null::text as materiali_copertura, \
null::text as materiali_copertura_alterazioni, \
null::text as materiali_gronda, \
null::text as materiali_gronda_alterazioni, \
null::text as elementi_architettonici_decorativi, \
null::text as elementi_architettonici_decorativi_alterazioni, \
null::text as superfetazioni_ed_incongruenze, \
null::text as superfetazioni_ed_incongruenze_alterazioni, \
null::text as elementi_articolazione_volumetrica, \
null::text as elementi_articolazione_volumetrica_alterazioni, \
null::text as condizioni_generali, \
null::text as valutazioni, \
null::text as valutazioni_incongruenza_note, \
null::text as informazioni_generali, \
null::text as stato_di_diritto, \
null::text as tipologia_schedatura;"

[gabrimonfa]

valgrind report (with —leak-check=full —show-leak-kinds=all —track-origins=yes)

[pramsey]

The self-contained example does not produce any errors under OSX 10.9

[pramsey]

Nor are there any errors running against a table in the database under OSX 10.9

[robe]

Note I didn't bother testing myself — too lazy for that but assume since pramsey can't replicate, it works fine for him.

[gabrimonfa]

This issue is still present, in my setup, in the following versions:

• OS: ubuntu Xenial
• postgis client version 2.3.2+dfsg-1~exp2.pgdg16.04

• postgis_full_version: POSTGIS="2.3.2 r15302" GEOS="3.4.2-CAPI-1.8.2 r3921" PROJ="Rel. 4.8.0, 6 March 2012" GDAL="GDAL 1.10.1, released 2013/08/26" LIBXML="2.9.1" LIBJSON="0.11.99" RASTER
• PostgreSQL 9.5.7 on x86_64-pc-linux-gnu, compiled by gcc (Debian 4.9.2-10) 4.9.2, 64-bit

It also happens on

• postgis_full_version: POSTGIS="2.3.2 r15302" GEOS="3.5.0-CAPI-1.9.0 r4084" PROJ="Rel. 4.8.0, 6 March 2012" GDAL="GDAL 1.9.2, released 2012/10/08" LIBXML="2.7.6"
• PostgreSQL 9.6.3 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18), 64-bit

It's very easy to reproduce, just execute the proposed self contained test against a db of your choice.
In my setup it crashes all the times.

It may also be a problem of my postgis binary, it comes from pgdg packages for debian/ubuntu.

May I ask you to check again?

[strk]

I can reproduce, with the standalone call and RELEASE: 2.4.0dev (r15464)

[strk]

==21601== Conditional jump or move depends on uninitialised value(s)
==21601==    at 0x4C30D29: strlen (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21601==    by 0x115C7B: ShpDumperOpenTable (pgsql2shp-core.c:1562)
==21601==    by 0x10AFDE: main (pgsql2shp-cli.c:191)
==21601== 
==21601== Conditional jump or move depends on uninitialised value(s)
==21601==    at 0x51DA289: __strncat_chk (strncat_chk.c:36)
==21601==    by 0x115C93: strncat (string3.h:156)
==21601==    by 0x115C93: ShpDumperOpenTable (pgsql2shp-core.c:1562)
==21601==    by 0x10AFDE: main (pgsql2shp-cli.c:191)
==21601== 
==21601== 
==21601== Process terminating with default action of signal 6 (SIGABRT): dumping core
==21601==    at 0x50F777F: raise (raise.c:58)
==21601==    by 0x50F9379: abort (abort.c:89)
==21601==    by 0x513B08F: __libc_message (libc_fatal.c:175)
==21601==    by 0x51DCF83: __fortify_fail (fortify_fail.c:37)
==21601==    by 0x51DAEFF: __chk_fail (chk_fail.c:28)
==21601==    by 0x51DA2A2: __strncat_chk (strncat_chk.c:33)
==21601==    by 0x115C93: strncat (string3.h:156)
==21601==    by 0x115C93: ShpDumperOpenTable (pgsql2shp-core.c:1562)
==21601==    by 0x10AFDE: main (pgsql2shp-cli.c:191)

We don't have a good testsuite for pgsql2shp, would be useful to build one. Volunteers ?

[strk]

In 15480:

Fix buffer overflow and use of uninitialized value

See #3101 (for trunk)

[strk]

Gabri may I ask you to try again with r15480 ? Then it'll need to be backported to stable branches (where a NEWS entry will be needed)

[strk]

In 15483:

Fix buffer overflow and use of uninitialized values

Closes #3101 for 2.3 branch

[strk]

Regina: milestone 2.3.4 is missing, this should be assigned there

[robe]

It's there I think maybe I forgot to make it the default. I'll change in a bit.

[robe]

actually strk anyway to check on trac. I had tried to switch default from 2.3.3 to 2.3.4 but it hung. Just tried again and it's hanging..

[gabrimonfa]

With r15484 I cannot reproduce the problem, it seems solved.
Thank you for super quick reply