Opened 11 years ago

Closed 7 years ago

#536 closed defect (fixed)

memory corruption in vector modules

Reported by: martinl Owned by: grass-dev@…
Priority: blocker Milestone: 7.0.0
Component: Vector Version: svn-trunk
Keywords: Cc:
CPU: x86-32 Platform: Linux

Description

Hi,

I discovered strange memory corruption in Vect_open_old_head(), e.g.

Vect_open_old_head (Map=0xbffee630, name=0x87140f0 "bs", mapset=0x804aea8 "") at open.c:475
(gdb) p Map->head.line_3
$21 = 0x87146e0 ""
(gdb) n
main (argc=2, argv=0xbffeeb84) at main.c:110
(gdb) p Map->head.line_3
$22 = 0x7b1ea71 <Address 0x7b1ea71 out of bounds>

In the result all vector modules segfaults.

Martin

Attachments (2)

v.info.patch1 (414 bytes) - added by mmetz 11 years ago.
patch for v.info
v.info.patch2 (626 bytes) - added by mmetz 11 years ago.
alternative patch for v.info

Download all attachments as: .zip

Change History (26)

comment:1 Changed 11 years ago by mmetz

If you svn upped recently, and did not make distclean, this could be the reason, because I have added LFS support to vector libs. The latest grass7 is in constant use by me, no segfault, all vector operations work. Please let me know if it works again after make distclean, configure, make.

Markus M

comment:2 in reply to:  1 ; Changed 11 years ago by martinl

Replying to mmetz:

If you svn upped recently, and did not make distclean, this could be the reason, because I have added LFS support to vector libs. The latest grass7 is in constant use by me, no segfault, all vector operations work. Please let me know if it works again after make distclean, configure, make.

I already did make distclean.

comment:3 in reply to:  2 Changed 11 years ago by martinl

Replying to martinl:

Replying to mmetz:

If you svn upped recently, and did not make distclean, this could be the reason, because I have added LFS support to vector libs. The latest grass7 is in constant use by me, no segfault, all vector operations work. Please let me know if it works again after make distclean, configure, make.

I already did make distclean.

I forgot to mention configure.

        CFLAGS="-g -Wall -Werror-implicit-function-declaration -fno-common -Wextra -Wunused" \
	    CXXFLAGS="-g -Wall"  \
	    ./configure --prefix=/usr/local \
	    --with-tcltk-includes=/usr/include/tcl8.4 --with-tcltk-libs=/usr/lib/tcl8.4 \
	    --with-postgres --with-postgres-includes=/usr/include/postgresql \
	    --with-mysql --with-mysql-includes=/usr/include/mysql \
	    --with-gdal=/usr/local/gdal-grass7/bin/gdal-config  \
	    --with-proj \
	    --with-motif --with-glw --with-nls --with-readline \
	    --with-cxx --enable-largefile \
	    --with-freetype --with-freetype-includes=/usr/include/freetype2 \
	    --with-sqlite --with-ffmpeg --with-ffmpeg-includes=/usr/include/ffmpeg/ \
	    --with-odbc --with-python=/usr/bin/python2.5-config --with-wxwidgets --with-pthread \
	    --with-ffmpeg=no

comment:4 in reply to:  2 Changed 11 years ago by mmetz

Replying to martinl:

Replying to mmetz:

If you svn upped recently, and did not make distclean, this could be the reason, because I have added LFS support to vector libs. The latest grass7 is in constant use by me, no segfault, all vector operations work. Please let me know if it works again after make distclean, configure, make.

I already did make distclean.

OK, now I did make distclean, configure and make on both Linux 32bit (Fedora 8) and Linux 64bit (Fedora 10) after svn up again. trunk works for me, display, query, rebuild topology, open vector with rebuilt topology in grass-6.4.0RC3, everything is fine. I used boundary_county of nc_spm_08. ???

Segfault not reproducible by me, can someone else reproduce this?

Markus M

comment:5 Changed 11 years ago by mmetz

I could still not reproduce it, but found a possible reason. Fixed in r36440 ???

comment:6 in reply to:  5 ; Changed 11 years ago by martinl

Replying to mmetz:

I could still not reproduce it, but found a possible reason. Fixed in r36440 ???

Not really, still segfaults, e.g.

v.info bs
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb6b016f0 (LWP 9066)]
0xb75bf1cb in strlen () from /lib/i686/cmov/libc.so.6
(gdb) bt
#0  0xb75bf1cb in strlen () from /lib/i686/cmov/libc.so.6
#1  0xb758b648 in vfprintf () from /lib/i686/cmov/libc.so.6
#2  0xb75a9cec in vsprintf () from /lib/i686/cmov/libc.so.6
#3  0xb7591b4b in sprintf () from /lib/i686/cmov/libc.so.6
#4  0x0804ab93 in main (argc=1651335500, argv=0x20747265) at main.c:333

comment:7 in reply to:  6 ; Changed 11 years ago by mmetz

Replying to martinl:

Replying to mmetz:

I could still not reproduce it, but found a possible reason. Fixed in r36440 ???

Not really, still segfaults

Is this only true for vector bs or for all vectors? What are the contents of vector/bs/head, is there something specified for OTHER INFO:?

I suspect Vectread_head because according to the description of this ticket, something is wrong with setting Map->head.line_3 within Vectread_head, and Vectread_head reads the plain text file named head.

Just a guess.

comment:8 in reply to:  7 ; Changed 11 years ago by martinl

Replying to mmetz:

Is this only true for vector bs or for all vectors?

No, for all vectors.

What are the contents of vector/bs/head, is there something specified for OTHER INFO:?

empty string

I suspect Vectread_head because according to the description of this ticket, something is wrong with setting Map->head.line_3 within Vectread_head, and Vectread_head reads the plain text file named head.

When leaving Vect_open_old_head everything seems OK.

Vect_open_old_head (Map=0xbffef630, name=0x8b200f0 "bs", mapset=0x804aea8 "") at open.c:474
(gdb) n
(gdb) p Map->head.line_3
$7 = 0x8b206e0 ""

Next step -> memory corrupted

(gdb) n
main (argc=2, argv=0xbffefb84) at main.c:110
(gdb) p Map->head.line_3
$8 = 0x7b1ea71 <Address 0x7b1ea71 out of bounds>

Sorry, no idea...

comment:9 in reply to:  8 ; Changed 11 years ago by mmetz

Replying to martinl:

When leaving Vect_open_old_head everything seems OK.

...

Next step -> memory corrupted

...

Sorry, no idea...

Have you looked at debug messages? Debug level 2 or 3? v.info in standard mode prints out exactly nothing for you I guess?

Now a nasty question: have you heard of anybody else experiencing this problem?

I'm lost here, also no idea...

comment:10 in reply to:  9 Changed 11 years ago by martinl

Resolution: worksforme
Status: newclosed

Replying to mmetz:

Have you looked at debug messages? Debug level 2 or 3? v.info in standard mode prints out exactly nothing for you I guess?

Now a nasty question: have you heard of anybody else experiencing this problem?

I'm lost here, also no idea...

Hm, I recompiled again and the problem disappeared. Closing ticket.

comment:11 Changed 11 years ago by martinl

Resolution: worksforme
Status: closedreopened

I was too quick - memory corrupted in v.info. Anybody else facing to the similar problem?

comment:12 in reply to:  11 ; Changed 11 years ago by mmetz

Replying to martinl:

I was too quick - memory corrupted in v.info.

The earlier gdb didn't tell me a lot. Can you please run v.info with debug=2 or 3? At least I can work better with that. I'm willing to help, but I can't reproduce the problem and need more information.

comment:13 in reply to:  12 Changed 11 years ago by martinl

Replying to mmetz:

Replying to martinl:

I was too quick - memory corrupted in v.info.

The earlier gdb didn't tell me a lot. Can you please run v.info with debug=2 or 3? At least I can work better with that. I'm willing to help, but I can't reproduce the problem and need more information.

Here it is:

D1/3: field = 1 name = (null), table = elev_ned10m_cont10m, key = cat, database = $GISDBASE/$LOCATION_NAME/$MAPSET/dbf/, driver = dbf
D1/3: Dblinks read
 +----------------------------------------------------------------------------+
 | Layer:           elev_ned10m_cont10m                                       |
 | Mapset:          PERMANENT                                                 |
 | Location:        nc_spm_08                                                 |
 | Database:        /home/martin/grassdata                                    |
 | Title:           South-West Wake 10m elevation contours from 10m NED       |
 | Map scale:       1:1                                                       |
 | Map format:      native                                                    |
 | Name of creator: helena                                                    |
 | Organization:    NC OneMap                                                 |
 | Source date:     Tue Nov  7 20:59:46 2006                                  |
 |----------------------------------------------------------------------------|
 |   Type of Map:  vector (level: 2)                                          |
 |                                                                            |
 |   Number of points:       0               Number of areas:      0          |
 |   Number of lines:        1008            Number of islands:    0          |
 |   Number of boundaries:   0               Number of faces:      0          |
 |   Number of centroids:    0               Number of kernels:    0          |
 |                                                                            |
 |   Map is 3D:              Yes                                              |
 |   Number of dblinks:      1                                                |
 |                                                                            |
D2/3: G__read_Cell_head
D2/3: G__read_Cell_head_array
D3/3: region item: proj:       99
D3/3: region item: zone:       0
D3/3: region item: north:      228495
D3/3: region item: south:      215005
D3/3: region item: east:       644995
D3/3: region item: west:       630005
D3/3: region item: cols:       1499
D3/3: region item: rows:       1349
D3/3: region item: e-w resol:  10
D3/3: region item: n-s resol:  10
D3/3: region item: top:        150
D3/3: region item: bottom:     60
D3/3: region item: cols3:      1499
D3/3: region item: rows3:      1349
D3/3: region item: depths:     90
D3/3: region item: e-w resol3: 10
D3/3: region item: n-s resol3: 10
D3/3: region item: t-b resol:  1
 |         Projection: Lambert Conformal Conic                                |
 |               N:            228495    S:            215005                 |
 |               E:            644995    W:            630005                 |
 |               B:                60    T:               150                 |
 |                                                                            |
D1/3: Vect_get_thresh(): thresh = 0.000000
 |   Digitization threshold: 0                                                |
 |   Comments:                                                                |
Segmentation fault

Debugging v.info in Emacs

  • In Vect_open_old_head() -> OK
(gdb) p Map->head.line_3
$1 = 0x823c700 ""
  • When I leave the function -> memory corrupted
main (argc=2, argv=0xbfac2e54) at main.c:110
(gdb) p Map->head.line_3
$2 = 0x7b1ea71 <Address 0x7b1ea71 out of bounds>

Strange, I have no idea where is problem :-(

Thanks for your help.

Changed 11 years ago by mmetz

Attachment: v.info.patch1 added

patch for v.info

Changed 11 years ago by mmetz

Attachment: v.info.patch2 added

alternative patch for v.info

comment:14 Changed 11 years ago by mmetz

Can you try the first patch, and if that doesn't work, the second patch?

I seems that v.info failed in main.c:334

My guess is that the real problem is in v.info main.c:112. It would be safer if v_head would be a pointer and line 112 would be v_head = &Map.head. Even better is patch1, if it works for you.

Hope that helps.

comment:15 in reply to:  14 ; Changed 11 years ago by martinl

Resolution: fixed
Status: reopenedclosed

Replying to mmetz:

Can you try the first patch, and if that doesn't work, the second patch?

I seems that v.info failed in main.c:334

My guess is that the real problem is in v.info main.c:112. It would be safer if v_head would be a pointer and line 112 would be v_head = &Map.head. Even better is patch1, if it works for you.

Hope that helps.

Right, it helps (committed in r36508). Anyway I do not understand why memory corruption was detected only in GRASS7 not GRASS6.

comment:16 in reply to:  15 Changed 11 years ago by martinl

Replying to martinl:

Replying to mmetz:

Can you try the first patch, and if that doesn't work, the second patch?

I seems that v.info failed in main.c:334

My guess is that the real problem is in v.info main.c:112. It would be safer if v_head would be a pointer and line 112 would be v_head = &Map.head. Even better is patch1, if it works for you.

Hope that helps.

Right, it helps (committed in r36508). Anyway I do not understand why memory corruption was detected only in GRASS7 not GRASS6.

I just forgot to thank you for the help! M.

comment:17 in reply to:  15 Changed 11 years ago by mmetz

Replying to martinl:

Right, it helps (committed in r36508). Anyway I do not understand why memory corruption was detected only in GRASS7 not GRASS6.

Thanks for committing. I think too that this memory corruption should have occurred before. As long as it works in grass6, I wouldn't fix it there. Shrug.

comment:18 Changed 11 years ago by martinl

Resolution: fixed
Status: closedreopened

Well, I cannot see any reason why memory was corrupted in v.info - only in GRASS7, but not in GRASS 6.5. When implementing very basic GEOS support in v.select I was facing to the similar problem.

In the loop (http://trac.osgeo.org/grass/browser/grass/trunk/vector/v.select/main.c#L176) is initialized ifield variable. Inside of the loop everything is OK - ifield[0] and ifield[1] are initialized properly. After leaving the loop in ifield[0] occurs random value.

More strangely when I add before line http://trac.osgeo.org/grass/browser/grass/trunk/vector/v.select/main.c#L37

struct GOption *in_opt[2];

memory is not corrupted at all.

I am going to be a little bit confused ;-) Has anybody any idea what is happening here. Compiling GRASS7 with

        CFLAGS="-g -Wall -Werror-implicit-function-declaration -fno-common -Wextra -Wunused" \
	    CXXFLAGS="-g -Wall"  \
	    ./configure --prefix=/usr/local \
	    --with-tcltk-includes=/usr/include/tcl8.4 --with-tcltk-libs=/usr/lib/tcl8.4 \
	    --with-postgres --with-postgres-includes=/usr/include/postgresql \
	    --with-mysql --with-mysql-includes=/usr/include/mysql \
	    --with-gdal=/usr/local/gdal-grass7/bin/gdal-config  \
	    --with-proj \
	    --with-motif --with-glw --with-nls --with-readline \
	    --with-cxx --enable-largefile \
	    --with-freetype --with-freetype-includes=/usr/include/freetype2 \
	    --with-sqlite --with-ffmpeg --with-ffmpeg-includes=/usr/include/ffmpeg/ \
	    --with-odbc --with-python=/usr/bin/python2.5-config --with-wxwidgets --with-pthread \
	    --with-ffmpeg=no --with-geos
gcc --version
gcc (Debian 4.3.3-5) 4.3.3

Thanks in advance. Martin

comment:19 in reply to:  18 ; Changed 11 years ago by mmetz

Replying to martinl:

When implementing very basic GEOS support in v.select I was facing to the similar problem.

In the loop (http://trac.osgeo.org/grass/browser/grass/trunk/vector/v.select/main.c#L176) is initialized ifield variable. Inside of the loop everything is OK - ifield[0] and ifield[1] are initialized properly. After leaving the loop in ifield[0] occurs random value.

More strangely when I add before line http://trac.osgeo.org/grass/browser/grass/trunk/vector/v.select/main.c#L37

struct GOption *in_opt[2];

memory is not corrupted at all.

Doesn't make any sense to me why ifield[0] should get a random value. Maybe just use the working solution?

FYI, I have already taken some algorithms from GEOS, translated from C++ to C, optimized where possible and put into the grass7 vector libs, replacing less efficient algorithms (topology building for areas). If you want some GEOS functionality to be available in the grass vector libs directly, we could join forces.

Best, Markus

comment:20 in reply to:  19 Changed 11 years ago by martinl

Replying to mmetz:

Doesn't make any sense to me why ifield[0] should get a random value. Maybe just use the working solution?

yes, there is something strange with GRASS7 compilation on my computer (note that I have no problems with GRASS6). E.g. debugging v.select

(gdb) b main.c:176 [for (iopt = 0; iopt < 2; iopt++) {]
(gdb) b main.c:184 [Vect_open_old(&(In[iopt]), parm.input[iopt]->answer, "");]
(gdb) b main.c:188 [IFi = Vect_get_field(&(In[0]), ifield[0]);]
(gdb) r
(gdb) p ifield[0]
$4 = -163754450
(gdb) c
(gdb) p ifield[0]
$5 = 1
(gdb) c
(gdb) c
(gdb) p ifield[0]
$10 = 142560728

Nice mystery ;-)

M.

comment:21 Changed 11 years ago by hamish

Summary: memory corruptionmemory corruption in vector modules

comment:22 Changed 11 years ago by mmetz

Martin, are you still observing memory corruption in vector modules in grass7?

If yes, is your system 32 bit or 64 bit? If 32 bit, does memory corruption disappear when configuring without --enable-largefile?

This memory corruption may have something to do with the new LFS support for vector libs, then I could have an idea how to fix it.

comment:23 Changed 11 years ago by mmetz

Hopefully fixed in r37861, please test.

comment:24 Changed 7 years ago by mmetz

Resolution: fixed
Status: reopenedclosed

No complaints in the last 3 years, closing as fixed.

Markus M

Note: See TracTickets for help on using tickets.