vector digitizer unstable

[cmbarton]

The vector digitizer is very unstable. Sometimes I got it to make a new map or edit an existing one, but other times this caused the GUI to crash. When it did work, I was unable to close out the vector digitizer and have the menubar deleted. This, in turn made it impossible to quit the GUI by normal means.

The error (pasted below) is a python segmentation fault: 11 error. This is the same error that crashes the GUI when launching i.gui.iclass (ticket ​https://trac.osgeo.org/grass/ticket/34722). Perhaps it has the same underlying cause.

This error happens with new Mac binaries that are full 64bit, using the current stable wxPython 3.0.2, and Python 2.7.14.

It is a problem with both GRASS 7.2.2 and GRASS 7.4 svn

[cmbarton]

Forgot to add the error:

/private/var/folders/65/pp9w7z0d1mj502pj8hhl7vfw0000gp/T/AppTranslocation/D15EBED1-582C-4F2B-A41A-2C96BE511484/d/GRASS-7.4svn.app/Contents/Resources/bin/pythonw: line 3: 48809 Segmentation fault: 11 /Applications/GRASS-7.4svn.app/Contents/Resources/python.app/Contents/MacOS/python "$@"

[cmbarton]

Upgrading this to critical. GRASS functionality as a full-featured GIS is severely limited without a digitizer. Also, we need to ensure that the GUI code is compatible with 64bit wxPython. wxPython 3.0.2 is the current stable version.

[balagates]

I believe I am experiencing this same bug. I have a new build of Grass 7.4.0 using MacPorts. I am using macOS 10.12.6. I had similar problems with Grass 7.2. I was able to use vdigit in Grass 7.0 but my OS and build process were likely different.

The GUI crash always seems to happen (it is not intermittend for me) with three different ways of launching vdigit. From the Layer Manager I have tried 1) right clicking on vector and selecting "start editing", 2) pressing the button "edit selected vector map". From the Grass display I can successfully change the mode to "vector digitizer" which shows the vdigit interface but when I select the vector from the pulldown I get the crash.

I have various crash logs and they all seem similar. The "Performing @selector(clickedAction:)" is slightly different based on which path I took to cause the crash. Parts of one, through the Display window, are pasted below.

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x00000000000000ad
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [0]

VM Regions Near 0xad:
--> 
    __TEXT                 0000000103706000-0000000103708000 [    8K] r-x/rwx SM=COW  /opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python

Application Specific Information:
Performing @selector(clickedAction:) from sender wxNSToolBarButton 0x7ff46c80f770

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   _ctypes.so                    	0x000000010b498628 PyObject_stgdict + 8
1   _ctypes.so                    	0x000000010b497360 ConvParam + 32
2   _ctypes.so                    	0x000000010b496ead _ctypes_callproc + 205
3   _ctypes.so                    	0x000000010b4909cd PyCFuncPtr_call + 1101
4   org.python.python             	0x000000010371dfe5 PyObject_Call + 101
5   org.python.python             	0x00000001037c49c3 PyEval_EvalFrameEx + 21123
6   org.python.python             	0x00000001037bf504 PyEval_EvalCodeEx + 2132
7   org.python.python             	0x00000001037c86fd fast_function + 109
8   org.python.python             	0x00000001037c4825 PyEval_EvalFrameEx + 20709
9   org.python.python             	0x00000001037bf504 PyEval_EvalCodeEx + 2132
10  org.python.python             	0x00000001037c86fd fast_function + 109
11  org.python.python             	0x00000001037c4825 PyEval_EvalFrameEx + 20709
12  org.python.python             	0x00000001037c87e3 fast_function + 339
13  org.python.python             	0x00000001037c4825 PyEval_EvalFrameEx + 20709
14  org.python.python             	0x00000001037c87e3 fast_function + 339
15  org.python.python             	0x00000001037c4825 PyEval_EvalFrameEx + 20709
16  org.python.python             	0x00000001037bf504 PyEval_EvalCodeEx + 2132
17  org.python.python             	0x0000000103744b31 function_call + 337
18  org.python.python             	0x000000010371dfe5 PyObject_Call + 101
19  org.python.python             	0x000000010372bf02 instancemethod_call + 162
20  org.python.python             	0x000000010371dfe5 PyObject_Call + 101
21  org.python.python             	0x00000001037c80ff PyEval_CallObjectWithKeywords + 159
22  _core_.so                     	0x000000010414d5a8 wxPyCallback::EventThunker(wxEvent&) + 504
23  libwx_baseu-3.0.dylib         	0x0000000104cc2207 wxEvtHandler::ProcessEventIfMatchesId(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&) + 93
24  libwx_baseu-3.0.dylib         	0x0000000104cc353f wxEvtHandler::SearchDynamicEventTable(wxEvent&) + 89
25  libwx_baseu-3.0.dylib         	0x0000000104cc3490 wxEvtHandler::TryHereOnly(wxEvent&) + 40
26  libwx_baseu-3.0.dylib         	0x0000000104cc33d4 wxEvtHandler::ProcessEventLocally(wxEvent&) + 40
27  libwx_baseu-3.0.dylib         	0x0000000104cc3367 wxEvtHandler::ProcessEvent(wxEvent&) + 185
28  libwx_baseu-3.0.dylib         	0x0000000104cc35ab wxEvtHandler::SafelyProcessEvent(wxEvent&) + 15
29  libwx_osx_cocoau_core-3.0.dylib	0x0000000104821259 wxToolBarBase::OnLeftClick(int, bool) + 79
30  libsystem_trace.dylib         	0x00007fff9d04f3a7 _os_activity_initiate_impl + 53
31  com.apple.AppKit              	0x00007fff854c7721 -[NSApplication(NSResponder) sendAction:to:from:] + 456
32  com.apple.AppKit              	0x00007fff84fabcc4 -[NSControl sendAction:to:] + 86
33  com.apple.AppKit              	0x00007fff84fabbec __26-[NSCell _sendActionFrom:]_block_invoke + 136
34  libsystem_trace.dylib         	0x00007fff9d04f3a7 _os_activity_initiate_impl + 53
35  com.apple.AppKit              	0x00007fff84fabb44 -[NSCell _sendActionFrom:] + 128
36  com.apple.AppKit              	0x00007fff84fee539 -[NSButtonCell _sendActionFrom:] + 98
37  libsystem_trace.dylib         	0x00007fff9d04f3a7 _os_activity_initiate_impl + 53
38  com.apple.AppKit              	0x00007fff84faa426 -[NSCell trackMouse:inRect:ofView:untilMouseUp:] + 2481
39  com.apple.AppKit              	0x00007fff84fee272 -[NSButtonCell trackMouse:inRect:ofView:untilMouseUp:] + 798
40  com.apple.AppKit              	0x00007fff84fa8ddb -[NSControl mouseDown:] + 832
41  com.apple.AppKit              	0x00007fff8564324f -[NSWindow(NSEventRouting) _handleMouseDownEvent:isDelayedEvent:] + 6341
42  com.apple.AppKit              	0x00007fff8563fa6c -[NSWindow(NSEventRouting) _reallySendEvent:isDelayedEvent:] + 1942
43  com.apple.AppKit              	0x00007fff8563ef0a -[NSWindow(NSEventRouting) sendEvent:] + 541
44  libwx_osx_cocoau_core-3.0.dylib	0x0000000104742bcc -[wxNSWindow sendEvent:] + 117
45  com.apple.AppKit              	0x00007fff854c3681 -[NSApplication(NSEvent) sendEvent:] + 1145
46  libwx_osx_cocoau_core-3.0.dylib	0x00000001046a433e -[wxNSApplication sendEvent:] + 98
47  com.apple.AppKit              	0x00007fff84d3e427 -[NSApplication run] + 1002
48  libwx_osx_cocoau_core-3.0.dylib	0x000000010473a1ec wxGUIEventLoop::OSXDoRun() + 158
49  libwx_baseu-3.0.dylib         	0x0000000104c9b3af wxCFEventLoop::DoRun() + 39
50  libwx_baseu-3.0.dylib         	0x0000000104bfeefd wxEventLoopBase::Run() + 165
51  libwx_baseu-3.0.dylib         	0x0000000104bd0cfa wxAppConsoleBase::MainLoop() + 102
52  _core_.so                     	0x0000000104148954 wxPyApp::MainLoop() + 84
53  _core_.so                     	0x00000001041aa212 _wrap_PyApp_MainLoop(_object*, _object*) + 82
54  org.python.python             	0x00000001037c04b0 PyEval_EvalFrameEx + 3440
55  org.python.python             	0x00000001037bf504 PyEval_EvalCodeEx + 2132
56  org.python.python             	0x0000000103744b31 function_call + 337
57  org.python.python             	0x000000010371dfe5 PyObject_Call + 101
58  org.python.python             	0x000000010372bf02 instancemethod_call + 162
59  org.python.python             	0x000000010371dfe5 PyObject_Call + 101
60  org.python.python             	0x00000001037c49c3 PyEval_EvalFrameEx + 21123
61  org.python.python             	0x00000001037c87e3 fast_function + 339
62  org.python.python             	0x00000001037c4825 PyEval_EvalFrameEx + 20709
63  org.python.python             	0x00000001037bf504 PyEval_EvalCodeEx + 2132
64  org.python.python             	0x00000001037c86fd fast_function + 109
65  org.python.python             	0x00000001037c4825 PyEval_EvalFrameEx + 20709
66  org.python.python             	0x00000001037bf504 PyEval_EvalCodeEx + 2132
67  org.python.python             	0x00000001037beca2 PyEval_EvalCode + 34
68  org.python.python             	0x00000001037eac1d PyRun_FileExFlags + 157
69  org.python.python             	0x00000001037ea799 PyRun_SimpleFileExFlags + 697
70  org.python.python             	0x000000010380080e Py_Main + 3006
71  libdyld.dylib                 	0x00007fff9ce1d235 start + 1

[cmbarton]

Here is possible clue to solving this:

​https://groups.google.com/forum/#!topic/wxpython-users/1mhTqSHvhTc

[cmbarton]

Also affects 7.4.0 and 7.5 trunk.

[neteler]

This may be related to the known and yet unsolved ctypes portability issues.

[cmbarton]

Crash error log

[cmbarton]

This is still a very critical error. There is no working digitizer for GRASS Mac. Attempting to create a new vector map or selecting an existing vector map, when you hit return it crashes the entire GUI. I am attaching a file of the Apple error report because the crash leaves nothing in the GRASS terminal, and the GUI crash means you can't see the console. This is a serious bug.

Other ctypes errors seem to have been fixed and this one persists.

[neteler]

Please test in current trunk with the new Python3 support [1].

[1] ​https://trac.osgeo.org/grass/wiki/Python3Support#Howtotest

[cmbarton]

I'll have to work out a way to compile this under Anaconda. It will be good to start this transition to Python 3 and especially to have the digitizer working. You are implying that this fix will not work with Python 2.7?

[neteler]

Replying to cmbarton:

I'll have to work out a way to compile this under Anaconda. It will be good to start this transition to Python 3 and especially to have the digitizer working.

Yes!

You are implying that this fix will not work with Python 2.7?

No but Python2 is end-of-life soon, see e.g. ​https://pythonclock.org/

[cmbarton]

Ah. I would like to test under Python 2 so as not to have too many things varying at once.

Then I'll work with Eric to see about how to compile this under Anaconda. It seems like it will be quite straightforward, just switching the requirements file to P3 instead of P2.7. But we'll see.

[cmbarton]

I am hoping that I don't hit the wall I've got with 7.4.2 right now.

[neteler]

Replying to cmbarton:

Ah. I would like to test under Python 2 so as not to have too many things varying at once.

I am not sure if the changes are related to Python 2 at all, I doubt it.

Then I'll work with Eric to see about how to compile this under Anaconda. It seems like it will be quite straightforward, just switching the requirements file to P3 instead of P2.7. But we'll see.

Yes, ... perhaps better invested time :-)

[cmbarton]

Unfortunately, I can't test this until we figure out why configure is looking for cpp and bombing when it doesn't find it now.

[martinl]

Ticket retargeted after milestone closed

[neteler]

Replying to cmbarton:

Unfortunately, I can't test this until we figure out why configure is looking for cpp and bombing when it doesn't find it now.

I'd suggest to move this to the 7.8 milestone, with focus on Python-3.

[cmbarton]

Replying to neteler:

Replying to cmbarton:

Unfortunately, I can't test this until we figure out why configure is looking for cpp and bombing when it doesn't find it now.

I'd suggest to move this to the 7.8 milestone, with focus on Python-3.

GRASS is compiling again. So I can test any fix.

I disagree with kicking this can down the road even farther. Digitizing has been broken on the Mac since 7.2. And now we are at 7.7 in trunk. It is a serious problem if you can't digitize in a full-featured GIS. I am skeptical that it is something that Python 3 will magically solve. Previously, the thought was that moving from 32 bit wxPython 2.8.12 to 64 bit wxPython 3 would fix it. We are now using wxPython 4.0.1 on the Mac and it still crashes.

It has something to do with how interactive windows are being defined/created because the same problem affects the interactive supervised classification interface. But is does NOT affect other interactive window functions like measurement and selection. One clue to tracking the bug perhaps is the following. In the digitizer, it does not crash as soon as the digitizing functions are invoked, but only when a new vector layer to be digitized is defined and "OK" is pressed. For the supervised classification, it crashes as soon as the interface is called.

[balagates]

Console output with DEBUG=5 and WX_DEBUG=5 during crash

[balagates]

I attached a file with console output with DEBUG=5 and WX_DEBUG=5 during crash using the downloaded version 7.6.0 on macOS 10.12.6. Maybe the "RenderMapMgr.Abort()" should be investigated?

I also see a likely bug when changing the Map Display mode to Vector digitizer. I believe the message "Expecting 825x586 image but got 825x622 image." is caused by the vector digitizer controls being added but the map size not being adjusted. I do not think this is causing the crash. After changing the mode you can resize the window to eliminate that error.

Another clue might be that after a crash the topology for the vector is messed up and needs to be rebuilt. That might only mean it got far enough to open the vector for editing but did not close it properly.

[cmbarton]

Thanks for the additional info

[martinl]

Ticket retargeted after milestone closed

[nila]

This is still an issue with 7.8 branch and master (7.9.dev), with Python 3.8.2 and wxPython 4.0.7.post2.

I managed to debug in steps:

scripts/g.gui.vdigit:80 VDigitMapFrame()
gui/wxpython/vdigit/toolbars.py:913 VDigitToolbar.StartEditing()
gui/wxpython/vdigit/wxdigit.py:1719 IVDigit.OpenMap()
gui/wxpython/vdigit/wxdisplay.py:1021 DisplayDriver.OpenMap()
lib/vector/Vlib/open.c:162-555 Vect__open_old)

Although it returns successfully (seemingly) from Vect__open_old, it crashes at a point (but rarely identical point) soon after (while leaving scope). This is why topology has to be rebuilt. While not yet been able to pinpoint the cause, it seems to be a background thread causing the crash. I'm thinking in lines of either wx binding issues or python garbage collection. Both of which may behave just enough different than on other platforms to make it a mac problem.

[neteler]

I just tested it on Fedora, the recently updated version is working well.

Please also test it.

[nila]

I have been able to track the origin of the problem. It is python ctypes related.

(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x100000019)
  * frame #0: 0x000000010d4ca8f2 Python`_PyObject_IsFreed(op=0x0000000100000001) at object.c:423:56
    frame #1: 0x000000010d4caa42 Python`_PyObject_Dump(op=0x0000000100000001) at object.c:450:9
    frame #2: 0x000000010d4ced5a Python`_Py_ForgetReference(op=0x0000000124b5c970) at object.c:1950:9
    frame #3: 0x000000010d4c95d5 Python`_Py_Dealloc(op=0x0000000124b5c970) at object.c:1973:5
    frame #4: 0x00000001175a8605 _ctypes.cpython-37dm-darwin.so`_ctypes_callproc(pProc=(libgrass_vector.7.9.dylib`Vect_open_update at open.c:641), argtuple=0x0000000115e64960, flags=4353, argtypes=0x0000000120895ce0, restype=0x00007fcd62120e40, checker=0x0000000000000000) at callproc.c:1226:9
    frame #5: 0x0000000117598f23 _ctypes.cpython-37dm-darwin.so`PyCFuncPtr_call(self=0x0000000120893560, inargs=0x0000000115e64960, kwds=0x0000000000000000) at _ctypes.c:4025:14
    frame #6: 0x000000010d44763a Python`_PyObject_FastCallKeywords(callable=0x0000000120893560, stack=0x00007fcd64629840, nargs=3, kwnames=0x0000000000000000) at call.c:199:18

The call (starting from ​wxdisplay.py#L1021) to GRASS c code function Vect_open_update() returns successfully, but ctypes' _ctypes_callproc() internal cleanup at ​callproc.c#L1226 hits bad memory access.

I'm not sure yet whether the issue is fixable within GRASS or if it has to be dealt with upstreams.

Sidenote: I made a python script calling only the relevant code without the gui with the same result (segfault), this eliminates the possibility of event (binding) related effects.

Additionally, the reported issue on "Expecting 825x586 image but got 825x622 image." ​comment:19 is an equally critical but a separate one to the originally reported issue. The following seems to fix this latter issue (but I'd wait to PR this before a solution for both of them can be presented):

diff --git a/gui/wxpython/mapdisp/toolbars.py b/gui/wxpython/mapdisp/toolbars.py
index e8b08ad79..9bce3a0ef 100644
--- a/gui/wxpython/mapdisp/toolbars.py
+++ b/gui/wxpython/mapdisp/toolbars.py
@@ -233,6 +233,7 @@ class MapToolbar(BaseToolbar):
         """Select / enable tool available in tools list
         """
         tool = event.GetSelection()
+        self.parent.mapWindowProperties.autoRender = False
 
         if tool == self.toolId['2d']:
             self.ExitToolbars()
@@ -253,6 +254,8 @@ class MapToolbar(BaseToolbar):
             self.ExitToolbars()
             self.parent.AddRDigit()
 
+        self.parent.mapWindowProperties.autoRender = True
+
     def OnAnalyze(self, event):
         """Analysis tools menu
         """

[nila]

This was a very tricky issue, but with a really simple solution. The ctypesgen generated python files are rendered incomplete, in this case in particular include/vect/dig_structs.h and the python mapping of Map_info.

E.g. the following parts of etc/python/grass/lib/vector.py were missing:

struct_Map_info.__slots__ = [
...
struct_Map_info._fields_ = [
..

This can be fixed with the addition of the compiler flag -D_Nullable=.

A fix for this issue has been proposed with ​PR456.

[annakrat]

Thanks for the fix. Merged into master and 7.8. Please create a new ticket if similar issues persist.