g.gui.* modules which use temporal framework leave processes after exiting

[annakrat]

During start, g.gui.animation and g.gui.timeline create 2 subprocesses, specifically calling tgis.init() does. I suppose these are related to the RCP server recently implemented. The problem is that these processes are not finished when the gui module is closed. This is caused by the fact that g.gui.* modules are called in the background by ​GuiModuleMain function. Is there anything what we can do about it? I guess it's not a big deal when a user runs the module a few times but for me it's pretty annoying.

This applies to Linux and probably Mac OSX, too.

[huhabla]

This is really strange. The subprocesses created by tgis.init() for the messenger and the C-interface are marked as daemon processes (they will be terminated if the parent process will exit). In addition, the destructor of the messenger and C-interface objects explicitly terminate the subprocesses.

It seems that booth mechanism do not work with the GUI? AFAIC the destructors are not called when the GUI process exits. I have no idea why.

[huhabla]

I tried to solve the problem by registering a function at exit that explicitly terminates the subprocesses. I added this code to lib/python/temporal/core.py:

import atexit

def stop_subprocesses():
    """!Stop the messenger and C-interface subprocesses
    """
    global message_interface
    global c_library_interface
    if message_interface:
        message_interface.stop()
    if c_library_interface:
        c_library_interface.stop()

atexit.register(stop_subprocesses)

This works for temporal modules, but not for the GUI.

The registered function is not be called at exit when g.gui.animate terminates! How does the GUI framework terminates so that this functionality does not work?

Besides of that, it seems to me that tgis.init() is called several time when g.gui.animate is opened and a space time raster dataset is chosen for visualization. It hope that this can be reduced to a single tgis.init() call?

[huhabla]

All the methods that i have tried are not working in case the GUI module uses os.fork() to start the module in the background.

Why do we need fork() at all? Unix already support job management in the shell and it does not work on windows?

If we really need it, can this be implemented in a better way?

[annakrat]

Replying to huhabla:

Besides of that, it seems to me that tgis.init() is called several time when g.gui.animate is opened and a space time raster dataset is chosen for visualization. It hope that this can be reduced to a single tgis.init() call?

Sure, done in r58578.

[wenzeslaus]

Replying to huhabla:

All the methods that i have tried are not working in case the GUI module uses os.fork() to start the module in the background.

Why do we need fork() at all? Unix already support job management in the shell and it does not work on windows?

The idea was to start the application in the background (r57388, r57370 - r57378 launch GUI in the background). There was also a discussion on the mailing list with reasoning that g.gui.* should behave in the same way as g.gui, that these (g.gui*) are special cases (not able to find link now).

While GuiModuleMain​ is using Python os.fork, g.gui is using G_spawn​ from grass/spawn.h.

When the generated GUI for a module is shown (v.pack or g.region --ui), no special functionality is provided and it blocks terminal unless you use &.

If we really need it, can this be implemented in a better way?

[annakrat]

I just tried this python ​recipe and it worked for me, the processes are terminated. I tried to find the difference and it seems that just this line is enough (in the GuiModuleMain function):

--- core/utils.py	(revision 58561)
+++ core/utils.py	(working copy)
@@ -1064,7 +1064,8 @@
         child_pid = os.fork()
         if child_pid == 0:
             mainfn()
-        os._exit(0)
+        else:
+            os._exit(0)
     else:
         mainfn()

Can I apply it safely or can it cause some other troubles?

[annakrat]

Replying to annakrat:

I just tried this python ​recipe and it worked for me, the processes are terminated. I tried to find the difference and it seems that just this line is enough (in the GuiModuleMain function):

--- core/utils.py	(revision 58561)
+++ core/utils.py	(working copy)
@@ -1064,7 +1064,8 @@
         child_pid = os.fork()
         if child_pid == 0:
             mainfn()
-        os._exit(0)
+        else:
+            os._exit(0)
     else:
         mainfn()

Can I apply it safely or can it cause some other troubles?

Soeren had the same idea (r58580). Unfortunately, I can't say it's solved completely. I have still problem with g.gui.timeline. The subprocesses don't terminate when it's called with parameter input or when it's called without any parameter but then you display the dataset. When it's just opened and closed the subprocesses terminate. For g.gui.animation it's seems ok now. So I am confused what's different in g.gui.timeline. Well, at least some progress...

[huhabla]

It seems to me that the g.gui.timeline child process is still alive after the fork. The init process is the new parent of the forked child??

I have patched g.gui.timeline a make it a bit faster and to assure at least the termination of the messenger and C-interface subprocesses of the temporal framework:

timeline/frame.py

@@ -78 +78 @@
         self._layout()
         self.temporalType = None
         self.unit = None
+        # We create a database interface here to speedup the GUI
+        self.dbif = tgis.SQLDatabaseInterfaceConnection()
+        self.dbif.connect()
 
+    def __del__(self):
+        """!Close the database interface and stop the messenger and C-interface
+           subprocesses.
+        """
+        if self.dbif.connected is True:
+            self.dbif.close()
+        tgis.stop_subprocesses()
+
     def _layout(self):
         """!Creates the main panel with all the controls on it:
              * mpl canvas
@@ -145 +156 @@
         self.timeData = {}
         mode = None
         unit = None
+
         for series in timeseries:
             name = series[0] + '@' + series[1]
             etype = series[2]
             sp = tgis.dataset_factory(etype, name)
-            if not sp.is_in_db():
+            if not sp.is_in_db(dbif=self.dbif):
                 GError(self, message=_("Dataset <%s> not found in temporal database") % (name))
                 return
 
-            sp.select()
+            sp.select(dbif=self.dbif)
 
             self.timeData[name] = {}
             self.timeData[name]['elementType'] = series[2]
@@ -167 +179 @@
                 return
 
             # check topology
-            maps = sp.get_registered_maps_as_objects()
-            self.timeData[name]['validTopology'] = sp.check_temporal_topology(maps)
+            maps = sp.get_registered_maps_as_objects(dbif=self.dbif)
+            self.timeData[name]['validTopology'] = sp.check_temporal_topology(maps=maps, dbif=self.dbif)
 
             self.timeData[name]['temporalMapType'] = sp.get_map_time()  # point/interval
             self.timeData[name]['unit'] = None  # only with relative
@@ -194 +206 @@
                                 'north', 'south', 'west', 'east'])
 
             rows = sp.get_registered_maps(columns=columns, where=None,
-                                          order='start_time', dbif=None)
+                                          order='start_time', dbif=self.dbif)
             if rows is None:
                 rows = []
             for row in rows:
@@ -385 +397 @@
         @return (mapName, mapset, type)
         """
         validated = []
-        tDict = tgis.tlist_grouped('stds', group_type=True)
+        tDict = tgis.tlist_grouped('stds', group_type=True, dbif=self.dbif)
         # nested list with '(map, mapset, etype)' items
         allDatasets = [[[(map, mapset, etype) for map in maps]
                      for etype, maps in etypesDict.iteritems()]

The patch is not applied yet, since it does not solve the issue that the child process is still alive. Maybe an explicit os.exit() should be added to g.gui.timeline? How about an exit button?

[annakrat]

Replying to huhabla:

The patch is not applied yet, since it does not solve the issue that the child process is still alive. Maybe an explicit os.exit() should be added to g.gui.timeline? How about an exit button?

os.exit would close the whole wxGUI when the timeline tool is called from the wxGUI. Exit button would not solve anything, it just does the same as the standard x close button or Alt+F4. I tried it to be sure and there is no difference. I think you can commit your changes, it's improvement. Could the problem come from the matplotlib? Maybe I am using it in an incorrect way.

[huhabla]

Replying to annakrat:

Replying to huhabla:

The patch is not applied yet, since it does not solve the issue that the child process is still alive. Maybe an explicit os.exit() should be added to g.gui.timeline? How about an exit button?

os.exit would close the whole wxGUI when the timeline tool is called from the wxGUI. Exit button would not solve anything, it just does the same as the standard x close button or Alt+F4. I tried it to be sure and there is no difference. I think you can commit your changes, it's improvement. Could the problem come from the matplotlib? Maybe I am using it in an incorrect way.

I have applied the patch and included an explicit kill call in the destructor of TimelineFrame, see r58597. The kill signal will only be send if g.gui.timeline works in stand-alone mode (hopefully, please cross-check that).

Killing the process is a crude solution, but i have no better idea how to avoid orphaned daemon g.gui.timeline processes.

[annakrat]

Replying to huhabla:

Replying to annakrat:

Replying to huhabla:

The patch is not applied yet, since it does not solve the issue that the child process is still alive. Maybe an explicit os.exit() should be added to g.gui.timeline? How about an exit button?

os.exit would close the whole wxGUI when the timeline tool is called from the wxGUI. Exit button would not solve anything, it just does the same as the standard x close button or Alt+F4. I tried it to be sure and there is no difference. I think you can commit your changes, it's improvement. Could the problem come from the matplotlib? Maybe I am using it in an incorrect way.

I have applied the patch and included an explicit kill call in the destructor of TimelineFrame, see r58597. The kill signal will only be send if g.gui.timeline works in stand-alone mode (hopefully, please cross-check that).

Killing the process is a crude solution, but i have no better idea how to avoid orphaned daemon g.gui.timeline processes.

I found the problem and removed it (r58598, r58599). Now it should terminate all processes but please check. It was caused by a line of code which actually did nothing for me but I thought it might be working for some other version of matplotlib. I should have looked at it earlier. What about the changes you did? Part of them like the os.kill are redundant now, not sure about tgis.stop_subprocesses()?

[huhabla]

Replying to annakrat:

Replying to huhabla:

Replying to annakrat:

Replying to huhabla:

The patch is not applied yet, since it does not solve the issue that the child process is still alive. Maybe an explicit os.exit() should be added to g.gui.timeline? How about an exit button?

os.exit would close the whole wxGUI when the timeline tool is called from the wxGUI. Exit button would not solve anything, it just does the same as the standard x close button or Alt+F4. I tried it to be sure and there is no difference. I think you can commit your changes, it's improvement. Could the problem come from the matplotlib? Maybe I am using it in an incorrect way.

I have applied the patch and included an explicit kill call in the destructor of TimelineFrame, see r58597. The kill signal will only be send if g.gui.timeline works in stand-alone mode (hopefully, please cross-check that).

Killing the process is a crude solution, but i have no better idea how to avoid orphaned daemon g.gui.timeline processes.

I found the problem and removed it (r58598, r58599). Now it should terminate all processes but please check. It was caused by a line of code which actually did nothing for me but I thought it might be working for some other version of matplotlib. I should have looked at it earlier. What about the changes you did? Part of them like the os.kill are redundant now, not sure about tgis.stop_subprocesses()?

I kept tgis.stop_subprocesses() within the destructor to assure the termination of the messenger and C-interface subprocesses when the time line tool was started from the main GUI. The explicit kill and dependent code was removed in svn (r58602 and r58609). It works for me, no orphaned processes after closing g.gui.timeline.

I will close this ticket, marking it as fixed.

[annakrat]

Replying to huhabla:

I will close this ticket, marking it as fixed.

Great, thanks for help