python script modules should use pythonw on OSX; they also ignore GRASS_PYTHON

[kyngchaos]

wxpython-based python scripts on OSX should be run with pythonw. Problem in 6.5 in the new v.krige, and of course in all 7.x scripts.

This is fine for the GUI itself, since the gui script is executed from Python (GRASS_PYTHON). But modules are executed directly, using the

#!/usr/bin/env/python

method.

For OSX, this should be

#!/usr/bin/env/pythonw

This will probably need to use some kind of .in file for all script modules that gets processed during make :(

Though, I see another potential problem with the #! -- it completely ignores GRASS_PYTHON, which could cause trouble if the user or packager wants to override the default python in the PATH. I don't know if it's possible to make the #! more dynamic.

[glynn]

Replying to kyngchaos:

wxpython-based python scripts on OSX should be run with pythonw. Problem in 6.5 in the new v.krige, and of course in all 7.x scripts.

Most 7.x scripts don't use wxPython directly. They might use it indirectly via g.parser, but that should be dealt with by $GRASS_PYTHON.

This is fine for the GUI itself, since the gui script is executed from Python (GRASS_PYTHON). But modules are executed directly, using the

#!/usr/bin/env/python

method.

That should be:

#!/usr/bin/env python

No slash between "env" and "python".

For OSX, this should be

#!/usr/bin/env/pythonw

This will probably need to use some kind of .in file for all script modules that gets processed during make :(

AFAICT, most scripts should be using "python", not "pythonw"

Though, I see another potential problem with the #! -- it completely ignores GRASS_PYTHON,

That shouldn't be a problem.

which could cause trouble if the user or packager wants to override the default python in the PATH. I don't know if it's possible to make the #! more dynamic.

It isn't. The kernel doesn't understand environment variables.

[kyngchaos]

Replying to glynn:

Most 7.x scripts don't use wxPython directly. They might use it indirectly via g.parser, but that should be dealt with by $GRASS_PYTHON.

OK. I was going by the new v.krige in 6.5, where I noticed the problem. So, the g.parser stuff happens in a completely separate process? What I saw in one module I looked at is that the script loads the grass module, then runs grass.parser(), then runs the main script. No chance for it to run anything with GRASS_PYTHON.

No slash between "env" and "python".

fingers acted on their own ;)

AFAICT, most scripts should be using "python", not "pythonw"

Except when they use wx, like v.krige.

Though, I see another potential problem with the #! -- it completely ignores GRASS_PYTHON,

That shouldn't be a problem.

How is that not a problem? Say there are 2 pythons in the PATH (or another not in the PATH) and I don't want to use the first one, so I set GRASS_PYTHON to the full path to the other. But then /usr/bin/env python[w] goes and runs the first one anyways.

[cmbarton]

For Mac OSX, I don't think we need to run pythonw anymore since 10.5 at least. The python.app launcher takes care of that. Scripts that start with /usr/bin/python start fine on my system, as do ones that start with /usr/bin/env python (testing this latter on 10.6).

Michael

[kyngchaos]

The pythonw requirement is a wxpython thing. At least they still say to use pythonw.

If you try to run a wxpython-based script from /usr/bin/env python (or pythonw), it will run 64bit by default on 10.6, which won't work because wxpython is not possible 64bit (yet).

The problem with the automatic pythonw is that the reexec didn't work and the GUI would never start on 10.6, and it even affected some non-OSX systems for some reason, so I removed the reexec and required pythonw to be used explicitly (r39207).

[cmbarton]

Yeah. We don't do the re-exec to pythonw. What I meant was that Apple's python launcher does now does the equivalent of execing in pythonw. So scripts that I used to have to run pythonw myscript.py now run with python myscript.py. I think this change came in with 10.5. It seems to work with wx too.

Michael

[kyngchaos]

Ah, I didn't know that. But,

1. That doesn't cover OSX < 10.5

2. That sounds like an Apple-only thing. What about the python.org Python binaries?

I say this because another Apple-only thing is that with /usr/bin/python you can select the architecture with an environment variable. This is not available in the python.org binaries. And it's a strange implementation: it responds to the env variable, but NOT to "arch -i386 python". BUT /usr/bin/python2.6 (which is a symlink to the framework executable) does respond to the arch command, but not to the env variable. So, does "python2.6" correctly switch to "pythonw2.6"?

Apple may think it made life simpler to execute graphic python scripts, but it made it more complex because now we have to consider different behaviors :(

[glynn]

Replying to kyngchaos:

Most 7.x scripts don't use wxPython directly. They might use it indirectly via g.parser, but that should be dealt with by $GRASS_PYTHON.

OK. I was going by the new v.krige in 6.5, where I noticed the problem. So, the g.parser stuff happens in a completely separate process?

Yes.

What I saw in one module I looked at is that the script loads the grass module, then runs grass.parser(), then runs the main script. No chance for it to run anything with GRASS_PYTHON.

grass.parser() runs g.parser, which calls G_parser(); if arguments are required but not provided (or if the --ui switch is given), this will run menuform.py via $GRASS_PYTHON (module_gui_wx() in lib/gis/parser.c). Eventually, g.parser will return the option/flag values back to the script, which doesn't care whether g.parser got the values from the command line or from a GUI dialog.

AFAICT, most scripts should be using "python", not "pythonw"

Except when they use wx, like v.krige.

Most scripts shouldn't be using wx. In fact, we may want a completely separate directory for this. IMHO, anything in the scripts directory should behave like a normal GRASS module (parse command line, perform processing, terminate). Applications are something else, even if they're written in Python.

Though, I see another potential problem with the #! -- it completely ignores GRASS_PYTHON,

That shouldn't be a problem.

How is that not a problem? Say there are 2 pythons in the PATH (or another not in the PATH) and I don't want to use the first one, so I set GRASS_PYTHON to the full path to the other. But then /usr/bin/env python[w] goes and runs the first one anyways.

That's already the case for every other Python script on your system (and similarly for /bin/sh etc). GRASS_PYTHON exists as a workaround for the case where the system Python cannot be used. AFAICT, the primary use case is where binary GRASS packages bundle a version of Python because that's the only way that we can ensure compatibility with extensions such as vdigit and nviz.

In any case, you can't use environment variables within the #! line (which is ignored on Windows anyhow). No other project seems to consider this a problem (not just for Python, but for any interpreted language).

[kyngchaos]

OK, here's r40103 + r40104 (dev6) and r40105 (trunk). I created a python wrapper so that "python" can be default for OSX and caught in /usr/bin/env python, and not depend on Apple's custom version to auto-reexec pythonw, and so that 32bits can be used as needed (solves another problem where I had an ugly workaround).

Can you check that it doesn't break the GUI, Michael (started OK in dev6).

Note that this is for the OSX app build only, but the idea (and the problem) does apply to an OSX unix build. For that, this stuff would have to be moved into init.sh and lib/init.

[cmbarton]

I haven't yet tried to compile since upgrading to Snow Leopard. I guess I'll try to do that tomorrow and see what happens. I'd like to try to compile for Python 2.5 32 bit if possible, so that I can see if it will run on 10.5.

Michael

[kyngchaos]

Haven't heard any complaints, closing (I also recently backported this to release 6.4).