ctype.POINTER function overloading prevents automatic byref() conversion

[artegion]

From "​https://docs.python.org/2/library/ctypes.html#structured-data-types" "In addition, if a function argument is explicitly declared to be a pointer type (such as POINTER(c_int)) in argtypes, an object of the pointed type (c_int in this case) can be passed to the function. ctypes will apply the required byref() conversion in this case automatically."

grass.lib module overwrites ctype.POINTER function in ctypes_preamble module. Consequently function arguments types are not recognized:

i.e. Vect_list_append.argtypes reports

<class 'grass.lib.ctypes_preamble.LP_struct_ilist'>

instead of

<class 'grass.lib.vector.LP_struct_ilist'>

and the required byref() conversion does't happen but ctpyes.byref explicit call is needed

example:

>>> from ctypes import byref
>>> from grass.lib.vector import struct_ilist, Vect_list_append
>>> a= struct_ilist()
>>> a.__class__
<class 'grass.lib.vector.struct_ilist'>
>>> Vect_list_append.argtypes
[<class 'grass.lib.ctypes_preamble.LP_struct_ilist'>, <class 'ctypes.c_long'>]
>>>
>>>
>>> if Vect_list_append(a, 1):
...    print "direct call failed"
... else:
...    print a.n_values, a.value[a.n_values-1]
...
direct call failed
>>>
>>>
>>> if Vect_list_append(byref(a), 2):
...    print "byref call failed"
... else:
...    print a.n_values, a.value[a.n_values-1]
...
1 2
>>>

Using a modified version of grass.lib.vector (added

from ctypes import POINTER

after

from ctypes_preamble import *

everything seems fine...

>>> from ctypes import byref
>>> from grassmod.lib.vector import struct_ilist, Vect_list_append
>>> a= struct_ilist()
>>> a.__class__
<class 'grassmod.lib.vector.struct_ilist'>
>>> Vect_list_append.argtypes
[<class 'grassmod.lib.vector.LP_struct_ilist'>, <class 'ctypes.c_long'>]
>>>
>>>
>>> if Vect_list_append(a, 1):
...    print "direct call failed"
... else:
...    print a.n_values, a.value[a.n_values-1]
...
1 1
>>>
>>>
>>> if Vect_list_append(byref(a), 2):
...    print "byref call failed"
... else:
...    print a.n_values, a.value[a.n_values-1]
...
2 2
>>>

[glynn]

Replying to artegion:

grass.lib module overwrites ctype.POINTER function in ctypes_preamble module.

The replacement POINTER function has the following comment:

    # Convert None to a real NULL pointer to work around bugs
    # in how ctypes handles None on 64-bit platforms

Have you tested whether this is still an issue (it might have been fixed since ctypesgen was last updated)?

Having to use byref() explicitly seems less of an issue than not working on 64-bit systems.

Actually, if an argument is a pointer, the caller should probably be using byref() explicitly regardless of whether or not ctypes has a workaround. What's the problem with needing to use byref()?

[artegion]

It seems a known bug of ctypesgen ​https://github.com/davidjamesca/ctypesgen/issues/26

[artegion]

Not a real problem, just a bit disappointing: python ctypes documentation describes an interesting feature but our code presents a different behavior (it is far more pythonic do not care about byreference/byvalue if ctypes can do the dirty work).

The odd thing is that happens unnoticed: while ctypes raises exceptions passing wrong argument type in this case only function return code (or segmentation fault exception) reveals troubles.

[artegion]

Replying to glynn:

Replying to artegion:

grass.lib module overwrites ctype.POINTER function in ctypes_preamble module.

The replacement POINTER function has the following comment:

    # Convert None to a real NULL pointer to work around bugs
    # in how ctypes handles None on 64-bit platforms

Have you tested whether this is still an issue (it might have been fixed since ctypesgen was last updated)?

Having to use byref() explicitly seems less of an issue than not working on 64-bit systems.

Actually, if an argument is a pointer, the caller should probably be using byref() explicitly regardless of whether or not ctypes has a workaround. What's the problem with needing to use byref()?

Not a real problem, just a bit disappointing: python ctypes documentation describes an interesting feature but our code presents a different behavior (it is far more pythonic do not care about byreference/byvalue if ctypes can do the dirty work).

The odd thing is that happens unnoticed: while ctypes raises exceptions passing wrong argument type in this case only function return code (or segmentation fault exception) reveals troubles.

[glynn]

Replying to artegion:

It seems a known bug of ctypesgen ​https://github.com/davidjamesca/ctypesgen/issues/26

That suggests that it's working around an issue which is specific to Python 2.5. Do we still support that? If not, the workaround can be removed.

[neteler]

Replying to glynn:

That suggests that it's working around an issue which is specific to Python 2.5. Do we still support that? If not, the workaround can be removed.

According to ​https://svn.osgeo.org/grass/grass/branches/releasebranch_7_0/REQUIREMENTS.html we support Python >= 2.6

[neteler]

Ticket retargeted after milestone closed

[neteler]

Ticket retargeted after 7.0.3 milestone closed

[neteler]

Replying to glynn:

Replying to artegion:

It seems a known bug of ctypesgen ​https://github.com/davidjamesca/ctypesgen/issues/26

That suggests that it's working around an issue which is specific to Python 2.5. Do we still support that? If not, the workaround can be removed.

Since we require at least Python >= 2.6, let's remove the workaround...