Opened 4 years ago

Closed 10 months ago

#2748 closed defect (worksforme)

ctype.POINTER function overloading prevents automatic byref() conversion

Reported by: artegion Owned by: grass-dev@…
Priority: normal Milestone: 7.6.2
Component: Python ctypes Version: svn-trunk
Keywords: ctypes Cc:
CPU: Unspecified Platform: Unspecified

Description

From "https://docs.python.org/2/library/ctypes.html#structured-data-types" "In addition, if a function argument is explicitly declared to be a pointer type (such as POINTER(c_int)) in argtypes, an object of the pointed type (c_int in this case) can be passed to the function. ctypes will apply the required byref() conversion in this case automatically."

grass.lib module overwrites ctype.POINTER function in ctypes_preamble module. Consequently function arguments types are not recognized:

i.e. Vect_list_append.argtypes reports

<class 'grass.lib.ctypes_preamble.LP_struct_ilist'>

instead of

<class 'grass.lib.vector.LP_struct_ilist'>

and the required byref() conversion does't happen but ctpyes.byref explicit call is needed

example:

>>> from ctypes import byref
>>> from grass.lib.vector import struct_ilist, Vect_list_append
>>> a= struct_ilist()
>>> a.__class__
<class 'grass.lib.vector.struct_ilist'>
>>> Vect_list_append.argtypes
[<class 'grass.lib.ctypes_preamble.LP_struct_ilist'>, <class 'ctypes.c_long'>]
>>>
>>>
>>> if Vect_list_append(a, 1):
...    print "direct call failed"
... else:
...    print a.n_values, a.value[a.n_values-1]
...
direct call failed
>>>
>>>
>>> if Vect_list_append(byref(a), 2):
...    print "byref call failed"
... else:
...    print a.n_values, a.value[a.n_values-1]
...
1 2
>>>

Using a modified version of grass.lib.vector (added

from ctypes import POINTER

after

from ctypes_preamble import *

everything seems fine...

>>> from ctypes import byref
>>> from grassmod.lib.vector import struct_ilist, Vect_list_append
>>> a= struct_ilist()
>>> a.__class__
<class 'grassmod.lib.vector.struct_ilist'>
>>> Vect_list_append.argtypes
[<class 'grassmod.lib.vector.LP_struct_ilist'>, <class 'ctypes.c_long'>]
>>>
>>>
>>> if Vect_list_append(a, 1):
...    print "direct call failed"
... else:
...    print a.n_values, a.value[a.n_values-1]
...
1 1
>>>
>>>
>>> if Vect_list_append(byref(a), 2):
...    print "byref call failed"
... else:
...    print a.n_values, a.value[a.n_values-1]
...
2 2
>>>

Change History (14)

comment:1 in reply to:  description ; Changed 4 years ago by glynn

Replying to artegion:

grass.lib module overwrites ctype.POINTER function in ctypes_preamble module.

The replacement POINTER function has the following comment:

    # Convert None to a real NULL pointer to work around bugs
    # in how ctypes handles None on 64-bit platforms

Have you tested whether this is still an issue (it might have been fixed since ctypesgen was last updated)?

Having to use byref() explicitly seems less of an issue than not working on 64-bit systems.

Actually, if an argument is a pointer, the caller should probably be using byref() explicitly regardless of whether or not ctypes has a workaround. What's the problem with needing to use byref()?

comment:2 Changed 4 years ago by artegion

It seems a known bug of ctypesgen https://github.com/davidjamesca/ctypesgen/issues/26

comment:3 Changed 4 years ago by artegion

Last edited 4 years ago by artegion (previous) (diff)

comment:4 in reply to:  1 Changed 4 years ago by artegion

Replying to glynn:

Replying to artegion:

grass.lib module overwrites ctype.POINTER function in ctypes_preamble module.

The replacement POINTER function has the following comment:

    # Convert None to a real NULL pointer to work around bugs
    # in how ctypes handles None on 64-bit platforms

Have you tested whether this is still an issue (it might have been fixed since ctypesgen was last updated)?

Having to use byref() explicitly seems less of an issue than not working on 64-bit systems.

Actually, if an argument is a pointer, the caller should probably be using byref() explicitly regardless of whether or not ctypes has a workaround. What's the problem with needing to use byref()?

Not a real problem, just a bit disappointing: python ctypes documentation describes an interesting feature but our code presents a different behavior (it is far more pythonic do not care about byreference/byvalue if ctypes can do the dirty work).

The odd thing is that happens unnoticed: while ctypes raises exceptions passing wrong argument type in this case only function return code (or segmentation fault exception) reveals troubles.

comment:5 in reply to:  2 ; Changed 4 years ago by glynn

Replying to artegion:

It seems a known bug of ctypesgen https://github.com/davidjamesca/ctypesgen/issues/26

That suggests that it's working around an issue which is specific to Python 2.5. Do we still support that? If not, the workaround can be removed.

comment:6 in reply to:  5 Changed 4 years ago by neteler

Milestone: 7.0.1

Replying to glynn:

That suggests that it's working around an issue which is specific to Python 2.5. Do we still support that? If not, the workaround can be removed.

According to https://svn.osgeo.org/grass/grass/branches/releasebranch_7_0/REQUIREMENTS.html we support Python >= 2.6

comment:7 Changed 4 years ago by neteler

Keywords: ctypes added
Milestone: 7.0.17.0.3
Version: 7.0.1svn-trunk

comment:8 Changed 4 years ago by neteler

Milestone: 7.0.3

Ticket retargeted after milestone closed

comment:9 Changed 4 years ago by neteler

Milestone: 7.0.4

Ticket retargeted after 7.0.3 milestone closed

comment:10 in reply to:  5 Changed 4 years ago by neteler

Replying to glynn:

Replying to artegion:

It seems a known bug of ctypesgen https://github.com/davidjamesca/ctypesgen/issues/26

That suggests that it's working around an issue which is specific to Python 2.5. Do we still support that? If not, the workaround can be removed.

Since we require at least Python >= 2.6, let's remove the workaround...

comment:11 Changed 4 years ago by martinl

Milestone: 7.0.47.0.5

comment:12 Changed 3 years ago by neteler

Milestone: 7.0.57.0.6

comment:13 Changed 2 years ago by neteler

Milestone: 7.0.67.0.7

comment:14 Changed 10 months ago by martinl

Milestone: 7.0.77.6.2
Resolution: worksforme
Status: newclosed
Note: See TracTickets for help on using tickets.