Opened 7 years ago

Closed 6 years ago

#3331 closed defect (fixed)

ctypes: ValueError: invalid literal for int() with base 8: '08420217248550443400745280086994171'

Reported by: neteler Owned by: grass-dev@…
Priority: normal Milestone: 7.0.6
Component: Python Version: svn-releasebranch72
Keywords: ctypes, python Cc:
CPU: Unspecified Platform: Unspecified

Description

While compiling GRASS GIS 7.2.1RC1 on Fedora26, I get

...
python -t -3 -m py_compile /builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/etc/python/grass/lib/ctypes_preamble.py
python -t -3 -m py_compile /builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/etc/python/grass/lib/ctypes_loader.py
GISRC=/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/demolocation/.grassrc72 GISBASE=/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu PATH="/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/bin:/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/bin:/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/scripts:$PATH" PYTHONPATH="/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/etc/python:/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/gui/wxpython:$PYTHONPATH" LD_LIBRARY_PATH="/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/bin:/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/bin:/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/scripts:/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/lib:/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/lib:" LC_ALL=C ./ctypesgen.py --cpp "gcc -E       -I/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/include -I/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG" -lgrass_datetime.7.2.1   /builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/include/grass/datetime.h /builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/include/grass/defs/datetime.h -o OBJ.x86_64-redhat-linux-gnu/date.py
GISRC=/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/demolocation/.grassrc72 GISBASE=/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu PATH="/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/bin:/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/bin:/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/scripts:$PATH" PYTHONPATH="/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/etc/python:/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/gui/wxpython:$PYTHONPATH" LD_LIBRARY_PATH="/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/bin:/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/bin:/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/scripts:/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/lib:/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/lib:" LC_ALL=C ./ctypesgen.py --cpp "gcc -E       -I/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/include -I/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG" -lgrass_gis.7.2.1   /builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/include/grass/gis.h /builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/include/grass/defs/gis.h -o OBJ.x86_64-redhat-linux-gnu/gis.py
Status: Preprocessing /tmp/tmp4SEb0S.h
Status: gcc -E       -I/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/include -I/builddir/build/BUILD/grass-7.2.1/dist.x86_64-redhat-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG -U __GNUC__ -dD "-Dinline=" "-D__inline__=" "-D__extension__=" "-D_Bool=uint8_t" "-D__const=const" "-D__asm__(x)=" "-D__asm(x)=" "-DCTYPESGEN=1" /tmp/tmp4SEb0S.h
Traceback (most recent call last):
  File "./ctypesgen.py", line 139, in <module>
    descriptions = ctypesgencore.parser.parse(options.headers, options)
  File "/builddir/build/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/__init__.py", line 22, in parse
    parser.parse()
  File "/builddir/build/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/datacollectingparser.py", line 74, in parse
    ctypesparser.CtypesParser.parse(self, fname, None)
  File "/builddir/build/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/cparser.py", line 120, in parse
    self.preprocessor_parser.parse(filename)
  File "/builddir/build/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/preprocessor.py", line 218, in parse
    token = self.lexer.token()
  File "/builddir/build/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/preprocessor.py", line 63, in token
    result = lex.Lexer.token(self)
  File "/builddir/build/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/lex.py", line 355, in token
    newtok = func(tok)
  File "/builddir/build/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/pplexer.py", line 262, in t_ANY_int
    g1 = str(long(g1, 8))
ValueError: invalid literal for int() with base 8: '08420217248550443400745280086994171'
make[6]: *** [Makefile:102: OBJ.x86_64-redhat-linux-gnu/date.py] Error 1

How to fix that?

Attachments (11)

build.log.xz (116.5 KB ) - added by brunofriedmann 7 years ago.
Full build log (in kvm on build.opensuse.org)
ctypes_print_debug.diff (1.3 KB ) - added by neteler 7 years ago.
Change to add some debug prints in lexer
ctypesgencore_preprocessor_fix.diff (557 bytes ) - added by neteler 7 years ago.
Attempt to fix preprocessor/lexer error when using GCC 7
gcc7_flags_standard.txt (12.2 KB ) - added by neteler 7 years ago.
Output of 'gcc -E -dM -x c /dev/null | sort'
gcc7_flags_with_c89.txt (12.1 KB ) - added by neteler 7 years ago.
Output of 'gcc -std=c89 -E -dM -x c /dev/null | sort'
gcc7_flags_with_c99.txt (12.2 KB ) - added by neteler 7 years ago.
Output of 'gcc -std=c99 -E -dM -x c /dev/null | sort'
floats1.diff (419 bytes ) - added by glynn 7 years ago.
Change integer literal regexp to avoid matching invalid octal literals
floats2.diff (890 bytes ) - added by glynn 7 years ago.
Expanded definition of floating-point suffix
gcc_fedora26_definitions.txt (12.2 KB ) - added by neteler 7 years ago.
gcc7.1.1 definitions on Fedora26
gcc_opensuse_definitions.txt (12.2 KB ) - added by neteler 7 years ago.
gcc7.1.1 definitions on openSUSE Tumbleweed
macosx_error.txt (20.8 KB ) - added by annakrat 7 years ago.
build error on Mac OSX

Download all attachments as: .zip

Change History (60)

comment:1 by martinl, 7 years ago

Milestone: 7.2.17.2.2

comment:2 by brunofriedmann, 7 years ago

We (openSUSE) also see the same error.

[  115s] make[6]: Entering directory '/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes'
[  115s] test -d OBJ.x86_64-pc-linux-gnu || mkdir -p OBJ.x86_64-pc-linux-gnu
[  115s] GISRC=/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/demolocation/.grassrc72 GISBASE=/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu PATH="/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/bin:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/bin:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/scripts:$PATH" PYTHONPATH="/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/gui/wxpython:$PYTHONPATH" LD_LIBRARY_PATH="/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/bin:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/bin:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/scripts:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/lib:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/lib:" LC_ALL=C ./ctypesgen.py --cpp "gcc -E       -I/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include -I/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG" -lgrass_datetime.7.2.1   /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include/grass/datetime.h /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include/grass/defs/datetime.h -o OBJ.x86_64-pc-linux-gnu/date.py
[  115s] Status: Preprocessing /tmp/tmpSIivMQ.h
[  115s] Status: gcc -E       -I/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include -I/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG -U __GNUC__ -dD "-Dinline=" "-D__inline__=" "-D__extension__=" "-D_Bool=uint8_t" "-D__const=const" "-D__asm__(x)=" "-D__asm(x)=" "-DCTYPESGEN=1" /tmp/tmpSIivMQ.h
[  115s] Traceback (most recent call last):
[  115s]   File "./ctypesgen.py", line 139, in <module>
[  115s]     descriptions = ctypesgencore.parser.parse(options.headers, options)
[  115s]   File "/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/__init__.py", line 22, in parse
[  115s]     parser.parse()
[  115s]   File "/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/datacollectingparser.py", line 74, in parse
[  115s]     ctypesparser.CtypesParser.parse(self, fname, None)
[  115s]   File "/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/cparser.py", line 120, in parse
[  115s]     self.preprocessor_parser.parse(filename)
[  115s]   File "/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/preprocessor.py", line 218, in parse
[  115s]     token = self.lexer.token()
[  115s]   File "/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/preprocessor.py", line 63, in token
[  115s]     result = lex.Lexer.token(self)
[  115s]   File "/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/lex.py", line 355, in token
[  115s]     newtok = func(tok)
[  115s]   File "/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/pplexer.py", line 262, in t_ANY_int
[  115s]     g1 = str(long(g1, 8))
[  115s] ValueError: invalid literal for int() with base 8: '08420217248550443400745280086994171'
[  115s] make[6]: *** [Makefile:102: OBJ.x86_64-pc-linux-gnu/date.py] Error 1
[  115s] make[6]: Leaving directory '/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes'
[  115s] make[5]: *** [Makefile:81: default] Error 2

in reply to:  2 comment:3 by neteler, 7 years ago

Replying to brunofriedmann:

We (openSUSE) also see the same error.

[  115s] make[6]: Entering directory '/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes'
[  115s] test -d OBJ.x86_64-pc-linux-gnu || mkdir -p OBJ.x86_64-pc-linux-gnu
[  115s] GISRC=/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/demolocation/.grassrc72 GISBASE=/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu PATH="/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/bin:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/bin:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/scripts:$PATH" PYTHONPATH="/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/gui/wxpython:$PYTHONPATH" LD_LIBRARY_PATH="/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/bin:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/bin:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/scripts:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/lib:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/lib:" LC_ALL=C ./ctypesgen.py --cpp "gcc -E       -I/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include -I/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG" -lgrass_datetime.7.2.1   /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include/grass/datetime.h /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include/grass/defs/datetime.h -o OBJ.x86_64-pc-linux-gnu/date.py
[  115s] Status: Preprocessing /tmp/tmpSIivMQ.h

Can you please attach this /tmp/tmpXXXX.h file for inspection?

comment:4 by brunofriedmann, 7 years ago

The file contain thoses lines

#include "/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include/grass/datetime.h"
#include "/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include/grass/defs/datetime.h"

Which are both present in the build env at that location

by brunofriedmann, 7 years ago

Attachment: build.log.xz added

Full build log (in kvm on build.opensuse.org)

comment:5 by neteler, 7 years ago

I installed Fedora26 (alpha) in docker in order to reproduce the problem locally:

[root@f1e5cbaaed18 ctypes]# make
make /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib
make[1]: Entering directory '/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes'
make[1]: '/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib' is up to date.
make[1]: Leaving directory '/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes'
make OBJ.x86_64-pc-linux-gnu/date.py OBJ.x86_64-pc-linux-gnu/gis.py OBJ.x86_64-pc-linux-gnu/raster.py OBJ.x86_64-pc-linux-gnu/gmath.py OBJ.x86_64-pc-linux-gnu/proj.py OBJ.x86_64-pc-linux-gnu/imagery.py OBJ.x86_64-pc-linux-gnu/vector.py OBJ.x86_64-pc-linux-gnu/rtree.py OBJ.x86_64-pc-linux-gnu/display.py OBJ.x86_64-pc-linux-gnu/stats.py OBJ.x86_64-pc-linux-gnu/dbmi.py OBJ.x86_64-pc-linux-gnu/raster3d.py OBJ.x86_64-pc-linux-gnu/arraystats.py OBJ.x86_64-pc-linux-gnu/cluster.py OBJ.x86_64-pc-linux-gnu/vedit.py OBJ.x86_64-pc-linux-gnu/segment.py OBJ.x86_64-pc-linux-gnu/rowio.py OBJ.x86_64-pc-linux-gnu/temporal.py OBJ.x86_64-pc-linux-gnu/ogsf.py OBJ.x86_64-pc-linux-gnu/nviz.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/date.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/gis.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/raster.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/gmath.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/proj.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/imagery.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/vector.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/rtree.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/display.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/stats.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/dbmi.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/raster3d.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/arraystats.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/cluster.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/vedit.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/segment.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/rowio.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/temporal.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/ogsf.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/nviz.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/__init__.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/ctypes_preamble.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/ctypes_loader.py /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/date.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/gis.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/raster.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/gmath.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/proj.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/imagery.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/vector.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/rtree.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/display.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/stats.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/dbmi.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/raster3d.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/arraystats.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/cluster.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/vedit.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/segment.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/rowio.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/temporal.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/ogsf.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/nviz.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/__init__.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/ctypes_preamble.pyc /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/ctypes_loader.pyc
make[1]: Entering directory '/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes'
GISRC=/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/demolocation/.grassrc72 GISBASE=/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu PATH="/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/bin:/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/bin:/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/scripts:$PATH" PYTHONPATH="/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/etc/python:/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/gui/wxpython:$PYTHONPATH" LD_LIBRARY_PATH="/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/bin:/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/bin:/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/scripts:/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/lib:/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/lib:" LC_ALL=C LANG=C LANGUAGE=C ./ctypesgen.py --cpp "gcc -E       -I/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/include -I/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG" -lgrass_datetime.7.2.2svn   /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/include/grass/datetime.h /root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/include/grass/defs/datetime.h -o OBJ.x86_64-pc-linux-gnu/date.py
Status: Preprocessing /tmp/tmpS5OAMT.h
Status: gcc -E       -I/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/include -I/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG -U __GNUC__ -dD "-Dinline=" "-D__inline__=" "-D__extension__=" "-D_Bool=uint8_t" "-D__const=const" "-D__asm__(x)=" "-D__asm(x)=" "-DCTYPESGEN=1" /tmp/tmpS5OAMT.h
Traceback (most recent call last):
  File "./ctypesgen.py", line 139, in <module>
    descriptions = ctypesgencore.parser.parse(options.headers, options)
  File "/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes/ctypesgencore/parser/__init__.py", line 22, in parse
    parser.parse()
  File "/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes/ctypesgencore/parser/datacollectingparser.py", line 74, in parse
    ctypesparser.CtypesParser.parse(self, fname, None)
  File "/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes/ctypesgencore/parser/cparser.py", line 120, in parse
    self.preprocessor_parser.parse(filename)
  File "/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes/ctypesgencore/parser/preprocessor.py", line 218, in parse
    token = self.lexer.token()
  File "/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes/ctypesgencore/parser/preprocessor.py", line 63, in token
    result = lex.Lexer.token(self)
  File "/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes/ctypesgencore/parser/lex.py", line 355, in token
    newtok = func(tok)
  File "/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes/ctypesgencore/parser/pplexer.py", line 262, in t_ANY_int
    g1 = str(long(g1, 8))
ValueError: invalid literal for int() with base 8: '08420217248550443400745280086994171'
make[1]: *** [Makefile:102: OBJ.x86_64-pc-linux-gnu/date.py] Error 1
make[1]: Leaving directory '/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes'
make: *** [Makefile:81: default] Error 2


[root@f1e5cbaaed18 ctypes]# cat /tmp/tmpS5OAMT.h
#include "/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/include/grass/datetime.h"
#include "/root/grass-7.2.svn_src_snapshot_2017_06_03/dist.x86_64-pc-linux-gnu/include/grass/defs/datetime.h"

I wonder if the "-D_Bool=uint8_t" above matters here?

comment:6 by brunofriedmann, 7 years ago

I've tried a stupid patch and remove -D_Bool=uint8_t and got never error like so the culprit is somewhere else.

[   65s] ValueError: invalid literal for int() with base 8: '08420217248550443400745280086994171'

in lib/python/ctypes/ctypesgencore/parser/preprocessor.py it seems its C99 syntax, and I wonder if on newer plateform we don't have C11

Last edited 7 years ago by brunofriedmann (previous) (diff)

by neteler, 7 years ago

Attachment: ctypes_print_debug.diff added

Change to add some debug prints in lexer

comment:7 by neteler, 7 years ago

The relevant difference is that Fedora 26 comes with GCC 7 while F25 comes with GCC 6. Probably something changed in the preprocessor?

Adding some debug output shows where in the lexer the problem occurs:

[root@f1e5cbaaed18 ctypes]# make | grep 084202
#define __LDBL_EPSILON__ 1.08420217248550443400745280086994171e-19L
#define __FLT64X_EPSILON__ 1.08420217248550443400745280086994171e-19F64x
Traceback (most recent call last):
  File "./ctypesgen.py", line 139, in <module>
    descriptions = ctypesgencore.parser.parse(options.headers, options)
  File "/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes/ctypesgencore/parser/__init__.py", line 22, in parse
    parser.parse()
  File "/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes/ctypesgencore/parser/datacollectingparser.py", line 74, in parse
    ctypesparser.CtypesParser.parse(self, fname, None)
  File "/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes/ctypesgencore/parser/cparser.py", line 120, in parse
    self.preprocessor_parser.parse(filename)
  File "/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes/ctypesgencore/parser/preprocessor.py", line 219, in parse
    token = self.lexer.token()
  File "/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes/ctypesgencore/parser/preprocessor.py", line 63, in token
    result = lex.Lexer.token(self)
  File "/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes/ctypesgencore/parser/lex.py", line 357, in token
    newtok = func(tok)
  File "/root/grass-7.2.svn_src_snapshot_2017_06_03/lib/python/ctypes/ctypesgencore/parser/pplexer.py", line 262, in t_ANY_int
    g1 = str(long(g1, 8))
ValueError: invalid literal for int() with base 8: '08420217248550443400745280086994171'
make[1]: *** [Makefile:102: OBJ.x86_64-pc-linux-gnu/proj.py] Error 1
make: *** [Makefile:81: default] Error 2

As seen above the 'LDBL_EPSILON' definition is not parsed properly in GCC 7 (or in our ctypesgencore software which seems to come from here: https://github.com/davidjamesca/ctypesgen).

Perhaps we need a sync with the official code repo?

Interim work-around (also attached):

--- lib/python/ctypes/ctypesgencore/parser/preprocessor.py	(Revision 71170)
+++ lib/python/ctypes/ctypesgencore/parser/preprocessor.py	(Arbeitskopie)
@@ -150,7 +150,6 @@
         if sys.platform == 'darwin':
             cmd += " -U __BLOCKS__"
         cmd += " -U __GNUC__"
-        cmd += " -dD"
         for path in self.options.include_search_paths:
             cmd += " -I%s" % path
         for define in self.defines:

The question is: does ctypes still work? How to run the needed testsuite parts?

by neteler, 7 years ago

Attempt to fix preprocessor/lexer error when using GCC 7

in reply to:  7 comment:8 by neteler, 7 years ago

Replying to neteler:

our ctypesgencore software which seems to come from here: https://github.com/davidjamesca/ctypesgen).

BTW: I'd suggest to submit r68348 as a patch to the upstream repo.

Last edited 7 years ago by neteler (previous) (diff)

comment:9 by brunofriedmann, 7 years ago

With your patch the build was also working for openSUSE Tumbleweed (gcc7)

comment:10 by brunofriedmann, 7 years ago

Also +1 for the merge of r68348 in main.

in reply to:  10 comment:11 by neteler, 7 years ago

Replying to brunofriedmann:

Also +1 for the merge of r68348 in main.

s/main/upstream/

comment:12 by neteler, 7 years ago

Version: unspecifiedsvn-releasebranch72

In r71180 I have now removed the -dD flag in ctypesgencore/preprocessor because it is failing with gcc7 and apparently not causing any harm with gcc6.

To be backported once sufficiently tested.

comment:13 by brunofriedmann, 7 years ago

I would have reference this bug number in the comment, to ease reading source code and find the root cause of gcc7 failing. We still don't know if its a gcc7 regression or not right ?

in reply to:  13 comment:14 by neteler, 7 years ago

Replying to brunofriedmann:

I would have reference this bug number in the comment, to ease reading source code and find the root cause of gcc7 failing.

Fully agreed, done in r71182

We still don't know if its a gcc7 regression or not right ?

Right - the question is: how to find out?

in reply to:  12 ; comment:15 by annakrat, 7 years ago

Replying to neteler:

In r71180 I have now removed the -dD flag in ctypesgencore/preprocessor because it is failing with gcc7 and apparently not causing any harm with gcc6.

To be backported once sufficiently tested.

Seems to break parts of WxGUI using ctypes - vector digitizer and 3d view - on Ubuntu and MS Windows.

in reply to:  15 comment:16 by neteler, 7 years ago

Replying to annakrat:

Replying to neteler:

In r71180 I have now removed the -dD flag in ctypesgencore/preprocessor because it is failing with gcc7 and apparently not causing any harm with gcc6.

To be backported once sufficiently tested.

Seems to break parts of WxGUI using ctypes - vector digitizer and 3d view - on Ubuntu and MS Windows.

Would it be possible to isolate the breakage with a small code example? Otherwise debugging the ctypes lexer and bindings is quite hard.

in reply to:  7 ; comment:17 by glynn, 7 years ago

Replying to neteler:

Adding some debug output shows where in the lexer the problem occurs:

[root@f1e5cbaaed18 ctypes]# make | grep 084202
#define __LDBL_EPSILON__ 1.08420217248550443400745280086994171e-19L
#define __FLT64X_EPSILON__ 1.08420217248550443400745280086994171e-19F64x

    g1 = str(long(g1, 8))
ValueError: invalid literal for int() with base 8: '08420217248550443400745280086994171'

As seen above the __LDBL_EPSILON__ definition is not parsed properly

It's more likely to be an issue with __FLT64X_EPSILON__, as that's a recent addition. __LDBL_EPSILON__ has been defined since forever, and hasn't caused problems before.

Looking through pplexer.py shows that it specifically tries to handle literals with an "L" or "l" suffix, but there's nothing there for an "F64x" suffix. So the value won't match the regex for a float literal, which probably results in it matching "1." as a float literal then trying to match the remainder. The part up to the exponent will match an int literal, but the leading zero causes it to be parsed as octal, which then fails due to the presence of digits 8 and 9.

You can dump out gcc's built-in macro definitions with

gcc -E -dM -x c /dev/null | sort

Does adding e.g. -std=c89 or -std=c99 eliminate the F64x literals?

FWIW, F64x appears to derive from ISO/IEC TS 18661-3:2015.

in reply to:  12 comment:18 by neteler, 7 years ago

Replying to neteler:

In r71180 I have now removed the -dD flag in ctypesgencore/preprocessor because it is failing with gcc7 and apparently not causing any harm with gcc6.

Reverted along with r71182 in r71187 since not solving the problem (but breaking v.digit)

in reply to:  17 comment:19 by neteler, 7 years ago

Replying to glynn:

Looking through pplexer.py shows that it specifically tries to handle literals with an "L" or "l" suffix, but there's nothing there for an "F64x" suffix.

We were wondering here about the L suffix(es) but didn't realize that a "F64x" suffix could be an issue.

...

You can dump out gcc's built-in macro definitions with

gcc -E -dM -x c /dev/null | sort

Does adding e.g. -std=c89 or -std=c99 eliminate the F64x literals?

Apparently not, here the diffences (full outputs attached to the ticket):

# see attachments:
[root@f1e5cbaaed18 /]# gcc -E -dM -x c /dev/null | sort > gcc7_flags_standard.txt
[root@f1e5cbaaed18 /]# gcc -std=c89 -E -dM -x c /dev/null | sort > gcc7_flags_with_c89.txt
[root@f1e5cbaaed18 /]# gcc -std=c99 -E -dM -x c /dev/null | sort > gcc7_flags_with_c99.txt

# differences:
[root@f1e5cbaaed18 /]# diff -u gcc7_flags_standard.txt gcc7_flags_with_c89.txt
--- gcc7_flags_standard.txt	2017-06-16 05:35:10.481505202 +0000
+++ gcc7_flags_with_c89.txt	2017-06-16 05:35:37.813790118 +0000
@@ -161,10 +161,10 @@
 #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_8 1
 #define __GCC_IEC_559 2
 #define __GCC_IEC_559_COMPLEX 2
+#define __GNUC_GNU_INLINE__ 1
 #define __GNUC_MINOR__ 1
 #define __GNUC_PATCHLEVEL__ 1
 #define __GNUC_RH_RELEASE__ 2
-#define __GNUC_STDC_INLINE__ 1
 #define __GNUC__ 7
 #define __GXX_ABI_VERSION 1011
 #define __INT16_C(c) c
@@ -278,10 +278,8 @@
 #define __STDC_IEC_559__ 1
 #define __STDC_ISO_10646__ 201605L
 #define __STDC_NO_THREADS__ 1
-#define __STDC_UTF_16__ 1
-#define __STDC_UTF_32__ 1
-#define __STDC_VERSION__ 201112L
 #define __STDC__ 1
+#define __STRICT_ANSI__ 1
 #define __UINT16_C(c) c
 #define __UINT16_MAX__ 0xffff
 #define __UINT16_TYPE__ short unsigned int
@@ -339,5 +337,3 @@
 #define __unix__ 1
 #define __x86_64 1
 #define __x86_64__ 1
-#define linux 1
-#define unix 1

[root@f1e5cbaaed18 /]# diff -u gcc7_flags_standard.txt gcc7_flags_with_c99.txt
--- gcc7_flags_standard.txt	2017-06-16 05:35:10.481505202 +0000
+++ gcc7_flags_with_c99.txt	2017-06-16 05:35:49.742914468 +0000
@@ -278,10 +278,9 @@
 #define __STDC_IEC_559__ 1
 #define __STDC_ISO_10646__ 201605L
 #define __STDC_NO_THREADS__ 1
-#define __STDC_UTF_16__ 1
-#define __STDC_UTF_32__ 1
-#define __STDC_VERSION__ 201112L
+#define __STDC_VERSION__ 199901L
 #define __STDC__ 1
+#define __STRICT_ANSI__ 1
 #define __UINT16_C(c) c
 #define __UINT16_MAX__ 0xffff
 #define __UINT16_TYPE__ short unsigned int
@@ -339,5 +338,3 @@
 #define __unix__ 1
 #define __x86_64 1
 #define __x86_64__ 1
-#define linux 1
-#define unix 1

[root@f1e5cbaaed18 /]# grep F64x gcc7_flags_*
gcc7_flags_standard.txt:#define __FLT64X_DENORM_MIN__ 3.64519953188247460252840593361941982e-4951F64x
gcc7_flags_standard.txt:#define __FLT64X_EPSILON__ 1.08420217248550443400745280086994171e-19F64x
gcc7_flags_standard.txt:#define __FLT64X_MAX__ 1.18973149535723176502126385303097021e+4932F64x
gcc7_flags_standard.txt:#define __FLT64X_MIN__ 3.36210314311209350626267781732175260e-4932F64x
gcc7_flags_with_c89.txt:#define __FLT64X_DENORM_MIN__ 3.64519953188247460252840593361941982e-4951F64x
gcc7_flags_with_c89.txt:#define __FLT64X_EPSILON__ 1.08420217248550443400745280086994171e-19F64x
gcc7_flags_with_c89.txt:#define __FLT64X_MAX__ 1.18973149535723176502126385303097021e+4932F64x
gcc7_flags_with_c89.txt:#define __FLT64X_MIN__ 3.36210314311209350626267781732175260e-4932F64x
gcc7_flags_with_c99.txt:#define __FLT64X_DENORM_MIN__ 3.64519953188247460252840593361941982e-4951F64x
gcc7_flags_with_c99.txt:#define __FLT64X_EPSILON__ 1.08420217248550443400745280086994171e-19F64x
gcc7_flags_with_c99.txt:#define __FLT64X_MAX__ 1.18973149535723176502126385303097021e+4932F64x
gcc7_flags_with_c99.txt:#define __FLT64X_MIN__ 3.36210314311209350626267781732175260e-4932F64x

FWIW, F64x appears to derive from ISO/IEC TS 18661-3:2015.

by neteler, 7 years ago

Attachment: gcc7_flags_standard.txt added

Output of 'gcc -E -dM -x c /dev/null | sort'

by neteler, 7 years ago

Attachment: gcc7_flags_with_c89.txt added

Output of 'gcc -std=c89 -E -dM -x c /dev/null | sort'

by neteler, 7 years ago

Attachment: gcc7_flags_with_c99.txt added

Output of 'gcc -std=c99 -E -dM -x c /dev/null | sort'

comment:20 by glynn, 7 years ago

Replying to glynn:

Incidentally, it doesn't correctly parse the decimal floating-point (dec32/64/128) literals produced by earlier versions of gcc (DF, DD, DL suffixes) either. Again, it ends up treating them as a sequence of integer/float/identifier tokens. But as these only occur in the definitions of macros which are never used, it doesn't actually matter.

Essentially, it's just bad luck that the value of FLT64X_EPSILON happens to have a zero immediately after the decimal point, resulting in an attempt to parse as octal.

There seem to be a couple of options for avoiding this. The regex for integer literals is (pplexer.py:243):

INT_LITERAL = sub(r"(?P<p1>(?:0x{H}+)|(?:{D}+))(?P<suf>{IS})")

where sub() performs the substitutions found at pplexer.py:50:

subs = {
    'D': '[0-9]',
    'L': '[a-zA-Z_]',
    'H': '[a-fA-F0-9]',
    'E': '[Ee][+-]?\s*{D}+',
    'FS': '[FflL]',
    'IS': '[uUlL]*',
}

Changing this to

INT_LITERAL = sub(r"(?P<p1>(?:0x{H}+)|(?:0[0-7]+)|(?:[1-9]{D}+))(?P<suf>{IS})")

and deleting lextab.{py,pyc} (which will be regenerated) seems to avoid the issue. Such literals are still parsed incorrectly, but they don't throw exceptions.

Another option is to change the regex used for FS (float suffix) from [FflL] to something which encompasses the TS 18661 types, e.g. ([FfLl]|d[dfl]|D[DFL]|[fFdD][0-9]+x?). But that also requires the suffix-handling logic in t_ANY_float to change accordingly.

by glynn, 7 years ago

Attachment: floats1.diff added

Change integer literal regexp to avoid matching invalid octal literals

by glynn, 7 years ago

Attachment: floats2.diff added

Expanded definition of floating-point suffix

comment:21 by brunofriedmann, 7 years ago

Excuse me, but how should apply the last 2 patches ? I've tried a build with them apply and the build failed at the same place as reported. Shall I remove the lextab.py ?

in reply to:  21 ; comment:22 by neteler, 7 years ago

Replying to brunofriedmann:

Excuse me, but how should apply the last 2 patches ? I've tried a build with them apply and the build failed at the same place as reported. Shall I remove the lextab.py ?

Yes. The procedure:

[root@f1e5cbaaed18 trunk ]# cd lib/python/ctypes
[root@f1e5cbaaed18 ctypes]# patch -p0 < /tmp/floats1.diff 
patching file ctypesgencore/parser/pplexer.py
[root@f1e5cbaaed18 ctypes]# patch -p0 < /tmp/floats2.diff 
patching file ctypesgencore/parser/pplexer.py
[root@f1e5cbaaed18 ctypes]# rm -f ctypesgencore/parser/lextab.py*
[root@f1e5cbaaed18 ctypes]# make

Here on Fedora 26/gcc7 it compiles again. Thanks Glynn!

Testing on Windows will probably be easiest after submission to trunk.

in reply to:  22 comment:23 by hellik, 7 years ago

Replying to neteler:

Testing on Windows will probably be easiest after submission to trunk.

Yes

comment:24 by neteler, 7 years ago

While we are at it: I found three parsing errors with the patched ctypes version (F25, gcc6):


GISRC=/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/demolocation/.grassrc72 GISBASE=/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu PATH="/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/bin:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/bin:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/scripts:$PATH" PYTHONPATH="/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/etc/python:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/gui/wxpython:$PYTHONPATH" LD_LIBRARY_PATH="/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/bin:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/bin:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/scripts:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/lib:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/lib:" LC_ALL=C ./ctypesgen.py --cpp "gcc -E       -I/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include -I/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG" -lgrass_gis.7.2.2svn   /home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include/grass/gis.h /home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include/grass/defs/gis.h -o OBJ.x86_64-pc-linux-gnu/gis.py
Status: Preprocessing /home/mundialis/tmp/tmpJOuynC.h
Status: gcc -E       -I/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include -I/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG -U __GNUC__ -dD "-Dinline=" "-D__inline__=" "-D__extension__=" "-D_Bool=uint8_t" "-D__const=const" "-D__asm__(x)=" "-D__asm(x)=" "-DCTYPESGEN=1" /home/mundialis/tmp/tmpJOuynC.h
Status: Parsing /home/mundialis/tmp/tmpJOuynC.h
Status: Processing description list.
Warning: Member "from" of Struct "DateTime" has been renamed to "_from" because it has the same name as a Python keyword.
Warning: Member "def" of Struct "Option" has been renamed to "_def" because it has the same name as a Python keyword.
Warning: Could not parse macro "#define serialize_int32_le(buf,x) do { ( buf ) [ i0 ] = ( ( x ) >> i0 ) & i255 ; ( buf ) [ i1 ] = ( ( x ) >> i8 ) & i255 ; ( buf ) [ i2 ] = ( ( x ) >> i16 ) & i255 ; ( buf ) [ i3 ] = ( ( x ) >> i24 ) & i255 ; } while ( i0 )"
Warning: Could not parse macro "#define serialize_int32_be(buf,x) do { ( buf ) [ i0 ] = ( ( x ) >> i24 ) & i255 ; ( buf ) [ i1 ] = ( ( x ) >> i16 ) & i255 ; ( buf ) [ i2 ] = ( ( x ) >> i8 ) & i255 ; ( buf ) [ i3 ] = ( ( x ) >> i0 ) & i255 ; } while ( i0 )"
Status: Writing to OBJ.x86_64-pc-linux-gnu/gis.py.
Status: Wrapping complete.

[...]


GISRC=/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/demolocation/.grassrc72 GISBASE=/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu PATH="/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/bin:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/bin:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/scripts:$PATH" PYTHONPATH="/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/etc/python:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/gui/wxpython:$PYTHONPATH" LD_LIBRARY_PATH="/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/bin:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/bin:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/scripts:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/lib:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/lib:" LC_ALL=C ./ctypesgen.py --cpp "gcc -E       -I/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include -I/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG" -lgrass_gmath.7.2.2svn   /home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include/grass/gmath.h /home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include/grass/defs/gmath.h -o OBJ.x86_64-pc-linux-gnu/gmath.py
Status: Preprocessing /home/mundialis/tmp/tmp956DqU.h
Status: gcc -E       -I/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include -I/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG -U __GNUC__ -dD "-Dinline=" "-D__inline__=" "-D__extension__=" "-D_Bool=uint8_t" "-D__const=const" "-D__asm__(x)=" "-D__asm(x)=" "-DCTYPESGEN=1" /home/mundialis/tmp/tmp956DqU.h
Status: Parsing /home/mundialis/tmp/tmp956DqU.h
Error: /usr/lib/gcc/x86_64-redhat-linux/6.3.1/include/stddef.h:427: Syntax error at '__attribute__'
Error: /usr/lib/gcc/x86_64-redhat-linux/6.3.1/include/stddef.h:427: Syntax error at '('
Error: /usr/lib/gcc/x86_64-redhat-linux/6.3.1/include/stddef.h:428: Syntax error at '__attribute__'
Error: /usr/lib/gcc/x86_64-redhat-linux/6.3.1/include/stddef.h:428: Syntax error at '('
Status: Processing description list.
Status: Writing to OBJ.x86_64-pc-linux-gnu/gmath.py.

[...]

GISRC=/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/demolocation/.grassrc72 GISBASE=/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu PATH="/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/bin:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/bin:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/scripts:$PATH" PYTHONPATH="/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/etc/python:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/gui/wxpython:$PYTHONPATH" LD_LIBRARY_PATH="/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/bin:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/bin:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/scripts:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/lib:/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/lib:" LC_ALL=C ./ctypesgen.py --cpp "gcc -E       -I/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include -I/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG" -lgrass_temporal.7.2.2svn   /home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include/grass/temporal.h -o OBJ.x86_64-pc-linux-gnu/temporal.py
Status: Preprocessing /home/mundialis/tmp/tmppn67pF.h
Status: gcc -E       -I/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include -I/home/mundialis/software/grass72_svn/dist.x86_64-pc-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG -U __GNUC__ -dD "-Dinline=" "-D__inline__=" "-D__extension__=" "-D_Bool=uint8_t" "-D__const=const" "-D__asm__(x)=" "-D__asm(x)=" "-DCTYPESGEN=1" /home/mundialis/tmp/tmppn67pF.h
Status: Parsing /home/mundialis/tmp/tmppn67pF.h
Status: Processing description list.
Warning: Member "from" of Struct "DateTime" has been renamed to "_from" because it has the same name as a Python keyword.
Warning: Member "in" of Struct "_tgisDataset" has been renamed to "_in" because it has the same name as a Python keyword.
Warning: Could not parse macro "#define TGIS STR3DS i5"
Status: Writing to OBJ.x86_64-pc-linux-gnu/temporal.py.
Status: Wrapping complete.

I cannot judge how problematic they are...

in reply to:  22 ; comment:25 by brunofriedmann, 7 years ago

Replying to neteler:

Replying to brunofriedmann:

Excuse me, but how should apply the last 2 patches ? I've tried a build with them apply and the build failed at the same place as reported. Shall I remove the lextab.py ?

Yes. The procedure:

[root@f1e5cbaaed18 trunk ]# cd lib/python/ctypes
[root@f1e5cbaaed18 ctypes]# patch -p0 < /tmp/floats1.diff 
patching file ctypesgencore/parser/pplexer.py
[root@f1e5cbaaed18 ctypes]# patch -p0 < /tmp/floats2.diff 
patching file ctypesgencore/parser/pplexer.py
[root@f1e5cbaaed18 ctypes]# rm -f ctypesgencore/parser/lextab.py*
[root@f1e5cbaaed18 ctypes]# make

Here on Fedora 26/gcc7 it compiles again. Thanks Glynn!

Testing on Windows will probably be easiest after submission to trunk.

Ok strangely here with openSUSE Tumbleweed/gcc7 it still fail. As reported. I will dig more deeply Wednesday the reason why.

in reply to:  25 comment:26 by neteler, 7 years ago

Replying to brunofriedmann:

Ok strangely here with openSUSE Tumbleweed/gcc7 it still fail. As reported. I will dig more deeply Wednesday the reason why.

Is there a openSUSE Tumbleweed docker image available to try?

comment:27 by brunofriedmann, 7 years ago

It seems so :-) https://github.com/openSUSE/docker-containers-build/blob/eca61af4744f0170ae093efea658dcca3cb46538/docker/Dockerfile

But beware you will need to add Application:Geo repository to have the Geo stack. http://download.opensuse.org/repositories/Application:/Geo/openSUSE_Tumbleweed/

Actually I'm trying to fix the build with the newer patches, in my home repository located here https://build.opensuse.org/package/show/home:bruno_friedmann:branches:Application:Geo/grass

Actually I've some changes locally that are not yet pushed, since it doesn't build.

comment:28 by brunofriedmann, 7 years ago

Ok entering the chroot of build and running make in lib/python/ctypes give me the following result

abuild@qt-kt.labaroche.ioda.net:/home/abuild> ll
total 24
drwxr-xr-x 3 abuild abuild 4096 Jun 20 11:25 .
drwxr-xr-x 3 root   root   4096 Jun 20 11:25 ..
-rw-r--r-- 1 root   root   4353 Jun 20 11:25 .rpmmacros
-rw-r--r-- 1 root   root   2672 Jun 20 11:25 .rpmrc
drwxr-xr-x 9 abuild abuild 4096 Jun 20 11:25 rpmbuild
abuild@qt-kt.labaroche.ioda.net:/home/abuild> cd rpmbuild/
BUILD/     BUILDROOT/ OTHER/     RPMS/      SOURCES/   SPECS/     SRPMS/     
abuild@qt-kt.labaroche.ioda.net:/home/abuild> cd rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes/
abuild@qt-kt.labaroche.ioda.net:/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes> ll
total 52
drwxr-xr-x  4 abuild abuild 4096 Jun 20 11:25 .
drwxr-xr-x 11 abuild abuild 4096 Nov 29  2014 ..
-rw-r--r--  1 abuild abuild 3390 Aug 27  2016 Makefile
drwxr-xr-x  2 abuild abuild 4096 Jun 20 11:25 OBJ.x86_64-pc-linux-gnu
-rw-r--r--  1 abuild abuild  243 May  2  2016 __init__.py
-rwxr-xr-x  1 abuild abuild 7208 May  2  2016 ctypesgen.py
drwxr-xr-x  5 abuild abuild 4096 Jun 20 11:25 ctypesgencore
-rw-r--r--  1 abuild abuild  225 May  2  2016 fix.sed
-rw-r--r--  1 abuild abuild 8518 May  2  2016 loader.py
-rw-r--r--  1 abuild abuild 2287 May  2  2016 preamble.py
abuild@qt-kt.labaroche.ioda.net:/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes> make
make /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib
make[1]: Entering directory '/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes'
make[1]: '/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib' is up to date.
make[1]: Leaving directory '/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes'
make OBJ.x86_64-pc-linux-gnu/date.py OBJ.x86_64-pc-linux-gnu/gis.py OBJ.x86_64-pc-linux-gnu/raster.py OBJ.x86_64-pc-linux-gnu/gmath.py OBJ.x86_64-pc-linux-gnu/proj.py OBJ.x86_64-pc-linux-gnu/imagery.py OBJ.x86_64-pc-linux-gnu/vector.py OBJ.x86_64-pc-linux-gnu/rtree.py OBJ.x86_64-pc-linux-gnu/display.py OBJ.x86_64-pc-linux-gnu/stats.py OBJ.x86_64-pc-linux-gnu/dbmi.py OBJ.x86_64-pc-linux-gnu/raster3d.py OBJ.x86_64-pc-linux-gnu/arraystats.py OBJ.x86_64-pc-linux-gnu/cluster.py OBJ.x86_64-pc-linux-gnu/vedit.py OBJ.x86_64-pc-linux-gnu/segment.py OBJ.x86_64-pc-linux-gnu/rowio.py OBJ.x86_64-pc-linux-gnu/temporal.py OBJ.x86_64-pc-linux-gnu/ogsf.py OBJ.x86_64-pc-linux-gnu/nviz.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/date.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/gis.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/raster.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/gmath.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/proj.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/imagery.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/vector.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/rtree.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/display.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/stats.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/dbmi.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/raster3d.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/arraystats.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/cluster.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/vedit.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/segment.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/rowio.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/temporal.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/ogsf.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/nviz.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/__init__.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/ctypes_preamble.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/ctypes_loader.py /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/date.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/gis.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/raster.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/gmath.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/proj.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/imagery.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/vector.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/rtree.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/display.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/stats.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/dbmi.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/raster3d.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/arraystats.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/cluster.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/vedit.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/segment.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/rowio.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/temporal.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/ogsf.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/nviz.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/__init__.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/ctypes_preamble.pyc /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python/grass/lib/ctypes_loader.pyc
make[1]: Entering directory '/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes'
make[1]: 'OBJ.x86_64-pc-linux-gnu/date.py' is up to date.
make[1]: 'OBJ.x86_64-pc-linux-gnu/gis.py' is up to date.
make[1]: 'OBJ.x86_64-pc-linux-gnu/raster.py' is up to date.
make[1]: 'OBJ.x86_64-pc-linux-gnu/gmath.py' is up to date.
GISRC=/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/demolocation/.grassrc72 GISBASE=/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu PATH="/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/bin:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/bin:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/scripts:$PATH" PYTHONPATH="/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/etc/python:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/gui/wxpython:$PYTHONPATH" LD_LIBRARY_PATH="/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/bin:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/bin:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/scripts:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/lib:/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/lib:" LC_ALL=C ./ctypesgen.py --cpp "gcc -E       -I/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include -I/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG" -lgrass_gproj.7.2.1 -I/usr/include/gdal  /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include/grass/gprojects.h /home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include/grass/defs/gprojects.h -o OBJ.x86_64-pc-linux-gnu/proj.py
Status: Preprocessing /tmp/tmpU_Xy2Y.h
Status: gcc -E       -I/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include -I/home/abuild/rpmbuild/BUILD/grass-7.2.1/dist.x86_64-pc-linux-gnu/include -D__GLIBC_HAVE_LONG_LONG -U __GNUC__ -dD -I/usr/include/gdal "-Dinline=" "-D__inline__=" "-D__extension__=" "-D_Bool=uint8_t" "-D__const=const" "-D__asm__(x)=" "-D__asm(x)=" "-DCTYPESGEN=1" /tmp/tmpU_Xy2Y.h
Traceback (most recent call last):
  File "./ctypesgen.py", line 139, in <module>
    descriptions = ctypesgencore.parser.parse(options.headers, options)
  File "/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/__init__.py", line 22, in parse
    parser.parse()
  File "/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/datacollectingparser.py", line 74, in parse
    ctypesparser.CtypesParser.parse(self, fname, None)
  File "/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/cparser.py", line 120, in parse
    self.preprocessor_parser.parse(filename)
  File "/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/preprocessor.py", line 218, in parse
    token = self.lexer.token()
  File "/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/preprocessor.py", line 63, in token
    result = lex.Lexer.token(self)
  File "/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes/ctypesgencore/parser/lex.py", line 404, in token
    lexpos:])
ctypesgencore.parser.lex.LexError: Scanning error. Illegal character '0'
make[1]: *** [Makefile:102: OBJ.x86_64-pc-linux-gnu/proj.py] Error 1
make[1]: Leaving directory '/home/abuild/rpmbuild/BUILD/grass-7.2.1/lib/python/ctypes'
make: *** [Makefile:81: default] Error 2

comment:29 by neteler, 7 years ago

Just to be sure: Did you remove ctypesgencore/parser/lextab.py* prior to the recompilation?

in reply to:  29 comment:30 by brunofriedmann, 7 years ago

Replying to neteler:

Just to be sure: Did you remove ctypesgencore/parser/lextab.py* prior to the recompilation?

Yes I've a line like this find . -iname "lextab.py*" -ls -delete

and in the build log the job is done. in chroot the file is datetime of build

comment:31 by neteler, 7 years ago

In r71196 I have submitted Glynn's lexer fixes, please test.

in reply to:  25 comment:32 by neteler, 7 years ago

Replying to brunofriedmann:

Ok strangely here with openSUSE Tumbleweed/gcc7 it still fail. As reported.

Confirmed in my new openSUSE VM. Apparently the regex needs some more tweaking.

Here the FLT64 related definitions on openSUSE:

# openSUSE Tumbleweed
gcc -E -dM -x c /dev/null | sort | grep FLT64
#define __FLT64_DECIMAL_DIG__ 17
#define __FLT64_DENORM_MIN__ 4.94065645841246544176568792868221372e-324F64
#define __FLT64_DIG__ 15
#define __FLT64_EPSILON__ 2.22044604925031308084726333618164062e-16F64
#define __FLT64_HAS_DENORM__ 1
#define __FLT64_HAS_INFINITY__ 1
#define __FLT64_HAS_QUIET_NAN__ 1
#define __FLT64_MANT_DIG__ 53
#define __FLT64_MAX_10_EXP__ 308
#define __FLT64_MAX__ 1.79769313486231570814527423731704357e+308F64
#define __FLT64_MAX_EXP__ 1024
#define __FLT64_MIN_10_EXP__ (-307)
#define __FLT64_MIN__ 2.22507385850720138309023271733240406e-308F64
#define __FLT64_MIN_EXP__ (-1021)
#define __FLT64X_DECIMAL_DIG__ 21
#define __FLT64X_DENORM_MIN__ 3.64519953188247460252840593361941982e-4951F64x
#define __FLT64X_DIG__ 18
#define __FLT64X_EPSILON__ 1.08420217248550443400745280086994171e-19F64x
#define __FLT64X_HAS_DENORM__ 1
#define __FLT64X_HAS_INFINITY__ 1
#define __FLT64X_HAS_QUIET_NAN__ 1
#define __FLT64X_MANT_DIG__ 64
#define __FLT64X_MAX_10_EXP__ 4932
#define __FLT64X_MAX__ 1.18973149535723176502126385303097021e+4932F64x
#define __FLT64X_MAX_EXP__ 16384
#define __FLT64X_MIN_10_EXP__ (-4931)
#define __FLT64X_MIN__ 3.36210314311209350626267781732175260e-4932F64x
#define __FLT64X_MIN_EXP__ (-16381)

gcc -v
gcc version 7.1.1 20170530 [gcc-7-branch revision 248621] (SUSE Linux) 

They are (of course) identical to those of Fedora26.

Glynn, any other output which I should generate?

by neteler, 7 years ago

gcc7.1.1 definitions on Fedora26

by neteler, 7 years ago

gcc7.1.1 definitions on openSUSE Tumbleweed

in reply to:  21 comment:33 by glynn, 7 years ago

Replying to brunofriedmann:

Excuse me, but how should apply the last 2 patches ?

Either patch should work on its own. The first (floats1.diff) just avoids the exception, by preventing strings of digits which contain 8 or 9 from being parsed as octal. The second (floats2.diff) expands the regexp for floating-point literals so that the additional forms are parsed correctly.

in reply to:  24 ; comment:34 by glynn, 7 years ago

Replying to neteler:

While we are at it: I found three parsing errors with the patched ctypes version (F25, gcc6):

Warning: Could not parse macro "#define serialize_int32_le(buf,x)
Warning: Could not parse macro "#define serialize_int32_be(buf,x)

Long-standing, harmless. It just means that those definitions won't be added to gis.py. It's not as if anything is likely to want them; they don't even appear to be used within GRASS itself.

This:

Error: /usr/lib/gcc/x86_64-redhat-linux/6.3.1/include/stddef.h:427: Syntax error at '__attribute__'

might be a problem if it prevents gmath.py from being generated. It can probably be worked around with something like:

#ifdef CTYPESGEN
#undef __attribute__
#define __attribute__(x)
#endif

at the top of gmath.h (before including stddef.h).

This:

Warning: Could not parse macro "#define TGIS STR3DS i5"

is caused by an error in temporal.h; it should be

#define TGIS_STR3DS       5

with an underscore.

comment:35 by brunofriedmann, 7 years ago

Ok finally I understood my error, I've applied both patches. If I only work with floats2 the build work. I will now commit my changes, and check if it build on every plateform (from arm to s390 and powerpc) and also old release

in reply to:  34 ; comment:36 by neteler, 7 years ago

Replying to glynn:

Replying to neteler:

While we are at it: I found three parsing errors with the patched ctypes version

...

This:

Error: /usr/lib/gcc/x86_64-redhat-linux/6.3.1/include/stddef.h:427: Syntax error at '__attribute__'

might be a problem if it prevents gmath.py from being generated. It can probably be worked around with something like:

#ifdef CTYPESGEN
#undef __attribute__
#define __attribute__(x)
#endif

at the top of gmath.h (before including stddef.h).

That helps, submitted in r71201 and backported to relbr72 and relbr70.

This:

Warning: Could not parse macro "#define TGIS STR3DS i5"

is caused by an error in temporal.h; it should be

#define TGIS_STR3DS       5

with an underscore.

Oh :) Fixed in r71198 and backported to relbr72 and relbr70.

Remains how to deal with the gcc7 ctpes lexer problem. At time both floats1.diff and floats2.diff are added to trunk, perhaps only one should be used? Hints welcome.

in reply to:  36 ; comment:37 by hellik, 7 years ago

Replying to neteler:

Remains how to deal with the gcc7 ctpes lexer problem. At time both floats1.diff and floats2.diff are added to trunk, perhaps only one should be used? Hints welcome.

windows builds are broken in grass_trunk/lib/python/ctypes error log

GRASS GIS 7.3.svn r71204 compilation log
--------------------------------------------------
Started compilation: Wed Jun 21 21:28:48     2017
--
Errors in:
/c/msys64/usr/src/grass_trunk/lib/python/ctypes
--
In case of errors please change into the directory with error and run 'make'.
If you get multiple errors, you need to deal with them in the order they
appear in the error log. If you get an error building a library, you will
also get errors from anything which uses the library.
--
Finished compilation: Wed Jun 21 22:01:28     2017

in reply to:  37 ; comment:38 by annakrat, 7 years ago

Replying to hellik:

Replying to neteler:

Remains how to deal with the gcc7 ctpes lexer problem. At time both floats1.diff and floats2.diff are added to trunk, perhaps only one should be used? Hints welcome.

windows builds are broken in grass_trunk/lib/python/ctypes error log

Here is the error:

https://wingrass.fsv.cvut.cz/grass73/x86_64/logs/log-r71204-275/package.log

Also currently it doesn't compile on Mac OSX (Yosemite).

by annakrat, 7 years ago

Attachment: macosx_error.txt added

build error on Mac OSX

in reply to:  38 comment:39 by neteler, 7 years ago

Replying to annakrat:

https://wingrass.fsv.cvut.cz/grass73/x86_64/logs/log-r71204-275/package.log

Here the excerpt:

Traceback (most recent call last):
  File "./ctypesgen.py", line 139, in <module>
    descriptions = ctypesgencore.parser.parse(options.headers, options)
  File "./ctypesgencore/parser/__init__.py", line 22, in parse
    parser.parse()
  File "./ctypesgencore/parser/datacollectingparser.py", line 74, in parse
    ctypesparser.CtypesParser.parse(self, fname, None)
  File "./ctypesgencore/parser/cparser.py", line 120, in parse
    self.preprocessor_parser.parse(filename)
  File "./ctypesgencore/parser/preprocessor.py", line 218, in parse
    token = self.lexer.token()
  File "./ctypesgencore/parser/preprocessor.py", line 63, in token
    result = lex.Lexer.token(self)
  File "./ctypesgencore/parser/lex.py", line 404, in token
    lexpos:])
ctypesgencore.parser.lex.LexError: Scanning error. Illegal character '0'
Makefile:102: recipe for target 'OBJ.x86_64-w64-mingw32/ogsf.py' failed

in reply to:  38 comment:40 by neteler, 7 years ago

Replying to annakrat:

Also currently it doesn't compile on Mac OSX (Yosemite).

Can you please locally revert the float1.diff change, remove the lextab.py* and try again?

comment:41 by neteler, 7 years ago

I have partial reverted r71196 in r71213, i.e. removed the float1.diff change. Hope it now compiles again (on F26/gcc7 it does).

Last edited 7 years ago by neteler (previous) (diff)

in reply to:  41 comment:42 by annakrat, 7 years ago

Replying to neteler:

I have partial reverted r71196: removed the float1.diff change. Hope it now compiles again (on F26/gcc7 it does).

Yes, now it compiles on Mac and ctypes work.

in reply to:  37 ; comment:43 by neteler, 7 years ago

Replying to hellik:

windows builds are broken in grass_trunk/lib/python/ctypes error log

After the float1.diff revert from yesterday also the Windows binaries are compiled again (see log in https://wingrass.fsv.cvut.cz/grass73/x86_64/logs/log-r71214-276/)

Please install the latest binary and check if the digitizer works ok, thanks.

in reply to:  43 comment:44 by hellik, 7 years ago

Replying to neteler:

Replying to hellik:

windows builds are broken in grass_trunk/lib/python/ctypes error log

After the float1.diff revert from yesterday also the Windows binaries are compiled again (see log in https://wingrass.fsv.cvut.cz/grass73/x86_64/logs/log-r71214-276/)

Please install the latest binary and check if the digitizer works ok, thanks.

tested with

System Info                                                                     
GRASS version: 7.3.svn                                                          
GRASS SVN revision: r71214                                                      
Build date: 2017-06-24                                                          
Build platform: x86_64-w64-mingw32                                              
GDAL: 2.2.0                                                                     
PROJ.4: 4.9.3                                                                   
GEOS: 3.5.0                                                                     
SQLite: 3.17.0                                                                  
Python: 2.7.5                                                                   
wxPython: 2.8.12.1                                                              
Platform: Windows-8-6.2.9200 (OSGeo4W) 
Editing of vector map <testdigi@user1> successfully finished                    
Building topology for vector map <testdigi@user1>...
Registering primitives...
4 primitives registered
17 vertices registered
Building areas...
One area built
One isle built
Attaching islands...
Attaching centroids...
Number of nodes: 3
Number of primitives: 4
Number of points: 1
Number of lines: 1
Number of boundaries: 1
Number of centroids: 1
Number of areas: 1
Number of isles: 1

confirmed, it works again.

comment:45 by neteler, 7 years ago

If there are no objections I'll backport the needed changes.

in reply to:  45 comment:46 by neteler, 7 years ago

Milestone: 7.2.27.0.6
Resolution: fixed
Status: newclosed

Replying to neteler:

If there are no objections I'll backport the needed changes.

Backported

Closing.

comment:47 by lucadelu, 6 years ago

Resolution: fixed
Status: closedreopened

Should the patch also backported to GRASS 6.4?

I got the same bug and I fixed applying the patch...

in reply to:  47 comment:48 by neteler, 6 years ago

Replying to lucadelu:

Should the patch also backported to GRASS 6.4?

I got the same bug and I fixed applying the patch...

Yes, please apply to releasebranch_6_4

comment:49 by lucadelu, 6 years ago

Resolution: fixed
Status: reopenedclosed

Ok, backported in r72361.

Note: See TracTickets for help on using tickets.