Opened 5 years ago
Closed 5 years ago
#3841 closed defect (invalid)
script/core.py: create_location() fails with FileNotFoundError: [Errno 2] No such file or directory: '/tmp/grass7-root-94/tmploc/PERMANENT/MYNAME'
| Reported by: | neteler | Owned by: | |
|---|---|---|---|
| Priority: | normal | Milestone: | 7.8.0 |
| Component: | Python | Version: | svn-trunk |
| Keywords: | python3, grass.py | Cc: | |
| CPU: | x86-64 | Platform: | Linux |
Description
In a docker environment (Ubuntu 18:04 based) I get this startup error with trunk (on a pure Python 3 environment):
# note: the second line from bottom is local debug output via print() root@6ade79b80a2b:/grassdb# grass --tmp-location EPSG:25832 --exec g.extension -s extension=i.sentinel url=/src/i.sentinel Starting GRASS GIS... Creating new GRASS GIS location <tmploc>... /tmp/grass7-root-37 tmploc EPSG:25832 ERROR: FileNotFoundError(2, 'No such file or directory')
With Python debugger (pdb) I see that it cannot open a file with codecs.open():
root@6ade79b80a2b:/grassdb# grass --tmp-location EPSG:25832 --exec g.extension -s extension=i.sentinel url=/src/i.sentinel
Starting GRASS GIS...
Creating new GRASS GIS location <tmploc>...
/tmp/grass7-root-94 tmploc EPSG:25832
25832 None
> /usr/local/bin/grass(853)create_location()
-> gcore.create_location(gisdbase, location,
(Pdb) s
> /usr/local/bin/grass(854)create_location()
-> epsg=epsg, datum_trans=datum_trans)
(Pdb) s
--Call--
> /usr/local/grass77/etc/python/grass/script/core.py(1518)create_location()
-> def create_location(dbase, location, epsg=None, proj4=None, filename=None,
(Pdb) s
> /usr/local/grass77/etc/python/grass/script/core.py(1538)create_location()
-> gisdbase = None
(Pdb) s
> /usr/local/grass77/etc/python/grass/script/core.py(1539)create_location()
-> if epsg or proj4 or filename or wkt:
(Pdb) s
> /usr/local/grass77/etc/python/grass/script/core.py(1541)create_location()
-> gisdbase = gisenv()['GISDBASE']
(Pdb) s
--Call--
> /usr/local/grass77/etc/python/grass/script/core.py(1075)gisenv()
-> def gisenv(env=None):
(Pdb) s
> /usr/local/grass77/etc/python/grass/script/core.py(1086)gisenv()
-> s = read_command("g.gisenv", flags='n', env=env)
(Pdb) s
--Call--
> /usr/local/grass77/etc/python/grass/script/core.py(491)read_command()
-> def read_command(*args, **kwargs):
(Pdb) s
> /usr/local/grass77/etc/python/grass/script/core.py(500)read_command()
-> encoding = 'default'
(Pdb) s
> /usr/local/grass77/etc/python/grass/script/core.py(501)read_command()
-> if 'encoding' in kwargs:
(Pdb) s
> /usr/local/grass77/etc/python/grass/script/core.py(504)read_command()
-> if _capture_stderr and 'stderr' not in kwargs.keys():
(Pdb) s
> /usr/local/grass77/etc/python/grass/script/core.py(506)read_command()
-> process = pipe_command(*args, **kwargs)
[...]
> /usr/local/grass77/etc/python/grass/script/core.py(512)read_command()
-> if _capture_stderr and returncode:
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(514)read_command()
-> return handle_errors(returncode, stdout, args, kwargs)
(Pdb) n
--Return--
> /usr/local/grass77/etc/python/grass/script/core.py(514)read_command()->'GISDBASE=/gr...T=<UNKNOWN>\n'
-> return handle_errors(returncode, stdout, args, kwargs)
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1087)gisenv()
-> return parse_key_val(s)
(Pdb) n
--Return--
> /usr/local/grass77/etc/python/grass/script/core.py(1087)gisenv()->{'GISDBASE': ...: '<UNKNOWN>'}
-> return parse_key_val(s)
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1542)create_location()
-> run_command('g.gisenv', set='GISDBASE=%s' % dbase)
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1544)create_location()
-> if not os.path.exists(dbase):
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1548)create_location()
-> if os.path.exists(os.path.join(dbase, location)):
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1556)create_location()
-> kwargs = dict()
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1557)create_location()
-> if datum:
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1559)create_location()
-> if datum_trans:
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1562)create_location()
-> if epsg:
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1563)create_location()
-> ps = pipe_command('g.proj', quiet=True, flags='t', epsg=epsg,
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1564)create_location()
-> location=location, stderr=PIPE, **kwargs)
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1577)create_location()
-> if epsg or proj4 or filename or wkt:
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1578)create_location()
-> error = ps.communicate()[1]
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1579)create_location()
-> run_command('g.gisenv', set='GISDBASE=%s' % gisdbase)
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1581)create_location()
-> if ps.returncode != 0 and error:
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1584)create_location()
-> try:
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1585)create_location()
-> fd = codecs.open(os.path.join(dbase, location, 'PERMANENT', 'MYNAME'),
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1586)create_location()
-> encoding='utf-8', mode='w')
(Pdb) n
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/grass7-root-94/tmploc/PERMANENT/MYNAME'
> /usr/local/grass77/etc/python/grass/script/core.py(1586)create_location()
-> encoding='utf-8', mode='w')
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1592)create_location()
-> except OSError as e:
(Pdb) n
> /usr/local/grass77/etc/python/grass/script/core.py(1593)create_location()
-> raise ScriptError(repr(e))
(Pdb) n
grass.exceptions.ScriptError: FileNotFoundError(2, 'No such file or directory')
> /usr/local/grass77/etc/python/grass/script/core.py(1593)create_location()
-> raise ScriptError(repr(e))
(Pdb) n
--Return--
> /usr/local/grass77/etc/python/grass/script/core.py(1593)create_location()->None
-> raise ScriptError(repr(e))
(Pdb) n
grass.exceptions.ScriptError: FileNotFoundError(2, 'No such file or directory')
> /usr/local/bin/grass(854)create_location()
-> epsg=epsg, datum_trans=datum_trans)
(Pdb) n
> /usr/local/bin/grass(864)create_location()
-> except gcore.ScriptError as err:
(Pdb) n
> /usr/local/bin/grass(865)create_location()
-> fatal(err.value.strip('"').strip("'").replace('\\n', os.linesep))
(Pdb) n
ERROR: FileNotFoundError(2, 'No such file or directory')
SystemExit: Exiting...
> /usr/local/bin/grass(865)create_location()
-> fatal(err.value.strip('"').strip("'").replace('\\n', os.linesep))
(Pdb) n
--Return--
> /usr/local/bin/grass(865)create_location()->None
-> fatal(err.value.strip('"').strip("'").replace('\\n', os.linesep))
The code in question:
https://trac.osgeo.org/grass/browser/grass/trunk/lib/python/script/core.py#L1573
1573 try: 1574 fd = codecs.open(os.path.join(dbase, location, 'PERMANENT', 'MYNAME'), 1575 encoding='utf-8', mode='w') 1576 if desc: 1577 fd.write(desc + os.linesep) 1578 else: 1579 fd.write(os.linesep) 1580 fd.close() 1581 except OSError as e: 1582 raise ScriptError(repr(e))
Version used:
lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.04.2 LTS Release: 18.04 Codename: bionic python3 --version Python 3.6.7
Any ideas here?
Change History (4)
comment:1 by , 5 years ago
follow-up: 3 comment:2 by , 5 years ago
Confirmed, that PERMANENT isn't there (would be cool if core.py could say so...).
After further debugging: The reason is indeed a crash of g.proj:
root@cea3e6ed4c0c:/grassdb# grass --tmp-location EPSG:25832 --exec g.version -gb Starting GRASS GIS... Creating new GRASS GIS location <tmploc>... > /usr/local/grass77/etc/python/grass/script/core.py(1584)create_location() -> try: (Pdb) s > /usr/local/grass77/etc/python/grass/script/core.py(1585)create_location() -> fd = open(os.path.join(dbase, location, 'PERMANENT', 'MYNAME'), (Pdb) [1]+ Stopped grass --tmp-location EPSG:25832 --exec g.version -gb root@cea3e6ed4c0c:/grassdb# l /tmp/ grass7-root-50/ simple.laz root@cea3e6ed4c0c:/grassdb# l /tmp/grass7-root-50/ gisrc
Test with an existing location:
GRASS 7.7.svn (nc_basic_spm_grass7):/grassdb > g.proj -t epsg=25832 location=myloc Illegal instruction (core dumped)
PROJ Version in this Ubuntu docker:
cs2cs -v Using from definition: Rel. 4.9.3, 15 August 2016 GRASS 7.7.svn (nc_basic_spm_grass7):/grassdb > ldd /usr/local/grass77/bin/g.proj | grep proj libgrass_gproj.7.7.so => /usr/local/grass77/lib/libgrass_gproj.7.7.so (0x00007f4b04011000) libproj.so.12 => /usr/lib/x86_64-linux-gnu/libproj.so.12 (0x00007f4b0294a000)
GNU Debugger:
GRASS 7.7.svn (nc_basic_spm_grass7):/grassdb > gdb g.proj GNU gdb (Ubuntu 8.1-0ubuntu3) 8.1.0.20180409-git (gdb) r -t epsg=25832 location=myloc Starting program: /usr/local/grass77/bin/g.proj -t epsg=25832 location=myloc warning: Error disabling address space randomization: Operation not permitted [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New Thread 0x7f5b60c7c700 (LWP 193)] [New Thread 0x7f5b6047b700 (LWP 194)] [New Thread 0x7f5b5dc7a700 (LWP 195)] [New Thread 0x7f5b59479700 (LWP 196)] [New Thread 0x7f5b56c78700 (LWP 197)] [New Thread 0x7f5b54477700 (LWP 198)] [New Thread 0x7f5b53c76700 (LWP 199)] Thread 1 "g.proj" received signal SIGILL, Illegal instruction. 0x00007f5b77d18b7e in read_datum_table () from /usr/local/grass77/lib/libgrass_gproj.7.7.so (gdb) bt full #0 0x00007f5b77d18b7e in read_datum_table () from /usr/local/grass77/lib/libgrass_gproj.7.7.so No symbol table info available. #1 0x00007f5b77d17d0b in GPJ_osr_to_grass () from /usr/local/grass77/lib/libgrass_gproj.7.7.so No symbol table info available. #2 0x00005621585e8ceb in input_epsg () No symbol table info available. #3 0x00005621585e7de3 in main () No symbol table info available.
Installed PROJ related packages:
dpkg-query -l | grep proj ii libproj-dev:amd64 4.9.3-2 amd64 Cartographic projection library (development files) ii libproj12:amd64 4.9.3-2 amd64 Cartographic projection library ii proj-data 4.9.3-2 all Cartographic projection filter and library (datum package) # the datum files are apparently installed: apt-file list proj-data proj-data: /usr/share/doc/proj-data/AUTHORS proj-data: /usr/share/doc/proj-data/NEWS.Debian.gz proj-data: /usr/share/doc/proj-data/NEWS.gz [...] proj-data: /usr/share/proj/alaska proj-data: /usr/share/proj/conus proj-data: /usr/share/proj/epsg [...]
I have no clue why g.proj crashes here.
comment:3 by , 5 years ago
Replying to neteler:
Confirmed, that PERMANENT isn't there (would be cool if core.py could say so...).
... to have a more reasonabe error message here would be greatly appreciated.
Back to the problem:
After further debugging: The reason is indeed a crash of g.proj: ... GNU Debugger:
GRASS 7.7.svn (nc_basic_spm_grass7):/grassdb > gdb g.proj GNU gdb (Ubuntu 8.1-0ubuntu3) 8.1.0.20180409-git (gdb) r -t epsg=25832 location=myloc Starting program: /usr/local/grass77/bin/g.proj -t epsg=25832 location=myloc ... Thread 1 "g.proj" received signal SIGILL, Illegal instruction. ......
I have no clue why g.proj crashes here.
Likely found it with support by Markus Metz: https://github.com/mundialis/grass-py3-pdal/commit/b7b4058a9e52a0a08dca5b819831b54583dcbacc
Take home message: do not use -march=native for gcc in a Dockerfile.
(docker image rebuilding is ongoing).
comment:4 by , 5 years ago
| Resolution: | → invalid |
|---|---|
| Status: | new → closed |
Solved by avoiding -march=native for gcc in a Dockerfile.
Note:
See TracTickets
for help on using tickets.
I think the error is that the PERMANENT directory does not exist.