Avoid errors in parallel creation of tempfiles

[sbl]

When tempfiles are created in parallel, conflicts and thus fatal errors can arise when two processes try to create the .tmp/HOST dir at the same time: ​https://lists.osgeo.org/pipermail/grass-dev/2017-March/084471.html

The conflict seems to arise only from the directory.

So, maybe it could help if the directory is created at the start of a GRASS session and removed only when the GRASS session is closed (exit)?

[mlennert]

Replying to sbl:

When tempfiles are created in parallel, conflicts and thus fatal errors can arise when two processes try to create the .tmp/HOST dir at the same time: ​https://lists.osgeo.org/pipermail/grass-dev/2017-March/084471.html

The conflict seems to arise only from the directory.

So, maybe it could help if the directory is created at the start of a GRASS session and removed only when the GRASS session is closed (exit)?

Now that you mention it, I can confirm that I've also been confronted to such errors when running commands in parallel that create temp files. So +1 to needing a solution.

I don't know though if the best is to create this directory once and for all at the startup, or whether the tempfile routines in lib/gis/tempfile.c could be modified to avoid such race conditions.

[mmetz]

Replying to mlennert:

Replying to sbl:

When tempfiles are created in parallel, conflicts and thus fatal errors can arise when two processes try to create the .tmp/HOST dir at the same time: ​https://lists.osgeo.org/pipermail/grass-dev/2017-March/084471.html

The conflict seems to arise only from the directory.

So, maybe it could help if the directory is created at the start of a GRASS session and removed only when the GRASS session is closed (exit)?

Now that you mention it, I can confirm that I've also been confronted to such errors when running commands in parallel that create temp files. So +1 to needing a solution.

Strange. When starting GRASS in text or GUI mode, the .tmp/HOST folder is automatically created, i.e. the error

ERROR: Unable to make mapset element .tmp/HOST (/grassdata/ETRS_33N/timseries/.tmp): File exists

should not happen, at least not in trunk and relb72.

I don't know though if the best is to create this directory once and for all at the startup, or whether the tempfile routines in lib/gis/tempfile.c could be modified to avoid such race conditions.

You could use

rm -f "`g.tempfile pid=$$`"

before running xargs, just to make sure that .tmp/HOST has been created.

[sbl]

I run the xargs command in a GRASS_BATCH_JOB in case that can be of relevance...

Will test if

rm -f "`g.tempfile pid=$$`"

helps and report back...

[sbl]

For the record:

Running

rm -f "`g.tempfile pid=$$`"

in advance works around the issue.

Using GRASS 7.2.1svn (r70188) in BATCH mode I do get the error message above. But if I run g.tempfile and make sure that the tmp directory is created before I run r.in.gdal through xargs, everything works just fine!

[martinl]

All enhancement tickets should be assigned to 7.4 milestone.

[neteler]

Ticket retargeted after milestone closed

[neteler]

Does this issue persist in GRASS GIS 7.6.svn?

[wenzeslaus]

Replying to neteler:

Does this issue persist in GRASS GIS 7.6.svn?

Please, if you can, test trunk (7.7) too - a lot of changes to temporary directory handling (#613, #1286, #3635, ...). And if possible, please, test also GRASS_BATCH_JOB versus --exec.

[martinl]

All enhancement tickets should be assigned to 7.6 milestone.

[martinl]

Ticket retargeted after milestone closed

[martinl]

Ticket retargeted after milestone closed