Need better handling of temporary files

[dmorissette]

At the moment, we write some temporary files in the web-accessible IMAGEPATH directory, this was a poor practice but still okay for some uses in the past (such as writing CONNECTIONTYPE WMS and WFS responses while we process them), but as our need for temporary files increase, we need to ensure that temp files are handled in a proper and safer way.

One option could be to adopt the use of a $MS_TMPDIR environment variable to specify the temporary directory path. If $MS_TMPDIR is not set, then we would fallback on $TMPDIR (commonly used on Unix), and if not, on /tmp on Unix/Linux, or some path TBD on Windows.

We could also look into the use of system calls to more safely handle temp files. One possibility is tmpfile(), but the drawback of this one is that it automatically deletes the files once it's closed and in some cases we don't want that.

[assefa]

On windows, environment variables TEMP and TMP could be used for testing, and then c:\temp as last resort.

[dmorissette]

See also #3355 (about OGC-specific uses of temp files)

[sdlime]

Agreed this is a need. Very few use cases need to use the image path, that is, only for writing images. What about explicitly setting a path in a mapfile (e.g. TEMPPATH) and then falling back to env vars?

Steve

[aboudreault]

I agree with the TMPPATH in the mapfile. If this var is now set, on Linux, then check $MS_TMPDIR env var, $TMPDIR, otherwise use /tmp. On Windows, TMPPATH, $MS_TMPDIR, $TEMP, $TMP, otherwise use c:\temp.

Does everyone agree with this behavior? What about MAC ?

[aboudreault]

I've just committed the RFC 66 in r10883. Assefa will check and fix the Windows part as soon as he can.

[assefa]

Alan,

builds on windows with r10888. Order of things on windows is as follows.

1 - TMPATH
2 - MS_TMPPATH env variable
3 - The path specified by the TMP environment variable.
4 - The path specified by the TEMP environment variable.
5 - The path specified by the USERPROFILE environment variable.
6 - The Windows directory.

I have tested this with the sld temporary file.

[assefa]

Alan,

I would like to be able to use the msTmpFile to get a temporary full path file name for a given filename. Right now the function allows the possibility to pass path and extension but not a file name.

In my case I would like to create a file and I know the name I want but I do not have to worry about the temporary directory location. Does this make sense? We could add an additional argument to the function.

[assefa]

Forgot to note that we can also have msTmpPath that will return the temp path and I can construct the full path. Both ways are the same to me.

[aboudreault]

I assume you wil always delete that file after the task... otherwise it wouldn't safe to have a static filename. I can effectively separate the msTmpFile function and create a msTmpPath.

[aboudreault]

Committed the msTmpPath function in r10900. The function will return the temporary path based on the os and arguments. ie: (linux)

msTmpPath(NULL,NULL,NULL); // returns /tmp
msTmpPath(NULL,"/a_map_path",NULL); // returns /a_map_path/tmp
msTmpPath(map,NULL,NULL); // with temppath set, returns /the_temppath/
msTmpPath(NULL,NULL,"/specific_path/tmp/"); // returns /specific_path/tmp/
// etc.

Please confirm that it fixes your need.

[assefa]

Thanks Alan for adding that. I use it in my case for #3662. I do not delete the tmp file and use it if it exists, the idea being able to use the tmp file as a cache for a symbol file downloaded.

[aboudreault]

Closing the ticket.

[havatv]

As far as I can see, this has not been documented. Reopening and moving to documentation. #4103 is related.

[havatv]

Documentation of TEMPPATH and MS_TEMPPATH was added to the mapfile web document for 6.0 and trunk in r12839. What else should be documented?

[aboudreault]

Looks good to me. Thanks a lot havatv!