Tile creation and symlink creation failure in cache_disk.c on nfs

[tomhel]

When seeding with more than one thread we get intermittent errors that seeding has failed because the directory where the tile is to be stored does not exist.

This error only happens when seeding to nfs mounted network storage. We do not get these errors when seeding to local disk.

Numerous mounting parameters have been tried, but no success.

What we did to solve this was to add a check to cache_disk.c, so it checks if a directory exists before writing a tile or symlink. If the directory does not exist, it will try to create it before writing the tile. We have it set to retry this 10 times before aborting. Our tests have shown that one retry might be sufficient.

The following patch adds the following functionality to cache_disk.c:

• if writing a tile fails, it will try to create the destination directory and retry. If the tile still fail to write. we try 10 more times before aborting.
• Same goes for symlink creation.

[tomhel]

adds functionality to cache_disk.c to retry tile and symlink creation

[tbonfort]

Could you provide a patch where the retry count is a configurable option of the disk cache? Should be easy enough to implement with an int member to mapcache_cache_disk and a couple of lines more in the xml parser.

-- thomas

[tomhel]

adds int member to disk cache for file creation retries

[tomhel]

adds xml parsing and retry logic for file creation

[tomhel]

Replying to tbonfort:

Could you provide a patch where the retry count is a configurable option of the disk cache? Should be easy enough to implement with an int member to mapcache_cache_disk and a couple of lines more in the xml parser.

-- thomas

Sure, no problem. The new element i introduced for the disk cache is called creation_retry, it will default to 0 if not specified or is invalid.

<cache name="disk" type="disk">
   <base>/tmp/cache</base>
   <symlink_blank/>
   <creation_retry>10</creation_retry>
</cache>

Tommy

[tbonfort]

applied in r13203 , thanks for the patch!

[tomhel]

adds retry logic for blank tile creation aswell (in cache_disk.c)

[tomhel]

I forgot to add retry logic for blank tile creation. I uploaded another patch that adds retry logic for this case as well. Sorry about that.