Opened 12 years ago
Last modified 12 years ago
#4223 new enhancement
Tile creation and symlink creation failure in cache_disk.c on nfs
Reported by: | tomhel | Owned by: | tbonfort |
---|---|---|---|
Priority: | normal | Milestone: | |
Component: | MapCache | Version: | svn-trunk (development) |
Severity: | normal | Keywords: | |
Cc: |
Description
When seeding with more than one thread we get intermittent errors that seeding has failed because the directory where the tile is to be stored does not exist.
This error only happens when seeding to nfs mounted network storage. We do not get these errors when seeding to local disk.
Numerous mounting parameters have been tried, but no success.
What we did to solve this was to add a check to cache_disk.c, so it checks if a directory exists before writing a tile or symlink. If the directory does not exist, it will try to create it before writing the tile. We have it set to retry this 10 times before aborting. Our tests have shown that one retry might be sufficient.
The following patch adds the following functionality to cache_disk.c:
- if writing a tile fails, it will try to create the destination directory and retry. If the tile still fail to write. we try 10 more times before aborting.
- Same goes for symlink creation.
Attachments (4)
Change History (8)
by , 12 years ago
Attachment: | retry_tile_creation.patch added |
---|
follow-up: 2 comment:1 by , 12 years ago
Could you provide a patch where the retry count is a configurable option of the disk cache? Should be easy enough to implement with an int member to mapcache_cache_disk and a couple of lines more in the xml parser.
-- thomas
by , 12 years ago
Attachment: | mapcache_h_creation_retry.patch added |
---|
adds int member to disk cache for file creation retries
by , 12 years ago
Attachment: | cache_disk_c_creation_retry.patch added |
---|
adds xml parsing and retry logic for file creation
comment:2 by , 12 years ago
Replying to tbonfort:
Could you provide a patch where the retry count is a configurable option of the disk cache? Should be easy enough to implement with an int member to mapcache_cache_disk and a couple of lines more in the xml parser.
-- thomas
Sure, no problem. The new element i introduced for the disk cache is called creation_retry, it will default to 0 if not specified or is invalid.
<cache name="disk" type="disk"> <base>/tmp/cache</base> <symlink_blank/> <creation_retry>10</creation_retry> </cache>
Tommy
by , 12 years ago
Attachment: | blank_tile_retry_logic.patch added |
---|
adds retry logic for blank tile creation aswell (in cache_disk.c)
comment:4 by , 12 years ago
I forgot to add retry logic for blank tile creation. I uploaded another patch that adds retry logic for this case as well. Sorry about that.
adds functionality to cache_disk.c to retry tile and symlink creation