Opened 9 years ago
Closed 9 years ago
#2607 closed defect (fixed)
[raster] Unable to open more than 1024 out-db files in a single connection
|Reported by:||Bborie Park||Owned by:||Bborie Park|
On a large query spanning every day of many years, I get the following error:
rt_band_load_offline_data: Cannot open offline raster: SOME_RASTER_FILE
In the PostgreSQL logs…
ERROR 4: `SOME_RASTER_FILE' not recognised as a supported file format.
I know these files are all valid rasters (as per GDAL).
Pruning the query to access ⇐ 1024 raster files permits the query to complete. This does have the assumption that the connection is new and there-fore the process has no open out-db raster files.
Googling (can't find link) indicated that GDAL limits the number of open files to 1024. Just in case this was not true, I did alter the soft and hard nofile limits of the system for the postgres user with no luck.
The immediate solution is to not use GDALOpenShared for opening out-db rasters but instead GDALOpen. A better solution is keep track of the out-db rasters opened in the process and then close those as needed.
Change History (2)
comment:1 by , 9 years ago
|Status:||new → assigned|
comment:2 by , 9 years ago
|Status:||assigned → closed|
Fixed in trunk as of r12185. Fixed in -2.1 as of r12186. Fixed in -2.0 as of r12187.
Fixed by replacing GDALOpenShared with GDALOpen. Need to explore how to get GDALOpenShared back by keeping a cache of open GDAL datasets and when the cache count gets close to the max permitted (~1000), close out the some of the oldest datasets.