[raster] Unable to open more than 1024 out-db files in a single connection
On a large query spanning every day of many years, I get the following error:
rt_band_load_offline_data: Cannot open offline raster: SOME_RASTER_FILE
In the PostgreSQL logs…
ERROR 4: `SOME_RASTER_FILE' not recognised as a supported file format.
I know these files are all valid rasters (as per GDAL).
Pruning the query to access ⇐ 1024 raster files permits the query to complete. This does have the assumption that the connection is new and there-fore the process has no open out-db raster files.
Googling (can't find link) indicated that GDAL limits the number of open files to 1024. Just in case this was not true, I did alter the soft and hard nofile limits of the system for the postgres user with no luck.
The immediate solution is to not use GDALOpenShared for opening out-db rasters but instead GDALOpen. A better solution is keep track of the out-db rasters opened in the process and then close those as needed.