Opened 8 years ago

Closed 8 years ago

Last modified 8 years ago

#1696 closed defect (wontfix)

[raster] raster2pgsql fails with big raster on Windows

Reported by: pracine Owned by: pramsey
Priority: medium Milestone: PostGIS 2.0.0
Component: postgis Version: trunk
Keywords: Cc:

Description

Trying to import a relatively big raster (500MB) without tiling it crashes without warning.

The image is available here: ftp://ftp.ccrs.nrcan.gc.ca/ftp/ad/NLCCLandCover/LandcoverCanada2005_250m/

Change History (4)

comment:1 Changed 8 years ago by pracine

Summary: [raster] raster2pgsql crashes badly on Windows[raster] raster2pgsql fails with big raster on Windows

C:\Windows\System32>raster2pgsql -s 3978 -F "C:/temp/x.tif" test.test > "C:/temp/x.sql" Processing 1/1: C:/temp/x.tif ERROR: rt_raster_to_hexwkb: Out of memory hexifying raster WKB ERROR: convert_raster: Could not convert PostGIS raster to hex WKB ERROR: process_rasters: Could not process raster: C:/temp/x.tif ERROR: Unable to process rasters

comment:2 Changed 8 years ago by Bborie Park

Resolution: wontfix
Status: newclosed

I expect that to fail. Everything of raster2psql makes use of the functions provide in rt_api, where all functions operate in memory. As such, if that raster was in the database, it would also fail when any database operation is done on it.

As to where memory is being consumed in the loader...

  1. GDAL natural block (some subset of the raster)
  1. postgis raster internal data structure (full raster)
  1. hexwkb of step #2 (full raster x2 as hex is twice the size of the binary raster)

comment:3 Changed 8 years ago by pracine

So what is the limit? Is it dependent on the memory available on one machine?

There is no way the error messages can be more friendly?: "The raster you're trying to load in PostGIS does not hold in memory. You should tile it using the -t raster2pgsql option"...

comment:4 Changed 8 years ago by Bborie Park

The limit depends on the machine.

It may be worth adding an "INFO" message to raster2pgsql that is emitted when the tile flag -t isn't set.

Note: See TracTickets for help on using tickets.