Opened 7 months ago

Last modified 7 months ago

#3801 new defect

r.watershed crashing on large rasters

Reported by: dnewcomb Owned by: grass-dev@…
Priority: normal Milestone: 7.6.2
Component: Default Version: 7.4.1
Keywords: Cc:
CPU: x86-64 Platform: Linux

Description

Attempting to run r.watershed on a 13.7 billion pixel raster elevation layer

r.watershed --overwrite elevation=cfsdem05 accumulation=cfs_accum drainage=cfs_drain

SECTION 1a (of 4): Initiating Memory. WARNING: Subprocess failed with exit code 11 WARNING: category information for [cfs_accum] in [PERMANENT] missing or

invalid

WARNING: category information for [cfs_drain] in [PERMANENT] missing or

invalid

I have tried this with Memory=100000 MB

Does r.watershed need more than 100GB of memory to run on a 14 billion pixel raster?

The raster is linked via r.external, doe it need to be imported ?

Looking backwards, the enhancement to run large raster was done for GRASS 7.0 , is this a regression, or was it not tested to larger than 9 billion cells?

Change History (2)

comment:1 in reply to:  description Changed 7 months ago by mmetz

Replying to dnewcomb:

Attempting to run r.watershed on a 13.7 billion pixel raster elevation layer

r.watershed --overwrite elevation=cfsdem05 accumulation=cfs_accum drainage=cfs_drain

SECTION 1a (of 4): Initiating Memory. WARNING: Subprocess failed with exit code 11 WARNING: category information for [cfs_accum] in [PERMANENT] missing or

invalid

WARNING: category information for [cfs_drain] in [PERMANENT] missing or

invalid

I have tried this with Memory=100000 MB

The memory option has an effect only when using the -m flag. I don't think r.watershed in all-in-memory mode (no -m flag) is supporting such large input maps.

Does r.watershed need more than 100GB of memory to run on a 14 billion pixel raster?

The raster is linked via r.external, doe it need to be imported ?

Looking backwards, the enhancement to run large raster was done for GRASS 7.0 , is this a regression, or was it not tested to larger than 9 billion cells?

comment:2 Changed 7 months ago by martinl

Milestone: 7.6.17.6.2

Ticket retargeted after milestone closed

Note: See TracTickets for help on using tickets.