Opened 9 years ago

Last modified 5 years ago

#5767 closed defect

gdal_calc: input data type used for output values — at Version 1

Reported by: cdestigter Owned by: Robert Coup
Priority: normal Milestone: closed_because_of_github_migration
Component: PythonBindings Version: svn-trunk
Severity: normal Keywords:
Cc: antonio

Description (last modified by cdestigter)

We're using gdal_calc.py like this to replace alpha with nodata (-1) in RGBA and grey+alpha images:

gdal_calc.py -A in.tif -B in.tif --A_band 2 --B_band 1 --outfile out.tif --calc="where(A < 128, -1, B)" --type=Int16 --NoDataValue=-1

The --calc formula should replace all transparent-ish pixels with -1, which is a valid value for the output image because it's an Int16 image.

However it seems gdal_calc is doing the type conversion implicitly, after the output values are calculated. So the -1 gets wrapped into a byte, and the values all get set to 255 instead of -1.

This patch fixes the problem by explicitly converting the array's data type, earlier in the process: https://github.com/koordinates/gdal/commit/efea5dafaf536213faadbbc3f671591aa6aa8c30

Change History (1)

comment:1 by cdestigter, 9 years ago

Description: modified (diff)
Owner: changed from hobu to Robert Coup
Note: See TracTickets for help on using tickets.