grid value shift (+256) for ArcSDE raster

[bartvde]

in ArcMap I see negative values, in GDAL all values seem to have a shift of 256, so -21 becomes 235. Nodata values are also off with 256, so -256 in ArcMap becomes 0 in GDAL.

The pixel type of your input is char (signed 8 bit). The no-data value is -256 (which cannot be represented by the signed 8 bit, btw).

For some reason your data gets converted from a signed 8 bit type to an unsigned 8 bit type.

Looking at the trunk version of sderasterband.cpp I see that GDT_BYTE is taken to be the equivalent of SE_PIXEL_TYPE_8BIT_S. If this means that an input of signed 8 bit value is stored in an unsigned 8 bit value, than that would explain what you are seeing. An alternative would be to update this code to return GDT_Int16, which will leave your data intact. Line 833.

This is very likely the culprit. Thanks Kor. Please file a bug on this and I will try to apply it before the release. I would note that I don't have SDE anymore to be able to test against, so I would have to depend on Bart's assessment of whether or not the issue is resolved.

Howard

[bartvde]

The change does not seem to work for me. Btw, the no data value is +256 instead of -256 as I reported before.

[Jukka Rahkonen]

It feels unlikely that this ticket will ever be solved without getting some more information. Frank, Bart and Hobu have obviously been able to analyze the problematic data but the description text is not detailed for the others to do anything. If Frank, Bart or Hoby won't react within reasonable time I will close this ticket as wontfix.

[Even Rouault]

Removing obsolete milestone

[Even Rouault]

This ticket has been automatically closed because Trac is no longer used for GDAL bug tracking, since the project has migrated to GitHub. If you believe this ticket is still valid, you may file it to ​https://github.com/OSGeo/gdal/issues if it is not already reported there.