Profile tool always casts to integers (incl. csv output)
|Reported by:||hamish||Owned by:|
|Keywords:||profile tool, precision||Cc:|
as reported on the grass-user mailing list yesterday, export to .csv always exports as integers, regardless of if the source data is CELL, FCELL, or DCELL.
Moritz prepared a patch:
--- profile.py 2011-08-19 13:28:17.000000000 +0200 +++ profile_new.py 2011-08-19 13:28:43.000000000 +0200 @@ -598,7 +598,7 @@ caption=_("Error"), style=wx.OK | wx.ICON_ERROR | wx.CENTRE) return False for datapair in r['datalist']: - file.write('%d,%d\n' % (float(datapair),float(datapair))) + file.write('%f,%f\n' % (float(datapair),float(datapair))) file.close()
but I think we need to go deeper than that, as
%d shows up all around the script, both in preparing the source points to feed into r.profile, and in processing the results of it.
- Rather importantly in lat/lon locations casting the tie points to int can silently return the wrong results, with coords shifted a long way from where you expected.
On the input side it will be easy enough to change
%.9g would be enough to cover sub-millimeter in lat/lon) for r.profile and r.what's coord input.
On the output site the distance along transect should be at least to the mm (
%.3g assuming the tool preserves r.profile's column 1 distance as meters), and test for map type, exporting column 2 as
%d for CELL,
%.7g for FCELL, and
%.15g for DCELL. (see #335)
another thing to keep in mind is that map_units is not always going to be feet, meters, or degrees. there's little reason it couldn't be microns or parsecs, so to avoid others being limited by our humble imaginations we should avoid unnecessary assumptions about that whenever possible.
ps- see also #1292 "Profile tool incorrectly processes no data values"