#5868 closed enhancement (fixed)
[PATCH] Add Quartiles, Min & Max to gdalwarp
Reported by: | davisnn | Owned by: | warmerdam |
---|---|---|---|
Priority: | normal | Milestone: | 2.0.0 |
Component: | GDAL_Raster | Version: | 1.11.1 |
Severity: | normal | Keywords: | |
Cc: |
Description
In addition to the Average and Mode statistics it is helpful for several of my projects to be able to extract the quartiles, max and min when warping the data.
I have attached a patch to gdal-1.11.1 that adds this functionality to gdalwarp.
Attachments (6)
Change History (15)
by , 9 years ago
Attachment: | quartiles.patch added |
---|
comment:1 by , 9 years ago
Summary: | Add Quartiles, Min & Max to gdalwarp → [PATCH] Add Quartiles, Min & Max to gdalwarp |
---|
Several remarks :
- this should be rebased on GDAL trunk. Note that the value 7 of GDALResampleAlg is now reserved for another purpose
- the numeric values algo = 4, 5, 6 are not very intuitive. I know this has started like this, but adding more to it increases lack of readability. Symbolic names would be better
- initialization to dfTotal = 1e6; for the min algorithm is arbitrary. Better use DBL_MAX (or std::limits<double>::max or whatever works)
- same for initialization to dfTotal = 0. Use -DBL_MAX
- nCount2++; is useless in all places where it is added
- gdalwarp Usage() should be updated with -rmx, etc... but I found the new dedicated options a bit too specialized, so perhaps just remove them and keep the -r max syntax.
- apps/gdal_utilities.dox should be updated
- autotest/alg/warp.py should be updated to test the new algos
comment:2 by , 9 years ago
Thank you for the feedback. I have made the suggested changes, except for the changes to autotest (quartiles_r1.patch). I was not able to find much in the way of documentation of how to add a test to the autotest. Do I just need to duplicate what is being done for the mode cases in say warp_36?
Also I was not quite sure what you meant by symbolic names. I have added an enum to gdalwarper.h, that sets names to each of the algorithms, if this is not what you meant please let me know.
Thanks, Neil
follow-up: 4 comment:3 by , 9 years ago
- DBL_MIN is not appropriate. It is the smallest positive number something like 1e-XXXX. That should rather be -DBL_MAX, otherwise that will not work with negative only values
- There are remaining printf() statements.
- Yes, you can take example on the mode case.
by , 9 years ago
Attachment: | autotest.patch added |
---|
Patch to add tests for max, min, median, Quartile 1 and Quartile3 to autotest
comment:4 by , 9 years ago
I have removed the two errors you mentioned and added test cases for both signed and unsigned files to the autotest.
comment:5 by , 9 years ago
Hum, I get failed tests with the new test steps (on a x86_64 with 4.8.2 in both -00 and -02 mode), and the visual difference is indeed huge between what I get from the vrt with gdal_translate and the reference tif, not just a subtle issue with floating point rounding. Are you sure the reference tif are uptodate in your archive ?
TEST: warp_47 ... Diff at pixel (1, 0) : -9.000000 Diff at pixel (2, 0) : -33.000000 Diff at pixel (3, 0) : -50.000000 Diff at pixel (4, 0) : -33.000000 Diff at pixel (5, 0) : -9.000000 Diff at pixel (12, 0) : -24.000000 Diff at pixel (15, 0) : -16.000000 Diff at pixel (16, 0) : -8.000000 Diff at pixel (17, 0) : -16.000000 Diff at pixel (6, 2) : -66.000000 Diff at pixel (9, 2) : -99.000000 Diff at pixel (29, 7) : -123.000000 Diff at pixel (2, 12) : -132.000000 Max diff : 132 Number of diffs : 589 fail line 1540: Image too different from reference TEST: warp_48 ... Diff at pixel (0, 0) : 25.000000 Diff at pixel (1, 0) : 16.000000 Diff at pixel (5, 0) : 16.000000 Diff at pixel (6, 0) : 33.000000 Diff at pixel (7, 0) : 25.000000 Diff at pixel (8, 0) : 17.000000 Diff at pixel (9, 0) : 57.000000 Diff at pixel (10, 0) : 33.000000 Diff at pixel (11, 0) : 8.000000 Diff at pixel (13, 0) : 58.000000 Diff at pixel (8, 3) : 82.000000 Diff at pixel (28, 4) : 99.000000 Diff at pixel (4, 11) : 107.000000 Diff at pixel (2, 13) : 132.000000 Diff at pixel (2, 16) : 140.000000 Diff at pixel (27, 25) : 148.000000 Max diff : 148 Number of diffs : 615 fail line 1559: Image too different from reference TEST: warp_49 ... Diff at pixel (0, 0) : 17.000000 Diff at pixel (1, 0) : 8.000000 Diff at pixel (2, 0) : -17.000000 Diff at pixel (3, 0) : -17.000000 Diff at pixel (4, 0) : -25.000000 Diff at pixel (6, 0) : 25.000000 Diff at pixel (7, 0) : 9.000000 Diff at pixel (8, 0) : 9.000000 Diff at pixel (9, 0) : 8.000000 Diff at pixel (13, 0) : 33.000000 Diff at pixel (13, 1) : 49.000000 Diff at pixel (26, 4) : 57.000000 Diff at pixel (27, 4) : 74.000000 Diff at pixel (4, 11) : 107.000000 Diff at pixel (2, 16) : 140.000000 Max diff : 140 Number of diffs : 579 fail line 1578: Image too different from reference TEST: warp_50 ... Diff at pixel (0, 0) : 25.000000 Diff at pixel (1, 0) : 16.000000 Diff at pixel (5, 0) : 16.000000 Diff at pixel (6, 0) : 33.000000 Diff at pixel (7, 0) : 25.000000 Diff at pixel (8, 0) : 17.000000 Diff at pixel (9, 0) : 57.000000 Diff at pixel (10, 0) : 33.000000 Diff at pixel (11, 0) : 8.000000 Diff at pixel (13, 0) : 58.000000 Diff at pixel (8, 3) : 82.000000 Diff at pixel (28, 4) : 99.000000 Diff at pixel (4, 11) : 107.000000 Diff at pixel (2, 13) : 132.000000 Diff at pixel (2, 16) : 140.000000 Diff at pixel (27, 25) : 148.000000 Max diff : 148 Number of diffs : 615 fail line 1597: Image too different from reference TEST: warp_51 ... Diff at pixel (0, 0) : 9.000000 Diff at pixel (2, 0) : -17.000000 Diff at pixel (3, 0) : -25.000000 Diff at pixel (4, 0) : -25.000000 Diff at pixel (10, 0) : 8.000000 Diff at pixel (12, 0) : -16.000000 Diff at pixel (13, 0) : 25.000000 Diff at pixel (15, 0) : -16.000000 Diff at pixel (17, 0) : -8.000000 Diff at pixel (24, 0) : -33.000000 Diff at pixel (13, 1) : 41.000000 Diff at pixel (6, 2) : -57.000000 Diff at pixel (28, 4) : 58.000000 Diff at pixel (26, 6) : -99.000000 Diff at pixel (29, 7) : -123.000000 Diff at pixel (2, 12) : -132.000000 Max diff : 132 Number of diffs : 542 fail line 1616: Image too different from reference
comment:6 by , 9 years ago
The problem was actually with the VRT files. I have uploaded a new copy of the data, with the updated VRT files. This passes all of the tests.
by , 9 years ago
Attachment: | quantile_data_r1.tar.gz added |
---|
Data for autotest with updated VRT files for negative value tests
comment:7 by , 9 years ago
comment:8 by , 9 years ago
Milestone: | → 2.0 |
---|---|
Resolution: | → fixed |
Status: | new → closed |
Patch to add quartiles, min, and max to gdalwarp