== ossim-batch-test == ossim-batch-test is a command line test application that can execute programs, test the results against expected results, and log the output with differences between expected results and run times. ossim-batch-test requires a test configuration file for input and generates a log file on output. Multiple test can be in a single configuration file. A single test, all test or multiple test can be ran with a single ossim-batch-test command. All times are logged for each command. So you can easily test for changes in output and changes in output times. This application is written in c++ so it's portable on windows, linux, MACs. Typically to switch from one operating system to another the file paths and diff commands have to be query / replaced in config file. On linux, MAC typically "diff" is used. On windows "fc" works well. '''Example of ossim-orthoigen test on a NITF image with RPC model:''' For this test three things:[[BR]] 1) The height about the center of the test image. Because this image has a RPC sensor model which is affected by elevation, the first test is the height at the center of the image. If this changes the output bounds will also change.[[BR]] 2) The initial output of ossim-info -d -i -p of the source test image. This alone will test many changes in the code.[[BR]] 3) The orthoigen output. Test reprojecting image to an output file. Testing the histogram checks for pixel changes. Testing the ossim-info -d -i -p tests for geometry changes. From here on out commands start with $ and comments start with //. // This example was generated on linux. Windows example coming soon... // Some things to note. Test image is under OSSIM_TEST environment variable: {{{ $ env | grep OSSIM_TEST OSSIM_TEST=/data1/test $ pwd /data1/test $ ls source/GeoEye/GE1_Hobart_GeoStereo_NITF-NCD/ 5V090205M0001912264B220000100072M_001508507.his 5V090205M0001912264B220000100072M_001508507.ntf 5V090205M0001912264B220000100072M_001508507.ovr Single_Organization_license.txt }}} // Generate a configuration file template.[[BR]] {{{ $ ossim-batch-test -w orthoigen-test.kwl Wrote file: orthoigen-test.kwl $ }}} // Edit as required.[[BR]] // Contents of this file:[[BR]] $ cat orthoigen-test.kwl[[BR]] {{{ // --- // File: orthoigen-test.kwl // Generated by command: ossim-batch-test -w orthoigen-test.kwl // Description: ossim-batch-test config file. // --- // --- // Control flags: // Use 1 or true, 0 or false to turn on and off sections. // Typically preprocess and expected results only turned on for first run. // Clean commands should erase anything created by this test. // Command order if flag turned on: // 1) clean, 2) pre-process, 3) expected, 4) test, 5) post-process // --- // --- // NOTES: // 1) Use forward slash, slash(c++ comment) at beginning of line for comments. // 2) For environment variables use $(YOUR_VARIABLE). They will be expanded at // run time if valid. // 3) Clean, preprocess, run, and accept test sections can be switched on or off // at run time. These override config file flags. Optional arguments are: // --clean-test // --preprocess-test // --run-test // --accept-test // Multiple tests can be entered by quoting string of space separated // test, e.g. "test1 test2". To accept all tests use "all" for test. // --- // Where you want the log files to go: log_directory: $(OSSIM_TEST)/logs/osgeo // --- // Begin test1: // --- test1.name: test1 test1.description: Test height for the center of the image for test2 and test3. // Controls/turns on/off whole test (all sections): test1.enabled: 1 // Individual control flags: test1.run_clean_commands: 0 test1.run_preprocessing_commands: 0 test1.run_expected_results_commands: 0 test1.run_test_commands: 1 test1.run_postprocessing_commands: 1 // Temp file to catch diff output. test1.temp_file: /tmp/tmp.txt // Clean up commands. test1.clean_command0: rm -rf $(OSSIM_TEST)/exp/orthoigen-test/test1 test1.clean_command1: rm -rf $(OSSIM_TEST)/out/orthoigen-test/test1 // Pre-process commands: test1.preprocess_command0: mkdir -p $(OSSIM_TEST)/exp/orthoigen-test/test1 test1.preprocess_command1: mkdir -p $(OSSIM_TEST)/out/orthoigen-test/test1 test1.preprocess_command1: mkdir -p $(OSSIM_TEST)/logs/osgeo // Commands to generate expected results: // Since test2 and test3 are dependent on elevation test the center of the image. test1.expected_results_command0: ossim-info --height -42.8508 147.2537 > $(OSSIM_TEST)/exp/orthoigen-test/test1/height.txt // The actual commands to test: test1.test_command0: ossim-info --height -42.8508 147.2537 > $(OSSIM_TEST)/out/orthoigen-test/test1/height.txt // Post process commands for diffs and stuff: test1.postprocess_command0: diff -w $(OSSIM_TEST)/exp/orthoigen-test/test1/height.txt $(OSSIM_TEST)/out/orthoigen-test/test1/height.txt // --- // End test1: // --- // --- // Begin test2: // --- test2.name: test2 test2.description: Test geometry for GeoEye NITF with RPC model. // Controls/turns on/off whole test (all sections): test2.enabled: 1 // Individual control flags: test2.run_clean_commands: 0 test2.run_preprocessing_commands: 0 test2.run_expected_results_commands: 0 test2.run_test_commands: 1 test2.run_postprocessing_commands: 1 // Temp file to catch diff output. test2.temp_file: /tmp/tmp.txt // Clean up commands. test2.clean_command0: rm -rf $(OSSIM_TEST)/exp/orthoigen-test/test2 test2.clean_command1: rm -rf $(OSSIM_TEST)/out/orthoigen-test/test2 // Pre-process commands: test2.preprocess_command0: mkdir -p $(OSSIM_TEST)/exp/orthoigen-test/test2 test2.preprocess_command1: mkdir -p $(OSSIM_TEST)/out/orthoigen-test/test2 test2.preprocess_command1: mkdir -p $(OSSIM_TEST)/logs/osgeo // Commands to generate expected results: test2.expected_results_command0: ossim-info -d -i -p $(OSSIM_TEST)/source/GeoEye/GE1_Hobart_GeoStereo_NITF-NCD/5V090205M0001912264B220000100072M_001508507.ntf > $(OSSIM_TEST)/exp/orthoigen-test/test2/info.txt // The actual commands to test: test2.test_command0: ossim-info -d -i -p $(OSSIM_TEST)/source/GeoEye/GE1_Hobart_GeoStereo_NITF-NCD/5V090205M0001912264B220000100072M_001508507.ntf > $(OSSIM_TEST)/out/orthoigen-test/test2/info.txt // Post process commands for diffs and stuff: test2.postprocess_command0: diff -w $(OSSIM_TEST)/exp/orthoigen-test/test2/info.txt $(OSSIM_TEST)/out/orthoigen-test/test2/info.txt // --- // End test2: // --- // --- // Begin test3: // --- test3.name: test3 test3.description: Test orthorectification of a GeoEye NITF with RPC model. // Controls/turns on/off whole test (all sections): test3.enabled: 1 // Individual control flags: test3.run_clean_commands: 0 test3.run_preprocessing_commands: 0 test3.run_expected_results_commands: 0 test3.run_test_commands: 1 test3.run_postprocessing_commands: 1 // Temp file to catch diff output. test3.temp_file: /tmp/tmp.txt // Clean up commands. test3.clean_command0: rm -rf $(OSSIM_TEST)/exp/orthoigen-test/test3 test3.clean_command1: rm -rf $(OSSIM_TEST)/out/orthoigen-test/test3 // Pre-process commands: test3.preprocess_command0: mkdir -p $(OSSIM_TEST)/exp/orthoigen-test/test3 test3.preprocess_command1: mkdir -p $(OSSIM_TEST)/out/orthoigen-test/test3 test3.preprocess_command1: mkdir -p $(OSSIM_TEST)/logs/osgeo // Commands to generate expected results: // Put the output in the "out" directory as the "exp" may be under svn for small things: // Ortho: test3.expected_results_command0: ossim-orthoigen -t 1024 --geo-scaled -42.85 --resample-type bilinear $(OSSIM_TEST)/source/GeoEye/GE1_Hobart_GeoStereo_NITF-NCD/5V090205M0001912264B220000100072M_001508507.ntf $(OSSIM_TEST)/out/orthoigen-test/test3/t1.tif // Histogram: test3.expected_results_command1: ossim-img2rr -r --create-histogram $(OSSIM_TEST)/out/orthoigen-test/test3/t1.tif // Geometry: test3.expected_results_command2: ossim-info -d -i -p $(OSSIM_TEST)/out/orthoigen-test/test3/t1.tif > $(OSSIM_TEST)/out/orthoigen-test/test3/t1-info.txt // Copy to exp directory which is under svn control: test3.expected_results_command3: cp $(OSSIM_TEST)/out/orthoigen-test/test3/t1.his $(OSSIM_TEST)/exp/orthoigen-test/test3/t1.his test3.expected_results_command4: cp $(OSSIM_TEST)/out/orthoigen-test/test3/t1-info.txt $(OSSIM_TEST)/exp/orthoigen-test/test3/t1-info.txt // The actual commands to test: // Ortho: test3.test_command0: ossim-orthoigen -t 1024 --geo-scaled -42.85 --resample-type bilinear $(OSSIM_TEST)/source/GeoEye/GE1_Hobart_GeoStereo_NITF-NCD/5V090205M0001912264B220000100072M_001508507.ntf $(OSSIM_TEST)/out/orthoigen-test/test3/t2.tif // Histogram: test3.test_command1: ossim-img2rr -r --create-histogram $(OSSIM_TEST)/out/orthoigen-test/test3/t2.tif // Geometry: test3.test_command2: ossim-info -d -i -p $(OSSIM_TEST)/out/orthoigen-test/test3/t2.tif > $(OSSIM_TEST)/out/orthoigen-test/test3/t2-info.txt // Post process commands for diffs and stuff: // Histogram: test3.postprocess_command0: diff -w $(OSSIM_TEST)/out/orthoigen-test/test3/t1.his $(OSSIM_TEST)/out/orthoigen-test/test3/t2.his // Geometry: test3.postprocess_command1: diff -w $(OSSIM_TEST)/out/orthoigen-test/test3/t1-info.txt $(OSSIM_TEST)/out/orthoigen-test/test3/t2-info.txt // --- // End test3: // --- }}} // ---[[BR]] // First time through run with the --accept-test all option.[[BR]] // Note this will run the preprocess commands that make directories and so on.[[BR]] // This will populate the expected results section.[[BR]] // ---[[BR]] {{{ $ ossim-batch-test --accept-test all orthoigen-test.kwl Logging to file: /data1/test/logs/osgeo/ossim-batch-test-log-20111216105557.txt 100% 100% Time elapsed : 12.510 MPI running with 1 processors... Creating overviews for file: /data1/test/out/orthoigen-test/test3/t1.tif creating r1... 100% creating r2... 100% creating r3... 100% creating r4... 100% creating r5... 100% creating r6... 100% creating r7... 100% Finished... Elapsed time: 0 Wrote: /data1/test/logs/osgeo/ossim-batch-test-log-20111216105557.txt }}} // Now test. A run with no options runs all the test in the configuration file.[[BR]] {{{ $ ossim-batch-test orthoigen-test.kwl Logging to file: /data1/test/logs/osgeo/ossim-batch-test-log-20111216105716.txt test1[0]: PASSED test2[0]: PASSED 100% 100% Time elapsed : 9.574 MPI running with 1 processors... Creating overviews for file: /data1/test/out/orthoigen-test/test3/t2.tif creating r1... 100% creating r2... 100% creating r3... 100% creating r4... 100% creating r5... 100% creating r6... 100% creating r7... 100% Finished... Elapsed time: 1 test3[0]: PASSED test3[1]: PASSED Wrote: /data1/test/logs/osgeo/ossim-batch-test-log-20111216105716.txt }}} // Contents of log file:[[BR]] {{{ // --- // ossim-batch-test log: // date format = yyyymmddhhmmss //--- start_time: 20111216105716 config_file: /data1/test/orthoigen-test.kwl ---------------------------------------------------------------------- processing_test: test1 description: Test height for the center of the image for test2 and test3. preProcessFlag: 0 expectedFlag: 0 testFlag: 1 postProcessFlag: 1 cleanFlag: 0 executing command: ossim-info --height -42.8508 147.2537 > /data1/test/out/orthoigen-test/test1/height.txt begin: 20111216105716 end: 20111216105716 test1[0]: elapsed time in seconds: 0.2918 return status: 0 executing command: diff -w /data1/test/exp/orthoigen-test/test1/height.txt /data1/test/out/orthoigen-test/test1/height.txt > /tmp/tmp.txt return status: 0 test1[0]: PASSED end_test: test1 ---------------------------------------------------------------------- ---------------------------------------------------------------------- processing_test: test2 description: Test geometry for GeoEye NITF with RPC model. preProcessFlag: 0 expectedFlag: 0 testFlag: 1 postProcessFlag: 1 cleanFlag: 0 executing command: ossim-info -d -i -p /data1/test/source/GeoEye/GE1_Hobart_GeoStereo_NITF-NCD/5V090205M0001912264B220000100072M_001508507.ntf > /data1/test/out/orthoigen-test/test2/info.txt begin: 20111216105716 end: 20111216105717 test2[0]: elapsed time in seconds: 0.3250 return status: 0 executing command: diff -w /data1/test/exp/orthoigen-test/test2/info.txt /data1/test/out/orthoigen-test/test2/info.txt > /tmp/tmp.txt return status: 0 test2[0]: PASSED end_test: test2 ---------------------------------------------------------------------- ---------------------------------------------------------------------- processing_test: test3 description: Test orthorectification of a GeoEye NITF with RPC model. preProcessFlag: 0 expectedFlag: 0 testFlag: 1 postProcessFlag: 1 cleanFlag: 0 executing command: ossim-orthoigen -t 1024 --geo-scaled -42.85 --resample-type bilinear /data1/test/source/GeoEye/GE1_Hobart_GeoStereo_NITF-NCD/5V090205M0001912264B220000100072M_001508507.ntf /data1/test/out/orthoigen-test/test3/t2.tif begin: 20111216105717 end: 20111216105726 test3[0]: elapsed time in seconds: 9.6004 return status: 0 executing command: ossim-img2rr -r --create-histogram /data1/test/out/orthoigen-test/test3/t2.tif begin: 20111216105726 end: 20111216105727 test3[1]: elapsed time in seconds: 0.7058 return status: 0 executing command: ossim-info -d -i -p /data1/test/out/orthoigen-test/test3/t2.tif > /data1/test/out/orthoigen-test/test3/t2-info.txt begin: 20111216105727 end: 20111216105727 test3[2]: elapsed time in seconds: 0.3115 return status: 0 executing command: diff -w /data1/test/out/orthoigen-test/test3/t1.his /data1/test/out/orthoigen-test/test3/t2.his > /tmp/tmp.txt return status: 0 test3[0]: PASSED executing command: diff -w /data1/test/out/orthoigen-test/test3/t1-info.txt /data1/test/out/orthoigen-test/test3/t2-info.txt > /tmp/tmp.txt return status: 0 test3[1]: PASSED end_test: test3 ---------------------------------------------------------------------- stop_time: 20111216105727 total elapsed time in seconds: 11.2535 }}} '''Failure Example''' // The expected results were edited to produce a failure, i.e. tie point shift:[[BR]] // Run the test:[[BR]] {{{ $ ossim-batch-test orthoigen-test.kwl Logging to file: /data1/test/logs/osgeo/ossim-batch-test-log-20111216110702.txt test1[0]: PASSED test2[0]: PASSED 100% 100% Time elapsed : 10.525 MPI running with 1 processors... Creating overviews for file: /data1/test/out/orthoigen-test/test3/t2.tif creating r1... 100% creating r2... 100% creating r3... 100% creating r4... 100% creating r5... 100% creating r6... 100% creating r7... 100% Finished... Elapsed time: 1 test3[0]: PASSED test3[1]: FAILED Wrote: /data1/test/logs/osgeo/ossim-batch-test-log-20111216110702.txt }}} // Contents of log file:[[BR]] {{{ // --- // ossim-batch-test log: // date format = yyyymmddhhmmss //--- start_time: 20111216110702 config_file: /data1/test/orthoigen-test.kwl ---------------------------------------------------------------------- processing_test: test1 description: Test height for the center of the image for test2 and test3. preProcessFlag: 0 expectedFlag: 0 testFlag: 1 postProcessFlag: 1 cleanFlag: 0 executing command: ossim-info --height -42.8508 147.2537 > /data1/test/out/orthoigen-test/test1/height.txt begin: 20111216110702 end: 20111216110703 test1[0]: elapsed time in seconds: 0.2289 return status: 0 executing command: diff -w /data1/test/exp/orthoigen-test/test1/height.txt /data1/test/out/orthoigen-test/test1/height.txt > /tmp/tmp.txt return status: 0 test1[0]: PASSED end_test: test1 ---------------------------------------------------------------------- ---------------------------------------------------------------------- processing_test: test2 description: Test geometry for GeoEye NITF with RPC model. preProcessFlag: 0 expectedFlag: 0 testFlag: 1 postProcessFlag: 1 cleanFlag: 0 executing command: ossim-info -d -i -p /data1/test/source/GeoEye/GE1_Hobart_GeoStereo_NITF-NCD/5V090205M0001912264B220000100072M_001508507.ntf > /data1/test/out/orthoigen-test/test2/info.txt begin: 20111216110703 end: 20111216110703 test2[0]: elapsed time in seconds: 0.2582 return status: 0 executing command: diff -w /data1/test/exp/orthoigen-test/test2/info.txt /data1/test/out/orthoigen-test/test2/info.txt > /tmp/tmp.txt return status: 0 test2[0]: PASSED end_test: test2 ---------------------------------------------------------------------- ---------------------------------------------------------------------- processing_test: test3 description: Test orthorectification of a GeoEye NITF with RPC model. preProcessFlag: 0 expectedFlag: 0 testFlag: 1 postProcessFlag: 1 cleanFlag: 0 executing command: ossim-orthoigen -t 1024 --geo-scaled -42.85 --resample-type bilinear /data1/test/source/GeoEye/GE1_Hobart_GeoStereo_NITF-NCD/5V090205M0001912264B220000100072M_001508507.ntf /data1/test/out/orthoigen-test/test3/t2.tif begin: 20111216110703 end: 20111216110713 test3[0]: elapsed time in seconds: 10.5560 return status: 0 executing command: ossim-img2rr -r --create-histogram /data1/test/out/orthoigen-test/test3/t2.tif begin: 20111216110713 end: 20111216110714 test3[1]: elapsed time in seconds: 0.5340 return status: 0 executing command: ossim-info -d -i -p /data1/test/out/orthoigen-test/test3/t2.tif > /data1/test/out/orthoigen-test/test3/t2-info.txt begin: 20111216110714 end: 20111216110714 test3[2]: elapsed time in seconds: 0.2372 return status: 0 executing command: diff -w /data1/test/out/orthoigen-test/test3/t1.his /data1/test/out/orthoigen-test/test3/t2.his > /tmp/tmp.txt return status: 0 test3[0]: PASSED executing command: diff -w /data1/test/out/orthoigen-test/test3/t1-info.txt /data1/test/out/orthoigen-test/test3/t2-info.txt > /tmp/tmp.txt return status: 256 test3[1]: FAILED diff -w /data1/test/out/orthoigen-test/test3/t1-info.txt /data1/test/out/orthoigen-test/test3/t2-info.txt > /tmp/tmp.txt output follows: 83c83 < image0.projection.tie_point_xy: (147.1657,-42.7826) --- > image0.projection.tie_point_xy: (147.165760750085,-42.7826084701764) end_test: test3 ---------------------------------------------------------------------- stop_time: 20111216110714 total elapsed time in seconds: 11.8294 }}} // Accept the new tie point as valid:[[BR]] {{{ $ ossim-batch-test --accept-test test3 orthoigen-test.kwlLogging to file: /data1/test/logs/osgeo/ossim-batch-test-log-20111216111011.txt test1: disabled test2: disabled 100% 100% Time elapsed : 12.526 MPI running with 1 processors... Creating overviews for file: /data1/test/out/orthoigen-test/test3/t1.tif creating r1... 100% creating r2... 100% creating r3... 100% creating r4... 100% creating r5... 100% creating r6... 100% creating r7... 100% Finished... Elapsed time: 0 Wrote: /data1/test/logs/osgeo/ossim-batch-test-log-20111216111011.txt }}} // Re-run test3:[[BR]] {{{ $ ossim-batch-test --run-test test3 orthoigen-test.kwl Logging to file: /data1/test/logs/osgeo/ossim-batch-test-log-20111216111102.txt test1: disabled test2: disabled 100% 100% Time elapsed : 12.504 MPI running with 1 processors... Creating overviews for file: /data1/test/out/orthoigen-test/test3/t2.tif creating r1... 100% creating r2... 100% creating r3... 100% creating r4... 100% creating r5... 100% creating r6... 100% creating r7... 100% Finished... Elapsed time: 1 test3[0]: PASSED test3[1]: PASSED Wrote: /data1/test/logs/osgeo/ossim-batch-test-log-20111216111102.txt }}}