GSoC 2014 Proposal:
​http://www.google-melange.com/gsoc/proposal/public/google/gsoc2014/martidi/5629499534213120

Project Git repository:
​https://github.com/martidi/OSSIM-GSoC-2014.git

Weekly Report 1 (2014, May 23)

Bonding period:

I have configured and built OSSIM in Ubuntu 12.04 OS and tried to built it also in Mac OS.

I have configured, compiled and installed the OpenCV library.

In accordance with mentors, I have decided the programming language.

I have read the documentation and I have begun to familiarize with some OSSIM plugins.

Week 1:

WHAT I GET DONE THIS WEEK:

I have prepared the wiki page (​http://trac.osgeo.org/ossim/wiki/GSoC_2014) and the Git repository (​https://github.com/martidi/OSSIM-GSoC-2014.git) for code hosting.

I have begun to setting up the ossim preferences file, in particular for the ossimopencv_plugin loading.

In accordance with mentors I have begun to see how OSSIM Geocell and the OpenCV library can integrate: I have compiled and installed the ossimopencv_plugin and loaded it in the OSSIM core library in order to analyze how OpenCV is so far used for automatic Tie Points extraction.

I have begun to analyze the ossimRpcModel class.

WHAT I PLAN ON DOING NEXT WEEK:

To solve the problem on which I am stuck.

To complete the ossimRpcModel class analysis.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

I have had some problems with satellite imagery loading: it seems that there are some issues for some particular images (e.g. images taken from IKONOS2 and Geoeye) with the ossimRpcModel class; as mentors pointed out this is probably due for some settings in the preferences file.

Weekly Report 2 (2014, May 30)

Week 2:

WHAT I GET DONE THIS WEEK:

I have created the Eclipse project for OSSIM to manage all the source code.

I have set up the ossim preferences file with the elevation information to try to solve the image loading problems and in order to prepare the proper files to begin using the orthorectification tool (ossim-orthoigen).

I have studied the code of the ossim-orthoigen tool and of the associated class ossimOrthoIGen in order to understand their implementation.

I have begun to write the ossim-epipolar app that is based on ossim-orthoigen tool and on the ossim OpenCv plugin. The algorithm for the quasi-epipolar images generation consists of two main steps: first it is necessary to project the raw images on an horizontal plane at a median height of the area of interest and subsequently it is required to search automatically Tie points to minimize the residual parallax error between the two images.

WHAT I PLAN ON DOING NEXT WEEK:

To solve the problem on which I am stuck.

To continue the ossim-epipolar tool implementation.

To verify and assess the automatic Tie Points collection.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

Despite I have worked on the preferences file and on image histogram remapping I still have problems in loading some particular images (e.g. images taken from IKONOS2 and Geoeye).

Weekly Report 3 (2014, June 06)

Week 3:

WHAT I GET DONE THIS WEEK:

As a result of a discussion with mentors I have discarded the ossim-orthoigen tool as a reference for my ossim-epipolar app and turned towards ossim-chipper, a newer and clearer one.

I have begun to write the ossim-epipolar app editing the ossimSrtmSupportData class, in order to compute the median height and the standard deviation of SRTM values, parameters that are required for the automatic generation of the quasi-epipolar images.

I have upload the drafted ossim-epipolar app and the related code modified (ossimSrtmSupportData) on the repository (​https://github.com/martidi/OSSIM-GSoC-2014).

WHAT I PLAN ON DOING NEXT WEEK:

To solve the problem on which I am stuck.

To continue working on epipolar images generation

To assess the automatic Tie Points identification.

To begin the implementation of the OpenCV SIFT procedure for automatic TPs generation

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

I'm trying to use an ASCII DEM file as input for the elevation data, but I have had some issues with it.

Weekly Report 4 (2014, June 13)

Week 4:

WHAT I GET DONE THIS WEEK:

Considering that, as well known the epipolar rectification is a complex topic, I have deeply studied how to implement it (goal of the mid-term). As a result of a discussion with mentors, I have also evaluate alternative approaches for the DSM extraction, such as Vertical Line Locus feature-based type approach, Vertical Line Locus based on a coarse external DSM (such as SRTM or ASTER) and integration of Disparity Map and Non-Disparity Map approaches.

As a result of a discussion with mentors, I have discarded the idea of using Open-CV SIFT procedure for automatic TPs generation since it cannot be included or used as part of a public distribution, and I have chosen another free Open Source detector (Orb-Feature-Detector). Therefore I have wrote a procedure for the automatic TPs detection with Orb-Feature-Detector using the implementation available in the Open-CV library and I have tested it on an Ortho Ready Geo-Eye stereo-pair: I have calculated the x and y parallax errors, and I have seen that these images are basically quasi-epipolar as already projected by provider on a constant ellipsoidical height.

Furthermore I have wrote a procedure for the implementation of the Semi Global Matching algorithm and I have tested it on the above-mentioned images generating the following Disparity Map.

I have upload the code produced on the Git repository (​https://github.com/martidi/OSSIM-GSoC-2014).

WHAT I PLAN ON DOING NEXT WEEK:

To solve the problem on which I am stuck.

To continue working on epipolar images generation and Disparity Map creation.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

I am having some issues in the integration between OpenCV algorithms and OSSIM Apps: I have found some compiling problems using some Open-CV basic class (i.e. cv::Mat, cv::imread, etc.). Probably I have not configured rightly the Cmake-List-File.

Weekly Report 5 (2014, June 20)

Week 5:

WHAT I GET DONE THIS WEEK:

As a result of a talk with mentors, I have changed the procedure I have thought for the orthorectification on a plane: instead of orthorectify on a plane using the OSSIM orthorectification class, I have thought to impose a fixed height and use it when SRTM data is not available. For this reason I have set up in the preference file a default height above ellipsoid, for use when no elevation data is present. In this way raw images are projected automatically at this height.

I have created the ortho function (ortho.cpp and ortho.h) and I have implemented it in the ossim-epipolar app that I'm designing, in order to be able to call the function easier for e.g. ortorectify more than an image per time.

I have detailed the procedure I will implement for DSM generation.

I have continued improving the procedure for the implementation of the Semi Global Matching algorithm: for every raw image I have computed the SGM in i and j directions both in negative and positive, so to obtain 4 Disparity Maps to compare. What I want to do is to mediate the 4 different DSMs, that it is possible to achieve from the Disparity Maps, in order to obtain a more robust result.

I have drafted the mid-term evaluation report.

WHAT I PLAN ON DOING NEXT WEEK:

To solve the problem on which I am stuck.

To finalize the mid-term evaluation report.

To continue working on epipolar images generation and to improve Disparity Map creation.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

I am having some issues with automatic TPs collection and orthorectification on particular imagery I am testing.

Weekly Report 6 (2014, June 27)

Week 6:

WHAT I GET DONE THIS WEEK:

Generally speaking, I've done some tests in order to find a better solution for the quasi-epipolar image generation.

I have solved the issue I had with automatic TPs collection and orthorectification on particular morphologically relevant imagery I am testing.

I have continued to improve the Semi Global Matching procedure implemented, such in a way that it could be used for any kind of images; for this reason I have used a stereo pair with a relevant mountain morphology in which it is achieved an height difference bigger than 1500 m. As expected, it results that a simple shift it is not sufficient to achieve an epipolar geometry and, following the paper by Wang et al. about epipolar resampling, I am implementing the transformation necessary for bring the projected images (actually on a constant ellipsoidical height) in the epipolar geometry.

I have solved some little issues related to SGM implementation (Disparity Map visualization and shift refinement) and I have improved the Disparity Map resulting from the same GeoEye stereo pair used for the first tests (see attached).

WHAT I PLAN ON DOING NEXT WEEK:

To solve the problem on which I am stuck.

To continue working on epipolar images generation and to integrate the developed code in an ossim app.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

I’m having some difficulties in finding the correct image transformation for epipolar generation. I have seen that using the affine transformation (estimated with the opencv api) the disparity between the objects is not well preserved. Actually, I think that I found a possible solution estimating a roto-translation without scale. (​https://github.com/martidi/OSSIM-GSoC-2014_draft/tree/master/TEST_AFFINE).

Weekly Report 7 (2014, July 04)

Week 7:

WHAT I GET DONE THIS WEEK:

I have solved the issue I had with OpenCV and OSSIM integration: I have created a class in which I have defined some OpenCV functions and I have included it in the plugIn generated. Then I have linked the library from the class to the plugIn above mentioned [target_link_libraries (ossim-opencv ${requiredLibs} ossimopencv_plugin)]; in this way I am able to use OpenCV algorithms inside the plugIn.

I have continued to improve both Epipolar image generation and the Semi Global Matching algorithm implementation.

WHAT I PLAN ON DOING NEXT WEEK:

To solve the problem on which I am stuck.

To continue working on epipolar images generation and to refine the SGM algorithm implementation.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

I’m keeping on having some difficulties in finding the correct image transformation for epipolar generation. I have seen that using the affine transformation (estimated with the OpenCV api) the disparity between the objects is not well preserved.

TIMELINE:

Below is detailed a reviewed timeline, changed in agreement with the mentoring organization.

Timeline has slightly changed with respect to the foreseen, since I'm having some problems with epipolar image generation; as a matter of fact I’m having some difficulties in finding the correct image transformation for epipolar generation. I have seen that using the affine transformation (estimated with the OpenCV api) the disparity between the objects is not well preserved. Furthermore as regards to the Semi Global Matching algorithm, it is scheduled just a refinement since the algorithm for Disparity Map creation has been already tested.

Documentation and Code report:

• Documenting work advancements every week on wiki page (this include code documentation)

• Updating repository regularly

• Weekly reporting about the task accomplished and what I plan doing next week

June 28 - July 25:

• Refine Epipolar image generation

• Refine Semi Global Matching algorithm implementation

• Test and control implementation reliability and efficacy - discuss results with mentor(s)

July 26 - August 8:

• Implement the conversion from the disparity map into a geocoded DSM

• Implement interpolation algorithm for areas not previously matched

• Test and control implementation efficacy - discuss results with mentor(s)

• Handle unpredictable delays

August 11 - August 18:

• Testing and debugging code – Bugs fixing and code review

• Refine documentation - exchange opinions with mentor(s)

AUGUST 22th FINAL EVALUATION DEADLINE

Weekly Report 8 (2014, July 11)

Week 8:

WHAT I GET DONE THIS WEEK:

I have created the OpenCVtestclass, in which I have begun to put all the OpenCV functions I need for my ossim-epipolar plugin (​https://github.com/martidi/OSSIM-GSoC-2014/tree/master/plugIn).

I have continued to work on the epipolar-plugIn in order to unify the orthophoto generation process by the ossim-chipper tool with the subsequent TPs automatic extraction and the ulterior phase for the epipolar images generation, for which I will use the OpenCVtestclass.

I have continued to improve both Epipolar image generation and the Semi Global Matching algorithm implementation.

WHAT I PLAN ON DOING NEXT WEEK:

To solve the problem on which I am stuck.

To continue working on the ossim-epipolar plugIn.

To refine the epipolar images generation.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

I'm still working on the theoretical problem for finding the correct image transformation for epipolar images generation.

Weekly Report 9 (2014, July 18)

Week 9:

WHAT I GET DONE THIS WEEK:

I have improved the OpenCVtestclass, in which I'm creating the various methods I need for my ossim-epipolar plugin (​https://github.com/martidi/OSSIM-GSoC-2014/tree/master/OSSIM_DEV_HOME/ossim_plugins/opencv).

I have continued to work on the epipolar-plugIn in order to unify the orthophoto generation process by the ossim-chipper tool with the subsequent TPs automatic extraction and the ulterior phase for the epipolar images generation, for which I will use the OpenCVtestclass.

WHAT I PLAN ON DOING NEXT WEEK:

To solve the problem on which I am stuck.

To create and implement the Disparity Map class for disparity map generation.

To complete the epipolar images generation plugIn.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

I'm still working on the theoretical problem for finding the correct image transformation for epipolar images generation.

Weekly Report 10 (2014, July 25)

Week 10:

WHAT I GET DONE THIS WEEK:

I have found and implemented a new image transformation model for epipolar generation: I have realized that nor an affine transformation neither a pure rototranslation without scale were sufficient to find a good epipolarity. Therefore this week I wrote a rigorous model to estimate a transformation on the generated TPs, which is able to return images with null parallax error and whatever disparity on the single TP. It still remains some little issue that seems to disappear once outliers in TPs are scratched out.

I have modified the code in order to be able to use a 16bit .tif image (and not a compressed one) and not to loose the radiometric information of the raw images: as a matter of fact I noticed that by deteriorating the radiometric information the SGM algorithm solution was compromised, and therefore tha Disparity Map was not properly generated.

I have solved some minor warning in the ossim-opencv app building, related to “deprecated conversion” and “unappropriate comparison” in the code.

I have created two classes:

• TP generator

• Disparity Map

and implemented it into the app. To do it I have reorganized the app structure.

I have implemented the Disparity Map class for disparity map generation.

I have realized that one of the major problem in finding the correct image transformation for epipolar images generation, was due to the wrong median computation in Boost library accumulators, that I have used to implement the transformation computation: this fact caused me a lot of misunderstanding in results interpretation. I'm implementing a new median algorithm.

WHAT I PLAN ON DOING NEXT WEEK:

To solve the problem on which I am stuck.

To test the new rigorous model for epipolar generation and upload the mathematical model.

To begin thinking how to implement the conversion from the disparity map into a geocoded DSM.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

---

Weekly Report 11 (2014, August 1)

Week 11:

WHAT I GET DONE THIS WEEK:

I have tested the rigorous model (see attached) for epipolar generation on various type of images.

I have solved the issue related to outliers in TPs introducing a 3 sigma test, a well-known statistical rule used to eliminate outliers in a population.

WHAT I PLAN ON DOING NEXT WEEK:

To solve the problem on which I am stuck.

To modify some SGM thresholds in order to adapt the algorithm on the basis of % of TPs found.

To continue thinking how to implement the conversion from the disparity map into a geocoded DSM.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

---

Weekly Report 12 (2014, August 8)

Week 12:

WHAT I GET DONE THIS WEEK:

Beside to the 3 sigma test, I have implemented a preliminary screening for the points found with the Orb Feature Detector: the filter selects the better 2% of TPs which have the highest values of Brief descriptors.

I have tested the 3 sigma test implemented last week and, as can be seen from the images attached, it works well for outliers removal.

I have solved the “radiometric” issue due to the conversion from raw images to opencv images used for the Disparity Map creation. I have noticed that different image equalization strongly change the SGM result.

Moreover, I have improved the Disparity Map generated (see attached) by manually refining the SGM parameters (i.e. ndisparities and minDisparities values).

I have uploaded the updated code in the repository (​https://github.com/martidi/OSSIM-GSoC-2014).

WHAT I PLAN ON DOING NEXT WEEK:

To investigate the influence that the change of the SGM parameters have on the quality of the Disparity Map generated.

To automate the correct search of SGM parameters in relation to the different input images.

To work on the conversion from the disparity map into a geocoded DSM.

To begin "scrubbing" the code for the final version.

To begin writing the documentation on how to compile and test the program.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

---

Weekly Report 13 (2014, August 15)

Week 13:

WHAT I GET DONE THIS WEEK:

I have eventually finished my plug-in!

I have selected the optimized parameters for the SGM algorithm in order to generate a good-quality Disparity Map.

I have added the possibility to orthorectify the images using SRTM data instead of projecting on an horizontal plane.

I have implemented the conversion from the Disparity Map into a geocoded DSM: the “local” Disparity Map obtained from the Disparity Map class is converted from pixels to meters, then summed up to the height of the surface used for the orthorectification. Finally I have geocoded the DSM obtained in a standard geotiff format, thanks to the ossim Image File Writer function.

I have begun “scrubbing” the code for the final version.

I have begun writing the final documentation that will be finished before the pencil-down day.

I have uploaded the updated code and the compiling instruction in the repository (​https://github.com/martidi/OSSIM-GSoC-2014).

WHAT I PLAN ON DOING IN THE NEXT FUTURE:

As a possible development of this project, it could be feasible to refine the DSM extraction procedure adopting a multi-resolution pyramidal approach. I would like to develop an algorithm based on a coarse-to-fine structure: it can be adopted to obtain several Disparity Map at different resolutions that can be combined using a wavelet transformation in order to refine DSMs generation. This kind of strategy have particular relevance for DSM extraction in areas characterized by complex terrain morphology (i.e. dense urban areas, mountain regions).

Final Report (2014, August 18)

I have finished the code "scrubbing".

At the following link can be found the final documentation of the Ossim developed PlugIn for the DSM extraction.

​https://www.dropbox.com/s/cg2xgubnowheh9e/GSoC_DiRita_FinalDocumentation.pdf?dl=0