GSoC 2015 Proposal:
​https://www.google-melange.com/gsoc/proposal/review/student/google/gsoc2015/martidi/5717271485874176

Project Git repository:
​https://github.com/martidi/opencv_dsm

Weekly Report 1 (2015, May 31)

WHAT I GET DONE THIS WEEK:

I have prepared the wiki page (​http://trac.osgeo.org/ossim/wiki/GSoC_2015) and the Git repository (​https://github.com/martidi/opencv_dsm) for code hosting.

I have scrubbed the Plug In developed last year in order to facilitate the deployment of the new funcionalities that I have planned to implement.

I have configured a new IDE for a better management of the source code (Qt Creator).

I have worked (and I am still working) on a new version of matching for epipolar imagery generation, since I have noticed that the old one had some effectiveness problem in finding good Tie Points: I am switching from a SIFT-oriented matching to a Area Based one.

I have deployed an images preprocessing Wallis filter (Baltsavias version) to enhance the local contrast throughout the image, while reducing the overall contrast between bright and dark areas, in order to improve the subsequent dense matching.

WHAT I PLAN ON DOING NEXT WEEK:

To continue working on the automatic TPs generation.

To implement tri-stereo projection.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

At the moment I am not blocked on anything.

Weekly Report 2 (2015, June 07)

WHAT I GET DONE THIS WEEK:

I have worked (and I am still working) on a new version of matching for epipolar imagery generation, trying to implement a new and effective way able to generate epipolar images.

I have begun to implement a tri-stereo projection for triplet input images.

WHAT I PLAN ON DOING NEXT WEEK:

To solve the problems on which I am stuck.

To continue working on epipolar images generation.

To finish tri-stereo projection implementation.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

At the moment I have some issues in finding the proper epipolar model, but I am pretty confident to being able to find soon a suitable solution.

Weekly Report 3 (2015, June 14)

WHAT I GET DONE THIS WEEK:

I have worked (and I am still working) on a new version of matching for epipolar imagery generation, trying to implement a new and effective way able to generate epipolar images. Mentors (with community support) have suggested me to have a look to an OSSIM stand-alone executable (ossim-autreg) that seems to have some functionalities suitable to fix this issue.

I have implemented a tri-stereo projection for triplet input images (​https://github.com/martidi/opencv_dsm/tree/triplet).

WHAT I PLAN ON DOING NEXT WEEK:

To solve the problems on which I am stuck.

To continue working on epipolar images generation and analyze ossim-autreg executable.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

At the moment I have some issues in finding the proper epipolar model, but I am pretty confident to being able to find soon a suitable solution.

Weekly Report 4 (2015, June 21)

WHAT I GET DONE THIS WEEK:

I have implemented a tri-stereo automatic TPs identification and filtering to remove outliers.

I have implemented a tri-stereo epipolar images generation.

I have begun to implement a tri-stereo disparity map generation.

I have decided that I will use these two freely available triplet dataset to test my tri-stereo Plug-in:

​http://www.isprs.org/data/zy-3/default.aspx

​http://www.geo-airbusds.com/en/23-sample-imagery

in addition to further digital images available at my lab.

I have been to EARSeL Conference in Stockholm to present last year GSoC work, for this reason I have not had so much time to write a well-structured code (but I will fix this next week). Here ​https://www.dropbox.com/s/5r8rvlii5x3locy/DiRita_et-al.pdf?dl=0 you can find the presentation I have given.

WHAT I PLAN ON DOING NEXT WEEK:

To solve the problems on which I am stuck.

To scrub the code that I have written this week in so little time (it works but it has some "architectural" problem).

To continue working on epipolar images generation and analyze ossim-autreg executable as mentors suggested.

To complete the tri-stereo disparity map implementation.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

At the moment I have some issues in finding the proper epipolar model, but I am pretty confident to being able to find soon a suitable solution.

Weekly Report 5 (2015, June 28)

WHAT I GET DONE THIS WEEK:

I have implemented a tri-stereo disparity map generation.

I have begun to analyze ossim-autreg executable as mentors suggested.

I have begun to write the midterm evaluation.

I have tested the tri-stereo Plug-in with Trento Pléiades imagery and Hong Kong ZY3-1 free imagery: hereafter it is possible to see some results of the workflow implemented so far for the Hong Kong dataset.

1) Wallis filtering for the nadiral, forward and backward images, to let SGBM algorithm be more efficient.

(Here is reported only the nadiral one as an example)

2a) Automatic TPs identification between NAD-FWD images (for a visualization problem it seems that only a dozen of points are found, whereas the Plug-in at this step finds more ore less a hundred points)

2b) Automatic TPs identification between NAD-BWD images

3a) Disparity map obtained from NAD-FWD images

3b) Disparity map obtained from NAD-BWD images

As it could be seen, as predictable and quite obvious, the Disparity map has got some problems above the sea, it is possible to obtain only a diffused noise, whereas in the other areas is obtained a good disparity map, in which it is possible to clearly identify the buildings shape (clearly visible in the tile zoomed below).

To date, it is therefore possible to achieve disparity maps starting from a tri-stereo dataset.

Next step to be implemented in the workflow is the one related to development and implementation of disparity maps fusion algorithm.

Moreover, I have seen that there are some issues in the epipolar constraint in morphologically complex areas: I'm planning to fix this in the next weeks, after a deep analysis of the problem. Furthermore I'm planning to spend some time scrubbing the Plug-in code and to re-design the created classes structure.

WHAT I PLAN ON DOING NEXT WEEK:

To solve the problems on which I am stuck.

To finalize and submit the midterm evaluation.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

At the moment I have some issues in finding the proper epipolar model, but I am pretty confident to being able to find soon a suitable solution.

Weekly Report 6 (2015, July 05)

WHAT I GET DONE THIS WEEK:

I have finalized and submitted the midterm evaluation.

I have begun to implement disparity map fusion: starting from two disparity maps (obtained from the 3 images processed) I would like to obtain a single disparity map resulting from a clever fusion of the two previous maps. I thought to get the mean disparity value from the two maps for each (i,j) position if the difference between the two maps is under a threshold, otherwise, if the difference is higher, I get the value closest to the reference DSM used for the initial projection, moreover where there is a no data value I get the other disparity map result.

Following advice from mentors and community, in order to improve automatic TPs identification enabling a more homogeneous points detection, I have implemented a new OpenCV class able to find detectors for a source image divided into a grid and able to detect points in each cell. As you can see from the images below, results seems enough satisfying.

Result for NAD-FWD pair:

Result for NAD-BWD pair:

I have also begun to try solving the epipolar issue.

WHAT I PLAN ON DOING NEXT WEEK:

To solve the problems on which I am stuck.

To continue implementing disparity map fusion and filtering.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

---

Weekly Report 7 (2015, July 12)

WHAT I GET DONE THIS WEEK:

I have worked a lot about the finding of the proper plane on which project input images for obtain epipolar images. I have implemented it using ossim built-in classes. At the moment, what I need is to create a tif file from this plane that for now is materialized just as discrete points entity. As a matter of fact to use this plane for my purposes I need to convert it in a tif format in order to use it as elevation source. I almost manage this thanks to a big support received from mentors.

WHAT I PLAN ON DOING NEXT WEEK:

Next week I will attend FOSS4G-E 2015 conference (​http://europe.foss4g.org/2015/) to have a presentation about my last year GSoC project and results ("Photogrammetric image processing: DSM generation tool for OSSIM"). For this reason probably I will not have so much time to work on this year GSoC project. When I will came back at the end of the week, I will try to find time to continue implementing disparity map fusion and filtering.

SOMETHING ON WHICH I AM BLOCKED:

I have some issues in managing the achievement of a tif file from ossim point entity, but fortunately I'm having a big support from mentors, so I think I will manage to fix this soon.

Weekly Report 8 (2015, July 19)

WHAT I GET DONE THIS WEEK:

I have attended the FOSS4G-E 2015 Conference in Como (Italy) and I have given a talk about my last year GSoC project and results ("Photogrammetric image processing: DSM generation tool for OSSIM").

WHAT I PLAN ON DOING NEXT WEEK:

To complete disparity map fusion and filtering implementation.

To implement the conversion from the disparity map to a geocoded DSM.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

I have some issues in managing the achievement of a tif file from ossim point entity, but fortunately I'm having a big support from mentors, so I think I will manage to fix this soon.

Weekly Report 9 (2015, July 26)

WHAT I GET DONE THIS WEEK:

I have solved the issue related to the generation of a tif file from ossim point entity thanks to mentors support: there was a problem in my preference file.

I have fixed a core dump I had (from last year GSoC) at the end of the workflow performed, related to an ossimImageFileWriter pointer badly deleted.

I have completed disparity map fusion and filtering implementation. It is possible to see a result obtained on a Pleiades triplet on Trento site.

I have implemented the conversion from the disparity map to a geocoded DSM. It is possible to see a result obtained on a Pleiades triplet on Trento site.

WHAT I PLAN ON DOING NEXT WEEK:

To begin working on SAR imagery processing.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

I still have some issues in finding the correct epipolar geometry.

Weekly Report 10 (2015, August 02)

WHAT I GET DONE THIS WEEK:

I have begun to implement a new way to reach the quasi-epipolar geometry, based on a pyramidal approach.

I have delayed the beginning of SAR processing implementation since, before starting this new step, I would like to fix once and for all the epipolar issue to generate a very good DSM in whatever morphological condition.

WHAT I PLAN ON DOING NEXT WEEK:

To continue working on my epipolar issue, trying to solve it definitively.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

---

Weekly Report 11 (2015, August 09)

WHAT I GET DONE THIS WEEK:

I have implemented a new way to optimize the quasi-epipolar geometry, based on a pyramidal approach, that allows to obtain a better result on the vertical component with an accuracy greater than about 1,3 m (the validation on the same area gives out a RMSE of 4,5 m instead of 5,8 m with the old approach). For now it is not an automatic processing chain, but I would like to make it totally automatic next week.

I have begun to implement SAR imagery processing in the plug-in: in particular I managed to handle a RADARSAT-2 stereo-pair.

WHAT I PLAN ON DOING NEXT WEEK:

To make automatic the pyramidal quasi-epipolar processing chain.

To continue working on RADARSAT-2 imagery and to finish implementing SAR processing in the plug-in.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

I have some problem in opening TerraSAR-X imagery with OSSIM functionalities.

Weekly Report 12 (2015, August 16)

WHAT I GET DONE THIS WEEK:

I have implemented a totally automatic pyramidal approach to optimize the quasi-epipolar geometry both for stereo and tri-stereo plug-in, useful in particular for DSM extraction in areas characterized by complex terrain morphology. At the moment it is possible to define as input parameters (in addition to those already defined) the final resolution that has to be achieved and the number of pyramidal steps to be performed: at the first step as coarse DSM the SRTM is used, at the following steps the DSM obtained from the previous step is used as coarse DSM in order to optimize the final DSM. As already mentioned in the last report, a big improvement in the accuracy obtained on the final DSM is thus achieved.

I have continued working on SAR imagery processing implementation, but I have had some problem in correctly reading the images.

WHAT I PLAN ON DOING NEXT WEEK:

To scrub code to make it suitable for OSSIM repository.

To make some further test for DSM validation.

To write documentation about how to compile and test the plug-in.

To test and control implementation efficacy – discuss results with mentors.

SOMETHING ON WHICH I AM BLOCKED:

I have not had enough time to work on SAR images processing implementation since, in order to improve the quasi-epipolar geometry, I have choose to implement the pyramidal approach that it was not scheduled in the initial timeline.

Final Report (2015, August 21)

I have finished the code "scrubbing" both for the stereo plug-in developed last summer and updated according to the new multi-resolution pyramidal approach (​https://github.com/martidi/opencv_dsm) and for the tri-stereo plug-in entirely developed within this GSoC (​https://github.com/martidi/opencv_dsm/tree/triplet).

At the following link the final documentation of the developed plug-in for DSM extraction from tri-stereo imagery can be found. ​https://www.dropbox.com/s/otauq0u3jd4cxiq/GSoC2015_DiRita_FinalReport.pdf?dl=0

WHAT I PLAN ON DOING IN THE NEXT FUTURE:

In addition to complete the SAR plug-in, as a possible development of this project, it could be feasible to implement an automatic DSM to DTM conversion. Many engineering and geomatics applications require a Digital Terrain Model (DTM) which represents the bare ground surface without any objects, but to convert a DSM to a DTM through manual editing is a very time consuming process, for this reason it would be particularly interesting and useful to implement this kind of functionality.