[PATCH] TSX COSAR driver and 'pixel validity mask'

[pmar]

The current COSAR reader/driver implementation does not correctly resolve for the 'pixel validity mask': it shifts the whole range line, including the 'no valid' pixels, rather then masking them out.

The listed patch fixes the problem.

The patch was tested on a stack of Stripmap data, and by visual comparison of the GDAL output with GeoTiff? Quicklooks (that are provided with images). Moreover, interferometric processing done on the COSAR data pre-processed by GDAL and patched COSAR driver, resulted in an expected values (no missing lines/pixels) and coherent interferometric signal.

Attached:

* patch: cosar_dataset_validity_mask.patch
* screenshot 1: problem.png (run with trunk cosar_dataset.cpp)
* screenshot 2: patched.png (run with patched cosar_dataset.cpp)
* screenshot 3: interferometric_check.png

Reference:

TX-GS-DD-3307 : TerraSAR-X Level 1b product format specification

[pmar]

frmts/cosar/cosar_dataset.cpp , patch for the pixel validity mask

[pmar]

visualization of the problem

[pmar]

interferometric check of the performance of the patched driver

[pmar]

output of the patched cosar driver

[pmar]

The current COSAR reader/driver implementation does not correctly resolve for the 'pixel validity mask': it shifts the whole range line, including the 'no valid' pixels, rather then masking them out.

The listed patch fixes the problem.

The patch was tested on a stack of Stripmap data, and by visual comparison of the GDAL output with GeoTiff?? Quicklooks (that are provided with images). Moreover, interferometric processing done on the COSAR data pre-processed by GDAL and patched COSAR driver, resulted in a coherent interferometric signal with no missing lines/pixels.

Attached:

* patch: cosar_dataset_validity_mask.patch * screenshot 1: problem.png (run with trunk cosar_dataset.cpp) * screenshot 2: patched.png (run with patched cosar_dataset.cpp) * screenshot 3: interferometric_check.png Reference:

TX-GS-DD-3307 : TerraSAR-X Level 1b product format specification

[warmerdam]

Chaitanya,

I have no test data for this format, so I'd ask you to carefully apply the patch in 1.6 branch and trunk without actual testing.

Phil - let us know if you have any concerns about the patch.

[pvachon]

The patch looks good, but I can't reproduce the original issue myself. It might be a lack of test data on my part though.

Cheers, Phil

[pmar]

this bug exposes itself only with CoSAR data, where not fully focused pixels are "blacked out" with the validity annotation. or, that for the tag nRSFV (Range Samples First Valid) values are larger then 1. from my experience, ~60 CoSAR images (both stripmap and spotlight modes), this was the case with only recently received stack of data. hence the patch.

cheers, petar

[chaitanya]

petar,

The reference TX-GS-DD-3307 also mentions the parameters ASFV (Azimuth Sample First Valid) and ASLV (Azimuth Sample Last Valid). Shouldn't the driver consider those too? I'm holding the patch until I hear your suggestion.

[pmar]

chaitanya,

i believe the patch is okay, and that if patched cosar_dataset.cpp would be consistent with the reference document 'TX-GS-DD-3307'. at least for a single 'burst' data, eg, stripmap and spotlight modes.

here is my explanation/comment:

indeed that document mentions the azimuth parameters, but there is no ambiguity between the 'validity annotation mask' in range (RSFV/RSLV) and azimuth (ASFV/ASLV). in other words, both azimuth/range validity annotations somehow define the same. which one to work with depends on the way the data is being read into memory. since the cosar_dataset.cpp reads the data 'horizontally' line by line, it's only important to correctly account for the 'mask' in range.

so, i'd say the original implementation of cosar_dataset.cpp is quite robust, with that small problem that the data lines with RFSV > 1 are shifted rather then masked out (see: http://trac.osgeo.org/gdal/attachment/ticket/3112/problem.png). something that the patch should fix.

cheers, petar

[chaitanya]

Replying to pmar:

I am not familiar with the COSAR format. Allow me a doubt. Wouldn't there be situations where the azimuth mask invalidates a block that comes after the first horizontal block?

In the figure below, the third block in the second line could only be invalidated by the azimuth but the range parameter cannot block this because the second block of the second line is the first block of that line.

000####00
0#0#####0
#########
0######00

[pmar]

hi, indeed that could be the case, but for CoSAR format i do not think that situations like that are possible and allowed by the actual format definitions.

here is why:

note that you only do have start/end values for that validity mask. if range/azimuth annotations would interleave, considering you only have start/end values, in an example bellow, it would be impossible to correctly annotate eg pixel (line:5,column:9).

# : is good
0 : is bad

  123456789
1 000####00
2 0#0#####0
3 #########
4 0######00
5 0##00####

so, i do not think such a situation is possible because of limited number of combination allowed by the format.

also, the physical interpretation of the image would be then very hard, since a single pixel cannot be 'good' in azimuth and 'bad' in range. moreover, the interpretation of the 'bad' pixels is that, those are the ones that are affected by the echo-window-position (or sampling-window-start-time) shifts, hence they could appear only at the beginning/end of the range/scan line.

i attached a screenshot of an example of the burst annotation from the format reference document.

finally, the metadata for the data segments is given in XML file, (at least to me? :) confirms that situations like that are not anticipated and accounted by the format. note, in a bellow listed xml segment, that all the annotations in both azimuth range are 'continues'. Azimuth: start/stopTimeUtc and Range: echoWindowPosition/echoWindowLength.

<numberOfEchoWindowPositionChanges>1</numberOfEchoWindowPositionChanges>
<numberOfEchoWindowLengthChanges>1</numberOfEchoWindowLengthChanges>
<numberOfSettingRecords>2</numberOfSettingRecords>
<settingRecord>
   <dataSegment segmentID="1">
      <startTimeUTC>2009-07-21T18:38:59.290603Z</startTimeUTC>
      <stopTimeUTC>2009-07-21T18:39:02.272935Z</stopTimeUTC>
      <numberOfRows>11011</numberOfRows>
   </dataSegment>
   <PRF code="0">3.69241021505376330E+03</PRF>
   <echoWindowPosition code="306">672</echoWindowPosition>
   <echowindowLength code="195">2.49600000000000000E+04</echowindowLength>
   <pulseType>Standard</pulseType>
   <echoIndex>14</echoIndex>
</settingRecord>

[pmar]

Figure 4.4 from TSX-GS-DD-3307

[chaitanya]

Applied the patch in trunk (r17585) and 1.6 branch (r17586). No tests were performed.

Thanks to petar for the prompt responses to the queries.