Opened 12 years ago

Closed 12 years ago

#3112 closed defect (fixed)

[PATCH] TSX COSAR driver and 'pixel validity mask'

Reported by: pmar Owned by: chaitanya
Priority: normal Milestone: 1.6.3
Component: GDAL_Raster Version: svn-trunk
Severity: normal Keywords: cosar driver pixel_validity_mask
Cc: pvachon, warmerdam

Description

The current COSAR reader/driver implementation does not correctly resolve for the 'pixel validity mask': it shifts the whole range line, including the 'no valid' pixels, rather then masking them out.

The listed patch fixes the problem.

The patch was tested on a stack of Stripmap data, and by visual comparison of the GDAL output with GeoTiff? Quicklooks (that are provided with images). Moreover, interferometric processing done on the COSAR data pre-processed by GDAL and patched COSAR driver, resulted in an expected values (no missing lines/pixels) and coherent interferometric signal.

Attached:

  • patch: cosar_dataset_validity_mask.patch
  • screenshot 1: problem.png (run with trunk cosar_dataset.cpp)
  • screenshot 2: patched.png (run with patched cosar_dataset.cpp)
  • screenshot 3: interferometric_check.png

Reference:

TX-GS-DD-3307 : TerraSAR-X Level 1b product format specification

Attachments (5)

cosar_dataset.patch (540 bytes) - added by pmar 12 years ago.
frmts/cosar/cosar_dataset.cpp , patch for the pixel validity mask
problem.png (886.6 KB) - added by pmar 12 years ago.
visualization of the problem
interferometric_check.png (972.5 KB) - added by pmar 12 years ago.
interferometric check of the performance of the patched driver
patched.png (910.2 KB) - added by pmar 12 years ago.
output of the patched cosar driver
burst_annotation.png (32.3 KB) - added by pmar 12 years ago.
Figure 4.4 from TSX-GS-DD-3307

Change History (16)

Changed 12 years ago by pmar

Attachment: cosar_dataset.patch added

frmts/cosar/cosar_dataset.cpp , patch for the pixel validity mask

comment:1 Changed 12 years ago by pmar

Priority: highnormal

Changed 12 years ago by pmar

Attachment: problem.png added

visualization of the problem

Changed 12 years ago by pmar

Attachment: interferometric_check.png added

interferometric check of the performance of the patched driver

Changed 12 years ago by pmar

Attachment: patched.png added

output of the patched cosar driver

comment:2 Changed 12 years ago by pmar

Component: defaultGDAL_Raster
Summary: [PATCH] TerraSAR-X COSAR Format Driver, does not correctly account for the 'pixel validity mask'[PATCH] TSX COSAR driver and 'pixel validity mask'

comment:3 Changed 12 years ago by pmar

The current COSAR reader/driver implementation does not correctly resolve for the 'pixel validity mask': it shifts the whole range line, including the 'no valid' pixels, rather then masking them out.

The listed patch fixes the problem.

The patch was tested on a stack of Stripmap data, and by visual comparison of the GDAL output with GeoTiff?? Quicklooks (that are provided with images). Moreover, interferometric processing done on the COSAR data pre-processed by GDAL and patched COSAR driver, resulted in a coherent interferometric signal with no missing lines/pixels.

Attached:

  • patch: cosar_dataset_validity_mask.patch
  • screenshot 1: problem.png (run with trunk cosar_dataset.cpp)
  • screenshot 2: patched.png (run with patched cosar_dataset.cpp)
  • screenshot 3: interferometric_check.png

Reference:

TX-GS-DD-3307 : TerraSAR-X Level 1b product format specification

comment:4 Changed 12 years ago by warmerdam

Cc: pvachon warmerdam added
Milestone: 1.6.3
Owner: changed from warmerdam to chaitanya

Chaitanya,

I have no test data for this format, so I'd ask you to carefully apply the patch in 1.6 branch and trunk without actual testing.

Phil - let us know if you have any concerns about the patch.

comment:5 Changed 12 years ago by pvachon

The patch looks good, but I can't reproduce the original issue myself. It might be a lack of test data on my part though.

Cheers, Phil

comment:6 Changed 12 years ago by pmar

this bug exposes itself only with CoSAR data, where not fully focused pixels are "blacked out" with the validity annotation. or, that for the tag nRSFV (Range Samples First Valid) values are larger then 1. from my experience, ~60 CoSAR images (both stripmap and spotlight modes), this was the case with only recently received stack of data. hence the patch.

cheers, petar

comment:7 Changed 12 years ago by chaitanya

Status: newassigned

petar,

The reference TX-GS-DD-3307 also mentions the parameters ASFV (Azimuth Sample First Valid) and ASLV (Azimuth Sample Last Valid). Shouldn't the driver consider those too? I'm holding the patch until I hear your suggestion.

comment:8 Changed 12 years ago by pmar

chaitanya,

i believe the patch is okay, and that if patched cosar_dataset.cpp would be consistent with the reference document 'TX-GS-DD-3307'. at least for a single 'burst' data, eg, stripmap and spotlight modes.

here is my explanation/comment:

indeed that document mentions the azimuth parameters, but there is no ambiguity between the 'validity annotation mask' in range (RSFV/RSLV) and azimuth (ASFV/ASLV). in other words, both azimuth/range validity annotations somehow define the same. which one to work with depends on the way the data is being read into memory. since the cosar_dataset.cpp reads the data 'horizontally' line by line, it's only important to correctly account for the 'mask' in range.

so, i'd say the original implementation of cosar_dataset.cpp is quite robust, with that small problem that the data lines with RFSV > 1 are shifted rather then masked out (see: http://trac.osgeo.org/gdal/attachment/ticket/3112/problem.png). something that the patch should fix.

cheers, petar

comment:9 in reply to:  8 Changed 12 years ago by chaitanya

Replying to pmar:

I am not familiar with the COSAR format. Allow me a doubt. Wouldn't there be situations where the azimuth mask invalidates a block that comes after the first horizontal block?

In the figure below, the third block in the second line could only be invalidated by the azimuth but the range parameter cannot block this because the second block of the second line is the first block of that line.

000####00
0#0#####0
#########
0######00

comment:10 Changed 12 years ago by pmar

hi, indeed that could be the case, but for CoSAR format i do not think that situations like that are possible and allowed by the actual format definitions.

here is why:

note that you only do have start/end values for that validity mask. if range/azimuth annotations would interleave, considering you only have start/end values, in an example bellow, it would be impossible to correctly annotate eg pixel (line:5,column:9).

# : is good
0 : is bad

  123456789
1 000####00
2 0#0#####0
3 #########
4 0######00
5 0##00####

so, i do not think such a situation is possible because of limited number of combination allowed by the format.

also, the physical interpretation of the image would be then very hard, since a single pixel cannot be 'good' in azimuth and 'bad' in range. moreover, the interpretation of the 'bad' pixels is that, those are the ones that are affected by the echo-window-position (or sampling-window-start-time) shifts, hence they could appear only at the beginning/end of the range/scan line.

i attached a screenshot of an example of the burst annotation from the format reference document.

finally, the metadata for the data segments is given in XML file, (at least to me? :) confirms that situations like that are not anticipated and accounted by the format. note, in a bellow listed xml segment, that all the annotations in both azimuth range are 'continues'. Azimuth: start/stopTimeUtc and Range: echoWindowPosition/echoWindowLength.

<numberOfEchoWindowPositionChanges>1</numberOfEchoWindowPositionChanges>
<numberOfEchoWindowLengthChanges>1</numberOfEchoWindowLengthChanges>
<numberOfSettingRecords>2</numberOfSettingRecords>
<settingRecord>
   <dataSegment segmentID="1">
      <startTimeUTC>2009-07-21T18:38:59.290603Z</startTimeUTC>
      <stopTimeUTC>2009-07-21T18:39:02.272935Z</stopTimeUTC>
      <numberOfRows>11011</numberOfRows>
   </dataSegment>
   <PRF code="0">3.69241021505376330E+03</PRF>
   <echoWindowPosition code="306">672</echoWindowPosition>
   <echowindowLength code="195">2.49600000000000000E+04</echowindowLength>
   <pulseType>Standard</pulseType>
   <echoIndex>14</echoIndex>
</settingRecord>

Changed 12 years ago by pmar

Attachment: burst_annotation.png added

Figure 4.4 from TSX-GS-DD-3307

comment:11 Changed 12 years ago by chaitanya

Resolution: fixed
Status: assignedclosed

Applied the patch in trunk (r17585) and 1.6 branch (r17586). No tests were performed.

Thanks to petar for the prompt responses to the queries.

Note: See TracTickets for help on using tickets.