Opened 7 years ago

Closed 6 years ago

#3930 closed defect (fixed)

RPF dataset error

Reported by: morabit Owned by: warmerdam
Priority: normal Milestone: 1.8.1
Component: GDAL_Raster Version: 1.8.0
Severity: normal Keywords: NITF
Cc: Even Rouault

Description (last modified by morabit)


Hi, we have problems to load a rpf dataset (perhaps CADRG?)
using the GDAL 1.8 API GDALOpen.
The error message returned is:
003PAN16.I12, assert '(entry->vertInterval - (-geoTransf[GEOTRSFRM_NS_RES])) / entry->horizInterval < 0.01' failed

The free version of FalconView? application loads the dataset apparently without problems.
This is strange beacuse to our knowledge it uses GDAL internally to handle raster data.
We attach one of the frame files that fails.

Thank you for your attention

B.

Attachments (1)

003PAN16.I12 (114.7 KB) - added by morabit 7 years ago.

Download all attachments as: .zip

Change History (5)

Changed 7 years ago by morabit

Attachment: 003PAN16.I12 added

comment:1 Changed 7 years ago by warmerdam

Cc: Even Rouault added
Keywords: NITF added
Priority: highestnormal

Quickly examining the .I12 file, I get a complain from "gdalinfo -mm" that the VQ LUT is missing and decoding of imagery fails. I don't see the mentioned assertion though it is possible my build has assertions disabled. I'm going to see if this is something Even wants to follow up on since he has an interest in NITF issues.

comment:2 Changed 7 years ago by Even Rouault

I knew it remembered me of something. Indeed, you reported identical problems in #3848 and indeed gdalinfo --debug on reports this :

NITF: Adjusting RPFIMG TRE size from 1400 to 1350, which is the remaining size
NITF: The CoverageSectionSubheader content isn't consistant
Warning 1: Ignoring NITF RPF Location table since it seems to be corrupt.
NITF: Adjusting RPFIMG TRE size from 1400 to 1350, which is the remaining size
NITF: Adjusting RPFIMG TRE size from 1400 to 1350, which is the remaining size
GDAL: NITFDataset::Open() wasn't able to derive a first order
geotransform.  It will be returned as GCPs.
NITF: Adjusting RPFIMG TRE size from 1400 to 1350, which is the remaining size
NITF: Adjusting RPFIMG TRE size from 1400 to 1350, which is the remaining size
NITF: Adjusting RPFIMG TRE size from 1400 to 1350, which is the remaining size
NITF: Adjusting RPFIMG TRE size from 1400 to 1350, which is the remaining size

Now the new crap is in the IGEOLO record.

GCP[  0]: Id=UpperLeft, Info=
          (0.5,0.5) -> (12.2436111111111,41.9847222222222,0)
GCP[  1]: Id=UpperRight, Info=
          (1535.5,0.5) -> (12.4263888888889,42.1227777777778,0)
GCP[  2]: Id=LowerRight, Info=
          (1535.5,1535.5) -> (12.4263888888889,41.9847222222222,0)
GCP[  3]: Id=LowerLeft, Info=
          (0.5,1535.5) -> (12.2436111111111,41.9847222222222,0)

It is pretty obvious that the GCP 0 is wrong : the latitude should likely be 42.1227777778. The test added in r21181 tried to check that the CoverageSectionSubheader? is consistant with the IGEOLO to decide whether to use the RPF location table at all. As it isn't, we reject the location of VQ tables as well.

The assertion "(entry->vertInterval - (-geoTransf[GEOTRSFRM_NS_RES])) / entry->horizInterval < 0.01" is a direct consequence of that when opening the RPF A.TOC file that refers to that file because there's no valid georeferencing.

So obviously this is still an invalid NITF file from the same data producer. Did you have any news from him ?

As I begin to be fed up with dealing with that rubbish, in a hope to "solve" definitely the issue with those datasets, I've added in trunk in r21591 a NITF_DISABLE_RPF_LOCATION_TABLE_SANITY_TESTS configuration option that you can set to TRUE to blindly trust the RPF location table even if it doesn't look sane. With the attached sample, it seems to do the job...

comment:3 in reply to:  2 Changed 6 years ago by morabit

Description: modified (diff)

Replying to rouault:

I knew it remembered me of something. Indeed, you reported identical problems in #3848 and indeed gdalinfo --debug on reports this :

NITF: Adjusting RPFIMG TRE size from 1400 to 1350, which is the remaining size
NITF: The CoverageSectionSubheader content isn't consistant
Warning 1: Ignoring NITF RPF Location table since it seems to be corrupt.
NITF: Adjusting RPFIMG TRE size from 1400 to 1350, which is the remaining size
NITF: Adjusting RPFIMG TRE size from 1400 to 1350, which is the remaining size
GDAL: NITFDataset::Open() wasn't able to derive a first order
geotransform.  It will be returned as GCPs.
NITF: Adjusting RPFIMG TRE size from 1400 to 1350, which is the remaining size
NITF: Adjusting RPFIMG TRE size from 1400 to 1350, which is the remaining size
NITF: Adjusting RPFIMG TRE size from 1400 to 1350, which is the remaining size
NITF: Adjusting RPFIMG TRE size from 1400 to 1350, which is the remaining size

Now the new crap is in the IGEOLO record.

GCP[  0]: Id=UpperLeft, Info=
          (0.5,0.5) -> (12.2436111111111,41.9847222222222,0)
GCP[  1]: Id=UpperRight, Info=
          (1535.5,0.5) -> (12.4263888888889,42.1227777777778,0)
GCP[  2]: Id=LowerRight, Info=
          (1535.5,1535.5) -> (12.4263888888889,41.9847222222222,0)
GCP[  3]: Id=LowerLeft, Info=
          (0.5,1535.5) -> (12.2436111111111,41.9847222222222,0)

It is pretty obvious that the GCP 0 is wrong : the latitude should likely be 42.1227777778. The test added in r21181 tried to check that the CoverageSectionSubheader? is consistant with the IGEOLO to decide whether to use the RPF location table at all. As it isn't, we reject the location of VQ tables as well.

The assertion "(entry->vertInterval - (-geoTransf[GEOTRSFRM_NS_RES])) / entry->horizInterval < 0.01" is a direct consequence of that when opening the RPF A.TOC file that refers to that file because there's no valid georeferencing.

So obviously this is still an invalid NITF file from the same data producer. Did you have any news from him ?

No, unfortunely it's extremely complicated for our developement team to trace back the process of data production performed by our actual client.

As I begin to be fed up with dealing with that rubbish, in a hope to "solve" definitely the issue with those datasets, I've added in trunk in r21591 a NITF_DISABLE_RPF_LOCATION_TABLE_SANITY_TESTS configuration option that you can set to TRUE to blindly trust the RPF location table even if it doesn't look sane. With the attached sample, it seems to do the job...

Thank you for your help, perhaps this is the best approach to handle this kind of data, I begin to fed up too with this weird datasets.

Best regards Bruno M.

comment:4 Changed 6 years ago by Even Rouault

Milestone: 1.8.1
Resolution: fixed
Severity: criticalnormal
Status: newclosed

ok, I've backported the change to 1.8 branch in r21619

Note: See TracTickets for help on using tickets.