Opened 12 years ago

Closed 12 years ago

#2956 closed defect (fixed)

Keyword causing format unrecognized (PDS)

Reported by: thare Owned by: warmerdam
Priority: normal Milestone: 1.6.3
Component: GDAL_Raster Version: unspecified
Severity: normal Keywords: PDS keyword read error, GDAL 1.7.0dev, FWTools 2.3.1
Cc: warmerdam

Description

A PDS keyword I have not seen before, MISSING_CONSTANT, is causing a format unrecognized error. The odd part is that it depends on where it is in the label. Thus if it is early in the "Object = Image" group it passes. If it is last in this group it fails. Note this is basically the same keyword as CORE_NULL thus it appears MISSING_CONSTANT = CORE_NULL. I don't think either of these keywords is actually propogated to the GDAL NULL value which is fine for now. If I remember, the GDAL NULL value is hardwired depending on the PDS bit type.

I have attached two PDS image labels with the keyword in different locations. The ".lbl" file works and the ".lab" file doesn't. The only difference is the location of the keyword MISSING_CONSTANT in the OBJECT = IMAGE group. I have not attached the large pixel file, "DTEEC_008669_1705_009025_1705_A01.img", which these labels point to and would be needed for a file conversion. gdalinfo can run on the labels without the pixel file.

My testing software is GDAL 1.7.0dev, FWTools 2.3.1

thanks, Trent

Attachments (2)

DTEEC_008669_1705_009025_1705_A01.lab (3.3 KB) - added by thare 12 years ago.
PDS label with read error from MISSING_CONSTANT
DTEEC_008669_1705_009025_1705_A01.lbl (3.3 KB) - added by thare 12 years ago.
PDS label file which reads correctly

Download all attachments as: .zip

Change History (13)

comment:1 Changed 12 years ago by warmerdam

Cc: warmerdam added
Milestone: 1.7.01.6.1
Owner: changed from warmerdam to chaitanya

Please look into this and resolve. Depending on the nature of the problem it might also have some impact on the ISIS2 and ISIS3 drivers. Fixes should be propagated into 1.6.x and even 1.5.x branches.

Changed 12 years ago by thare

PDS label with read error from MISSING_CONSTANT

Changed 12 years ago by thare

PDS label file which reads correctly

comment:2 Changed 12 years ago by chaitanya

Status: newassigned

Trent, I got an error with both the files. However, it was resolved when I enclosed the value of MISSING_CONSTANT in double quotes.

 MISSING_CONSTANT           = "#16FF7FFFFB#" 

Frank, the parser is skipping the value since it looked like a comment block to it. What shall we do?

comment:3 Changed 12 years ago by thare

Resolution: invalid
Status: assignedclosed

The value was written incorrectly in the label. I will let the creators know so they can fix it. We can probably call this bug dead-on-arrival. I set it as "resolve as: invalid" which sounded correct. All should be good. Thanks for looking into it and sorry for issuing.

It was set to

MISSING_CONSTANT = #16FF7FFFFB#

when it should be

MISSING_CONSTANT = 16#FF7FFFFB#

BTW, It is weird that the *.lbl didn't work. I did update the original attachment on the trac site maybe you had the original one?

-Trent

comment:4 Changed 12 years ago by chaitanya

Resolution: invalid
Status: closedreopened

Frank, According to http://pds.jpl.nasa.gov/documents/sr/Chapter12.pdf, comments in PDS are not handled correctly. Shall I correct this?

comment:5 Changed 12 years ago by warmerdam

Yes, please do. Also please add the above document as a See Also: at the bottom of the PDS/ISIS2/ISIS3 documents and in the comments at the top of the NASAKeywordParser which should be clarified to be an ODL parser.

comment:6 Changed 12 years ago by chaitanya

Modified gdal/frmts/pds/nasakeywordhandler.cpp to be more in compliant with Object Description Language as described in http://pds.jpl.nasa.gov/documents/sr/Chapter12.pdf in trunk (r16868).

However, there was a modification regarding ticket #2176 which made it to handle # style comments. The sample ISIS3 label in it had a comment

# Smoothed using spicefit 

The file also has C style comments. This # style comment seems to be added later by some other software.

I don't think adding # style comments is a good idea. I am withholding committing to 1.6 and 1.5 branches until this is resolved.

Trent, Could you comment on this please?

comment:7 Changed 12 years ago by chaitanya

Resolution: fixed
Status: reopenedclosed

Readded the # style comments in trunk (r16879). However, now considering only lines starting with a # as comments.

Propagated these modifications to 1.6 along with the changes from r16388 (r16880)

Skipping 1.5 for now. There were many modifications overlapping other files since 1.5 was branched off.

comment:8 Changed 12 years ago by thare

While the PDS and ISIS2 and ISIS3 formats are all similar, each format has their own particulars. While I don't think "#" is not really a valid comment, it has ended up as a reporting mechanism in ISIS3 and I think we should keep it in there. I don't think it will cause any future issues with the other formats. Thanks again for researching the PDS documents to get some good updates included.

-Trent

comment:9 Changed 12 years ago by thare

Resolution: fixed
Status: closedreopened

Sorry I have to open this one up again for a related issue. I am submitting this from a NASA Collogue who help to debug this. He states:

The problem is any #-style comment with any whitespace in front of it and comments - that there might need to be a more robust method for avoiding faults with whitespaces.

I have uploaded an ISIS3 image with the offending label. ftp://pdsimage2.wr.usgs.gov/pub/pigpen/.tmp/E0101026.clip.cub.gz (1MB)

thanks, Trent

Here is his optional patch "nasakeywordhandler.patch"

*** /Users/zm/packages/i386_darwin_gcc4.0/gdal-1.7.0-dev/SRC/frmts/pds/nasakeywordhandler.cpp	2009-04-29 01:55:10.000000000 -0700
--- nasakeywordhandler.cpp	2009-08-07 18:24:38.000000000 -0700
***************
*** 341,347 ****
          }
  
          // Skip # style comments 
!         if( (*pszHeaderNext == 10 || *pszHeaderNext == 13 )
              && pszHeaderNext[1] == '#' )
          {
              pszHeaderNext += 2;
--- 341,348 ----
          }
  
          // Skip # style comments 
!         if( (*pszHeaderNext == 10 || *pszHeaderNext == 13 ||
! 	     *pszHeaderNext == ' ' || *pszHeaderNext == '\t' )
              && pszHeaderNext[1] == '#' )
          {
              pszHeaderNext += 2;

comment:10 Changed 12 years ago by warmerdam

Milestone: 1.6.11.6.3
Owner: changed from chaitanya to warmerdam
Status: reopenednew

I'll take care of this one today or tomorrow.

comment:11 Changed 12 years ago by warmerdam

Resolution: fixed
Status: newclosed

I have confirmed the problem, and that the suggested code corrects it. I am somewhat concerned that some other configurations with " #" in the middle will get clipped.

I have updated the trunk testsuite to check for this issue (r17530) and added the change in trunk (r17531). The change has also been ported into the 1.6 branch (r1752).

Note: See TracTickets for help on using tickets.