Keyword causing format unrecognized (PDS)

[thare]

A PDS keyword I have not seen before, MISSING_CONSTANT, is causing a format unrecognized error. The odd part is that it depends on where it is in the label. Thus if it is early in the "Object = Image" group it passes. If it is last in this group it fails. Note this is basically the same keyword as CORE_NULL thus it appears MISSING_CONSTANT = CORE_NULL. I don't think either of these keywords is actually propogated to the GDAL NULL value which is fine for now. If I remember, the GDAL NULL value is hardwired depending on the PDS bit type.

I have attached two PDS image labels with the keyword in different locations. The ".lbl" file works and the ".lab" file doesn't. The only difference is the location of the keyword MISSING_CONSTANT in the OBJECT = IMAGE group. I have not attached the large pixel file, "DTEEC_008669_1705_009025_1705_A01.img", which these labels point to and would be needed for a file conversion. gdalinfo can run on the labels without the pixel file.

My testing software is GDAL 1.7.0dev, FWTools 2.3.1

thanks, Trent

[warmerdam]

Please look into this and resolve. Depending on the nature of the problem it might also have some impact on the ISIS2 and ISIS3 drivers. Fixes should be propagated into 1.6.x and even 1.5.x branches.

[thare]

PDS label with read error from MISSING_CONSTANT

[thare]

PDS label file which reads correctly

[chaitanya]

Trent, I got an error with both the files. However, it was resolved when I enclosed the value of MISSING_CONSTANT in double quotes.

 MISSING_CONSTANT           = "#16FF7FFFFB#" 

Frank, the parser is skipping the value since it looked like a comment block to it. What shall we do?

[thare]

The value was written incorrectly in the label. I will let the creators know so they can fix it. We can probably call this bug dead-on-arrival. I set it as "resolve as: invalid" which sounded correct. All should be good. Thanks for looking into it and sorry for issuing.

It was set to

MISSING_CONSTANT = #16FF7FFFFB#

when it should be

MISSING_CONSTANT = 16#FF7FFFFB#

BTW, It is weird that the *.lbl didn't work. I did update the original attachment on the trac site maybe you had the original one?

-Trent

[chaitanya]

Frank, According to ​http://pds.jpl.nasa.gov/documents/sr/Chapter12.pdf, comments in PDS are not handled correctly. Shall I correct this?

[warmerdam]

Yes, please do. Also please add the above document as a See Also: at the bottom of the PDS/ISIS2/ISIS3 documents and in the comments at the top of the NASAKeywordParser which should be clarified to be an ODL parser.

[chaitanya]

Modified gdal/frmts/pds/nasakeywordhandler.cpp to be more in compliant with Object Description Language as described in ​http://pds.jpl.nasa.gov/documents/sr/Chapter12.pdf in trunk (r16868).

However, there was a modification regarding ticket #2176 which made it to handle # style comments. The sample ISIS3 label in it had a comment

# Smoothed using spicefit 

The file also has C style comments. This # style comment seems to be added later by some other software.

I don't think adding # style comments is a good idea. I am withholding committing to 1.6 and 1.5 branches until this is resolved.

Trent, Could you comment on this please?

[chaitanya]

Readded the # style comments in trunk (r16879). However, now considering only lines starting with a # as comments.

Propagated these modifications to 1.6 along with the changes from r16388 (r16880)

Skipping 1.5 for now. There were many modifications overlapping other files since 1.5 was branched off.

[thare]

While the PDS and ISIS2 and ISIS3 formats are all similar, each format has their own particulars. While I don't think "#" is not really a valid comment, it has ended up as a reporting mechanism in ISIS3 and I think we should keep it in there. I don't think it will cause any future issues with the other formats. Thanks again for researching the PDS documents to get some good updates included.

-Trent

[thare]

Sorry I have to open this one up again for a related issue. I am submitting this from a NASA Collogue who help to debug this. He states:

The problem is any #-style comment with any whitespace in front of it and comments - that there might need to be a more robust method for avoiding faults with whitespaces.

I have uploaded an ISIS3 image with the offending label. ​ftp://pdsimage2.wr.usgs.gov/pub/pigpen/.tmp/E0101026.clip.cub.gz (1MB)

thanks, Trent

Here is his optional patch "nasakeywordhandler.patch"

*** /Users/zm/packages/i386_darwin_gcc4.0/gdal-1.7.0-dev/SRC/frmts/pds/nasakeywordhandler.cpp	2009-04-29 01:55:10.000000000 -0700
--- nasakeywordhandler.cpp	2009-08-07 18:24:38.000000000 -0700
***************
*** 341,347 ****
          }
  
          // Skip # style comments 
!         if( (*pszHeaderNext == 10 || *pszHeaderNext == 13 )
              && pszHeaderNext[1] == '#' )
          {
              pszHeaderNext += 2;
--- 341,348 ----
          }
  
          // Skip # style comments 
!         if( (*pszHeaderNext == 10 || *pszHeaderNext == 13 ||
! 	     *pszHeaderNext == ' ' || *pszHeaderNext == '\t' )
              && pszHeaderNext[1] == '#' )
          {
              pszHeaderNext += 2;

[warmerdam]

I'll take care of this one today or tomorrow.

[warmerdam]

I have confirmed the problem, and that the suggested code corrects it. I am somewhat concerned that some other configurations with " #" in the middle will get clipped.

I have updated the trunk testsuite to check for this issue (r17530) and added the change in trunk (r17531). The change has also been ported into the 1.6 branch (r1752).