Upcoming PDS raster not correctly read

[michaelaye]

The DAWN Framing Camera (Mission to Asteroids) will soon release data in PDS format, an example is attached. The errors vary, depending on the header and formats implemented. For the attached file the failure mode is, that the read data becomes of dtype int16, where it should have been uint16.

[warmerdam]

The header text looks like this:

PDS_VERSION_ID                = PDS3
LABEL_REVISION_NOTE           = "20080201, PGM, DAWN FC V1.5"

/* FILE CHARACTERISTICS */

RECORD_TYPE                   = FIXED_LENGTH
RECORD_BYTES                  = 512
FILE_RECORDS                  = 4258
LABEL_RECORDS                 = 22
FILE_NAME                     = "FC11A0000338_07344220211F2C.IMG"

/* POINTERS TO DATA OBJECTS */

^IMAGE                        = 24
^FRAME_2_IMAGE                = 4120
^FRAME_3_IMAGE                = 4162
^FRAME_4_IMAGE                = 4195
^FRAME_5_IMAGE                = 4227
^HISTORY                      = 23

/* SOFTWARE */

SOFTWARE_DESC                 = "TRAP.EXE"
SOFTWARE_LICENSE_TYPE         = "COMMERCIAL"
SOFTWARE_ID                   = "TRAP"
SOFTWARE_NAME                 = "TRAP"
SOFTWARE_VERSION_ID           = "2.33"
SOFTWARE_RELEASE_DATE         = 2009-06-19

/*   TELEMETRY IDENTIFICATION   */

TELEMETRY_FORMAT_ID           = "301"

/*   PRODUCT IDENTIFICATION   */

DATA_SET_NAME                 = 
    "DAWN FC1 RAW (EDR) CRUISE CHECKOUT/CALIBRATION IMAGES V1.0"
DATA_SET_ID                   = "DAWN-X-FC1-2-EDR-CRUISE-IMAGES-V1.0"
PRODUCT_ID                    = "0000338"
PRODUCT_TYPE                  = "DATA"
STANDARD_DATA_PRODUCT_ID      = "FC_IMAGE"
PRODUCER_FULL_NAME            = "PABLO GUTIERREZ-MARQUES"
PRODUCER_INSTITUTION_NAME     = 
    "MAX PLANCK INSTITUT FUER SONNENSYSTEMFORSCHUNG"
PRODUCT_CREATION_TIME         = 2009-06-29T15:01:50.000
PRODUCT_VERSION_ID            = "C"
RELEASE_ID                    = "0001"

/*   MISSION IDENTIFICATION   */

INSTRUMENT_HOST_ID            = "DAWN"
INSTRUMENT_HOST_NAME          = "DAWN"
MISSION_ID                    = "DAWN"
MISSION_NAME                  = "DAWN MISSION TO VESTA AND CERES"
MISSION_PHASE_NAME            = "INITIAL CHECKOUT (ICO)"

/*   INSTRUMENT DESCRIPTION   */

INSTRUMENT_ID                 = "FC1"
INSTRUMENT_NAME               = "FRAMING CAMERA 1"
OBSERVATION_ID                = "N/A"
OBSERVATION_TYPE              = "N/A"
INSTRUMENT_TYPE               = "FRAME CCD REFRACTING TELESCOPE"
DETECTOR_DESC                 = "1092x1056 PIXELS FRONTLIT FRAMETRANSFER CCD"
DETECTOR_TYPE                 = "SI CCD"
DETECTOR_TEMPERATURE          = 222.888 <kelvin>

/*   TIME IDENTIFICATION   */

SPACECRAFT_CLOCK_START_COUNT  = "250596196:050"
SPACECRAFT_CLOCK_STOP_COUNT   = "250596197:099"
START_TIME                    = 2007-344T22:02:11.451
STOP_TIME                     = 2007-344T22:02:12.642

/*   SYSTEM HARDWARE AND SOFTWARE CONFIGURATION   */

DAWN:DPU_HARDWARE_ID          = "1.04"
DAWN:DPU_SOFTWARE_VERSION     = "3.01"
DAWN:UDPLIB_SOFTWARE_VERSION  = "3.01.05"
DAWN:PCU_HARDWARE_ID          = 3.05
DAWN:FEE_HARDWARE_ID          = "017.10.10"
DAWN:MCU_HARDWARE_ID          = "10"

/*   MECHANISM STATUS   */

DAWN:FILTER_ENCODER           = 26
FILTER_NUMBER                 = "2"
DAWN:FRONT_DOOR_ENCODER       = 14
DAWN:FRONT_DOOR_STATUS_ID     = OPEN

/*   IMAGE ACQUISITION OPTIONS   */

DAWN:DATA_ROUTING_ID          = "Science 2"
EXPOSURE_DURATION             = 1000.000 <millisecond>
DAWN:USE_PRE_CLEAR            = OFF
DAWN:IMAGE_ACQUIRE_MODE       = NORMAL
DAWN:CALLAMP_STROBE_TIME      = "N/A"
DAWN:CALLAMP_DELAY_TIME       = "N/A"
DAWN:CALLAMP_FREQUENCY        = "N/A"
DAWN:CALLAMP_DUTY             = "N/A"

/*   DATA COMPRESSION   */

PIXEL_AVERAGING_WIDTH         = 1
PIXEL_AVERAGING_HEIGHT        = 1
INST_CMPRS_TYPE               = "NOTCOMP"
INST_CMPRS_NAME               = "N/A"

/*   SYSTEM HEATER STATUS   */


/*   POWER CONVERTER SWITCH STATUS   */

DAWN:FEE_FLAG                 = ON
DAWN:HEATER0_FLAG             = ON
DAWN:HEATER1_FLAG             = ON
DAWN:CALLAMP_ENABLE_FLAG      = OFF
DAWN:MCU_MOTOR_POWER_FLAG     = ON
DAWN:MCU_FLAG                 = ON
DAWN:FSA_SHOOT_FLAG           = OFF
DAWN:FSA_SHOOT_ENABLE_FLAG    = OFF

/*   POWER SYSTEM STATUS   */

DAWN:V_28                     = 30.680 <volt>
DAWN:V_16                     = 16.099 <volt>
DAWN:V_12                     = 12.056 <volt>
DAWN:V_5                      = 5.175 <volt>
DAWN:V_M5                     = -5.049 <volt>
DAWN:V_5_ANALOG               = 5.261 <volt>
DAWN:V_M5_ANALOG              = -5.212 <volt>
DAWN:V_3_3                    = 3.380 <volt>
DAWN:V_2_5                    = 2.538 <volt>
DAWN:I_28                     = 646.400 <milliampere>
DAWN:I_16                     = 61.911 <milliampere>
DAWN:I_12                     = 71.480 <milliampere>
DAWN:I_5                      = 297.900 <milliampere>
DAWN:I_M5                     = -88.395 <milliampere>
DAWN:I_5_ANALOG               = 466.800 <milliampere>
DAWN:I_M5_ANALOG              = -28.800 <milliampere>
DAWN:I_3_3                    = 246.000 <milliampere>
DAWN:I_2_5                    = 640.650 <milliampere>

/*   CALIBRATED TEMPERATURES   */

DAWN:T_CCD                    = 222.888 <kelvin>
DAWN:T_DPU                    = 304.736 <kelvin>
DAWN:T_DCDC                   = 289.035 <kelvin>
DAWN:T_F12                    = 293.881 <kelvin>
DAWN:T_CSC                    = 294.669 <kelvin>
DAWN:T_COVER_MOTOR            = 245.292 <kelvin>
DAWN:T_LENS_BARREL            = 249.282 <kelvin>
DAWN:T_BAFFLE                 = 242.297 <kelvin>
DAWN:T_FILTER_MOTOR           = 256.265 <kelvin>
DAWN:T_STRUCTURE              = 255.267 <kelvin>
DAWN:T_RAD_MOTOR              = 201.400 <kelvin>

/*   TEST SETUP CONFIGURATION   */

DAWN:PURPOSE                  = "N/A"
DAWN:OPERATOR                 = "N/A"
DAWN:SUBJECT                  = "N/A"
DAWN:TEST_LAMP                = "N/A"
DAWN:TARGET                   = "N/A"
DAWN:CHAMBER                  = "N/A"

/*   POINTING   */

RIGHT_ASCENSION               = 2.289 <degree>
DECLINATION                   = 0.810 <degree>
TWIST_ANGLE                   = 4.315 <degree>
CELESTIAL_NORTH_CLOCK_ANGLE   = 0.000 <degree>
QUATERNION                    = (
    0.5434590680
    ,0.3615065212
    ,-0.0837556380
    ,-0.7529610014
)

/*   SPICE Kernels   */

SPICE_FILE_NAME               = (
    "sclk\DAWN_203_SCLKSCET.00009.tsc"
    ,"lsk\naif0009.tls"
    ,"spk\dawn_rec_071201-080205_081218_v1.bsp"
    ,"ck\dawn_sc_071210_071216.bc"
    ,"fk\dawn_v05.tf"
    ,"spk\de421.bsp"
    ,"pck\pck00008.tpc"
    ,"spk\de421.bsp"
    ,"pck\pck00008.tpc"
    ,"spk\de421.bsp"
    ,"spk\de421.bsp"
)

/*   COORDINATE SYSTEM   */

COORDINATE_SYSTEM_NAME        = "N/A"
COORDINATE_SYSTEM_CENTER_NAME = "N/A"

/*   Orbit geometry   */

TARGET_NAME                   = "N/A"
TARGET_TYPE                   = "N/A"
SUB_SPACECRAFT_LATITUDE       = "N/A"
SUB_SPACECRAFT_LONGITUDE      = "N/A"
SUB_SPACECRAFT_AZIMUTH        = "N/A"
SPACECRAFT_ALTITUDE           = "N/A"
TARGET_CENTER_DISTANCE        = "N/A"
ORBIT_NUMBER                  = "N/A"
SC_TARGET_POSITION_VECTOR     = (
    "N/A"
    ,"N/A"
    ,"N/A"
)
SC_TARGET_VELOCITY_VECTOR     = (
    "N/A"
    ,"N/A"
    ,"N/A"
)
LOCAL_HOUR_ANGLE              = "N/A"
SUB_SOLAR_LATITUDE            = "N/A"
SUB_SOLAR_LONGITUDE           = "N/A"
SUB_SOLAR_AZIMUTH             = "N/A"
SOLAR_LONGITUDE               = "N/A"
SOLAR_ELONGATION              = "N/A"

/*   Solar geometry   */

SPACECRAFT_SOLAR_DISTANCE     = 170310065.025 <kilometer>
SC_SUN_POSITION_VECTOR        = (
    -33052968.176 <kilometer>
    ,-152634756.157 <kilometer>
    ,-67938580.766 <kilometer>
)
SC_SUN_VELOCITY_VECTOR        = (
    27.4272710087 <kilometer per second>
    ,-10.3931746345 <kilometer per second>
    ,-4.6676014582 <kilometer per second>
)

/*   Illumination   */

INCIDENCE_ANGLE               = "N/A"
EMISSION_ANGLE                = "N/A"
PHASE_ANGLE                   = "N/A"

/*   Image parameters   */

SAMPLE_DISPLAY_DIRECTION      = ""
LINE_DISPLAY_DIRECTION        = ""
SLANT_DISTANCE                = "N/A"
MINIMUM_LATITUDE              = "N/A"
CENTER_LATITUDE               = "N/A"
MAXIMUM_LATITUDE              = "N/A"
WESTERNMOST_LONGITUDE         = "N/A"
CENTER_LONGITUDE              = "N/A"
EASTERNMOST_LONGITUDE         = "N/A"
HORIZONTAL_PIXEL_SCALE        = "N/A"
VERTICAL_PIXEL_SCALE          = "N/A"
RETICLE_POINT_RA              = (
)
RETICLE_POINT_DECLINATION     = (
)
RETICLE_POINT_LONGITUDE       = (
)
RETICLE_POINT_LATITUDE        = (
)
NORTH_AZIMUTH                 = "N/A"
SMEAR_AZIMUTH                 = "N/A"
SMEAR_MAGNITUDE               = "N/A"
LIMB_ANGLE                    = "N/A"

/* IMAGE OBJECT */

OBJECT                        = IMAGE
    INTERCHANGE_FORMAT        = BINARY
    LINE_SAMPLES              = 1024
    LINES                     = 1024
    BANDS                     = 1
    SAMPLE_BITS               = 16
    SAMPLE_TYPE               = "LSB_UNSIGNED_INTEGER"
    FIRST_LINE                = 17
    FIRST_LINE_SAMPLE         = 35
    UNIT                      = "DU"
END_OBJECT                    = IMAGE


/* FRAME_2_IMAGE OBJECT */

OBJECT                        = FRAME_2_IMAGE
    INTERCHANGE_FORMAT        = BINARY
    LINE_SAMPLES              = 10
    LINES                     = 1054
    BANDS                     = 1
    SAMPLE_BITS               = 16
    SAMPLE_TYPE               = "LSB_UNSIGNED_INTEGER"
    FIRST_LINE                = 2
    FIRST_LINE_SAMPLE         = 2
    UNIT                      = "DU"
END_OBJECT                    = FRAME_2_IMAGE


/* FRAME_3_IMAGE OBJECT */

OBJECT                        = FRAME_3_IMAGE
    INTERCHANGE_FORMAT        = BINARY
    LINE_SAMPLES              = 8
    LINES                     = 1054
    BANDS                     = 1
    SAMPLE_BITS               = 16
    SAMPLE_TYPE               = "LSB_UNSIGNED_INTEGER"
    FIRST_LINE                = 2
    FIRST_LINE_SAMPLE         = 16
    UNIT                      = "DU"
END_OBJECT                    = FRAME_3_IMAGE


/* FRAME_4_IMAGE OBJECT */

OBJECT                        = FRAME_4_IMAGE
    INTERCHANGE_FORMAT        = BINARY
    LINE_SAMPLES              = 1024
    LINES                     = 8
    BANDS                     = 1
    SAMPLE_BITS               = 16
    SAMPLE_TYPE               = "LSB_UNSIGNED_INTEGER"
    FIRST_LINE                = 3
    FIRST_LINE_SAMPLE         = 35
    UNIT                      = "DU"
END_OBJECT                    = FRAME_4_IMAGE


/* FRAME_5_IMAGE OBJECT */

OBJECT                        = FRAME_5_IMAGE
    INTERCHANGE_FORMAT        = BINARY
    LINE_SAMPLES              = 1024
    LINES                     = 8
    BANDS                     = 1
    SAMPLE_BITS               = 16
    SAMPLE_TYPE               = "LSB_UNSIGNED_INTEGER"
    FIRST_LINE                = 1047
    FIRST_LINE_SAMPLE         = 35
    UNIT                      = "DU"
END_OBJECT                    = FRAME_5_IMAGE

END

[warmerdam]

I encountered three immediate problems, fixed in trunk (r19475)

• support quoted SAMPLE_TYPE.
• Check for UNSIGNED in SAMPLE_TYPE for UInt16
• support files where scanlines are broken over several records.

Note that we still only support the first image while the file contains several other pseudo-image products.

This image has been placed at:

​http://download.osgeo.org/gdal/data/pds/FC11A0000338_07344220211F2C.IMG

for reference.

[michaelaye]

A suggestion/question: If it is true, that PDS does not support bands anyway, could the other binaries (outer rims of the CCD with different lighting conditions, e.g. refractive light, only bias etc.) maybe support by the driver as bands?

[warmerdam]

Michael,

We can't represent these auxiliary products as bands because they are not the same size as the base image. If we want to support them we would need to go to the subdataset approach used for hdf, netcdf, etc. This would be fairly involved.