Opened 19 months ago

Closed 13 months ago

Last modified 13 months ago

#6293 closed defect (fixed)

Multiple Pleiades/Airbus Tiles Don't Georegister Properly With gdalwarp

Reported by: edjansing Owned by: warmerdam
Priority: high Milestone: 2.0.3
Component: default Version: unspecified
Severity: major Keywords: gdalwarp, tiles, tfw, geotiff, georegistration
Cc: bishop

Description (last modified by edjansing)

Attempting to use gdalwarp to georegister several MSI tiles from Airbus' Pleiades sensor. A single area is provided in several tiles. In this example, here is the MSI directory listing with two tiles:

DIM_PHR1A_MS_201509141606268_SEN_1627093101-002.XML
ICON_PHR1A_MS_201509141606268_SEN_1627093101-002.JPG
IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R1C1.TFW
IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R1C1.TIF
IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R2C1.TFW
IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R2C1.TIF
INDEX.HTM
LIBRARY
LINEAGE
MASKS
PREVIEW_PHR1A_MS_201509141606268_SEN_1627093101-002.JPG
PREVIEW_PHR1A_MS_201509141606268_SEN_1627093101-002.KMZ
RPC_PHR1A_MS_201509141606268_SEN_1627093101-002.XML

The TFW files are unique between the two tiles. When I import the tiles into qgis, without any modification, it stitches them together properly, but they are not in the proper CRS (they are essentially referenced to pixel space, not to any CRS coordinate system).

When I use gdalwarp to put them in EPSG:4326, the first image is perfectly registered (the image with R1C1 as a suffix). The command used:

gdalwarp -rpc IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R1C1.TIF IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R1C1_projected.TIF

When I then gdalwarp the second image:

gdalwarp -rpc IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R2C1.TIF IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R2C1_projected.TIF

The second image does not get stitched at the bottom of the first image, but is placed at the top of the image, as if the tile starts at the same lat/lon coordinate (or at the same pixel location, whichever).

Is this normal behavior? Is there another procedure I should use? Can provide image data if helpful.

gdalwarp reports 2.0.0 as version gdalinfo on two original images:

$ gdalinfo IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R1C1.TIF
Driver: GTiff/GeoTIFF
Files: IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R1C1.TIF
       IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R1C1.TIF.aux.xml
       ./DIM_PHR1A_MS_201509141606268_SEN_1627093101-002.XML
       ./RPC_PHR1A_MS_201509141606268_SEN_1627093101-002.XML
       IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R1C1.TFW
Size is 9458, 15000
Coordinate System is `'
Origin = (-1.000000000000000,3.000000000000000)
Pixel Size = (4.000000000000000,-4.000000000000000)
Metadata:
  METADATATYPE=DIMAP
  TIFFTAG_DATETIME=2015:12:23 18:32:39
  TIFFTAG_IMAGEDESCRIPTION=B2 B1 B0 B3
  TIFFTAG_XRESOLUTION=1
  TIFFTAG_YRESOLUTION=1
Image Structure Metadata:
  INTERLEAVE=PIXEL
RPC Metadata:
  HEIGHT_OFF=45
  HEIGHT_SCALE=95
  LAT_OFF=38.87154971192919
  LAT_SCALE=0.1416732123880315
  LINE_DEN_COEFF= 1 0.00026341144632701 -9.80822136398733e-05 -2.35953356209866e-06 -1.62170380697022e-05 -1.22640028446246e-08 1.55023313626127e-07 -1.5537111237091e-05 4.58036130936434e-05 -2.31198284873618e-05 3.2864533726195e-09 -3.06927464009735e-08 2.08793797077866e-06 -1.21475710839433e-08 1.54788420545811e-07 -4.44599257156653e-07 1.70368489248657e-09 1.33031932377729e-10 -1.54424096736091e-08 1.05222239987211e-10
  LINE_NUM_COEFF= -0.000119096471843013 0.00273796193211152 -1.00367656228961 -0.000859086493278233 0.000261713135774831 2.33979663689761e-07 -2.25402196362394e-06 -0.000340119441043266 2.19730836249524e-05 1.66642358408085e-08 -4.29909520634416e-08 4.95298240422465e-08 -2.62364432140933e-05 -6.28870839700389e-08 1.52651889380179e-05 8.08563520366722e-05 2.3195367391933e-05 1.1804689412366e-08 1.00010692110933e-07 1.98020806117544e-08
  LINE_OFF=7772.5
  LINE_SCALE=7771.5
  LONG_OFF=-77.13546672147527
  LONG_SCALE=0.1165198796426168
  SAMP_DEN_COEFF= 1 0.000472894792539169 0.000881419369988542 -7.53324742935084e-05 -3.46629852664066e-05 -4.87301506925132e-08 1.32614018054834e-08 -2.1872832721835e-07 1.49479836237352e-05 -6.59860144983547e-06 -1.48728353597154e-08 -3.52095449470832e-08 -2.14688694596795e-07 -3.3168465870712e-09 4.03572496745229e-08 3.97707771445365e-07 -1.15020506050241e-08 2.66109369887284e-09 1.05965377023676e-08 9.65442134520645e-10
  SAMP_NUM_COEFF= -0.000434608495681351 1.00210264963283 -0.000281300697070519 0.000316194516045025 -0.000879304997048266 7.52970914458278e-05 -5.20633271426872e-05 -3.73769088263009e-05 0.00138775409674169 -2.73849487440847e-08 5.19133691835542e-08 -1.24502711479507e-05 -2.1363347686686e-05 -6.59563969341986e-06 5.20138704377129e-05 -3.41383248102939e-05 -2.02958513833057e-09 -3.71155366535213e-08 -2.25289299184377e-07 -2.06504369348464e-09
  SAMP_OFF=5000.5
  SAMP_SCALE=4999.5
Corner Coordinates:
Upper Left  (  -1.0000000,   3.0000000)
Lower Left  (      -1.000,  -59997.000)
Upper Right (   37831.000,       3.000)
Lower Right (   37831.000,  -59997.000)
Center      (   18915.000,  -29997.000)
Band 1 Block=9458x1 Type=UInt16, ColorInterp=Red
  Min=0.000 Max=3924.000
  Minimum=0.000, Maximum=3924.000, Mean=241.479, StdDev=247.675
  Metadata:
    STATISTICS_MAXIMUM=3924
    STATISTICS_MEAN=241.47873181735
    STATISTICS_MINIMUM=0
    STATISTICS_STDDEV=247.67541356986
Band 2 Block=9458x1 Type=UInt16, ColorInterp=Green
  Min=0.000 Max=3956.000
  Minimum=0.000, Maximum=3956.000, Mean=300.351, StdDev=253.536
  Metadata:
    STATISTICS_MAXIMUM=3956
    STATISTICS_MEAN=300.35078661846
    STATISTICS_MINIMUM=0
    STATISTICS_STDDEV=253.53558755775
Band 3 Block=9458x1 Type=UInt16, ColorInterp=Blue
  Min=0.000 Max=4095.000
  Minimum=0.000, Maximum=4095.000, Mean=348.032, StdDev=266.569
  Metadata:
    STATISTICS_MAXIMUM=4095
    STATISTICS_MEAN=348.03234066915
    STATISTICS_MINIMUM=0
    STATISTICS_STDDEV=266.56884586952
Band 4 Block=9458x1 Type=UInt16, ColorInterp=Undefined
$ gdalinfo IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R2C1.TIF
Driver: GTiff/GeoTIFF
Files: IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R2C1.TIF
       IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R2C1.TIF.aux.xml
       ./DIM_PHR1A_MS_201509141606268_SEN_1627093101-002.XML
       ./RPC_PHR1A_MS_201509141606268_SEN_1627093101-002.XML
       IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R2C1.TFW
Size is 9458, 544
Coordinate System is `'
Origin = (-1.000000000000000,-59997.000000000000000)
Pixel Size = (4.000000000000000,-4.000000000000000)
Metadata:
  METADATATYPE=DIMAP
  TIFFTAG_DATETIME=2015:12:23 18:32:39
  TIFFTAG_IMAGEDESCRIPTION=B2 B1 B0 B3
  TIFFTAG_XRESOLUTION=1
  TIFFTAG_YRESOLUTION=1
Image Structure Metadata:
  INTERLEAVE=PIXEL
RPC Metadata:
  HEIGHT_OFF=45
  HEIGHT_SCALE=95
  LAT_OFF=38.87154971192919
  LAT_SCALE=0.1416732123880315
  LINE_DEN_COEFF= 1 0.00026341144632701 -9.80822136398733e-05 -2.35953356209866e-06 -1.62170380697022e-05 -1.22640028446246e-08 1.55023313626127e-07 -1.5537111237091e-05 4.58036130936434e-05 -2.31198284873618e-05 3.2864533726195e-09 -3.06927464009735e-08 2.08793797077866e-06 -1.21475710839433e-08 1.54788420545811e-07 -4.44599257156653e-07 1.70368489248657e-09 1.33031932377729e-10 -1.54424096736091e-08 1.05222239987211e-10
  LINE_NUM_COEFF= -0.000119096471843013 0.00273796193211152 -1.00367656228961 -0.000859086493278233 0.000261713135774831 2.33979663689761e-07 -2.25402196362394e-06 -0.000340119441043266 2.19730836249524e-05 1.66642358408085e-08 -4.29909520634416e-08 4.95298240422465e-08 -2.62364432140933e-05 -6.28870839700389e-08 1.52651889380179e-05 8.08563520366722e-05 2.3195367391933e-05 1.1804689412366e-08 1.00010692110933e-07 1.98020806117544e-08
  LINE_OFF=7772.5
  LINE_SCALE=7771.5
  LONG_OFF=-77.13546672147527
  LONG_SCALE=0.1165198796426168
  SAMP_DEN_COEFF= 1 0.000472894792539169 0.000881419369988542 -7.53324742935084e-05 -3.46629852664066e-05 -4.87301506925132e-08 1.32614018054834e-08 -2.1872832721835e-07 1.49479836237352e-05 -6.59860144983547e-06 -1.48728353597154e-08 -3.52095449470832e-08 -2.14688694596795e-07 -3.3168465870712e-09 4.03572496745229e-08 3.97707771445365e-07 -1.15020506050241e-08 2.66109369887284e-09 1.05965377023676e-08 9.65442134520645e-10
  SAMP_NUM_COEFF= -0.000434608495681351 1.00210264963283 -0.000281300697070519 0.000316194516045025 -0.000879304997048266 7.52970914458278e-05 -5.20633271426872e-05 -3.73769088263009e-05 0.00138775409674169 -2.73849487440847e-08 5.19133691835542e-08 -1.24502711479507e-05 -2.1363347686686e-05 -6.59563969341986e-06 5.20138704377129e-05 -3.41383248102939e-05 -2.02958513833057e-09 -3.71155366535213e-08 -2.25289299184377e-07 -2.06504369348464e-09
  SAMP_OFF=5000.5
  SAMP_SCALE=4999.5
Corner Coordinates:
Upper Left  (      -1.000,  -59997.000)
Lower Left  (      -1.000,  -62173.000)
Upper Right (   37831.000,  -59997.000)
Lower Right (   37831.000,  -62173.000)
Center      (   18915.000,  -61085.000)
Band 1 Block=9458x1 Type=UInt16, ColorInterp=Red
  Min=0.000 Max=3131.000
  Minimum=0.000, Maximum=3131.000, Mean=303.059, StdDev=222.224
  Metadata:
    STATISTICS_MAXIMUM=3131
    STATISTICS_MEAN=303.05929724396
    STATISTICS_MINIMUM=0
    STATISTICS_STDDEV=222.22368384095
Band 2 Block=9458x1 Type=UInt16, ColorInterp=Green
  Min=0.000 Max=3071.000
  Minimum=0.000, Maximum=3071.000, Mean=389.381, StdDev=199.449
  Metadata:
    STATISTICS_MAXIMUM=3071
    STATISTICS_MEAN=389.38147159371
    STATISTICS_MINIMUM=0
    STATISTICS_STDDEV=199.44912482837
Band 3 Block=9458x1 Type=UInt16, ColorInterp=Blue
  Min=0.000 Max=3020.000
  Minimum=0.000, Maximum=3020.000, Mean=453.803, StdDev=187.787
  Metadata:
    STATISTICS_MAXIMUM=3020
    STATISTICS_MEAN=453.8029269754
    STATISTICS_MINIMUM=0
    STATISTICS_STDDEV=187.78652664007
Band 4 Block=9458x1 Type=UInt16, ColorInterp=Undefined

TFW files:

$ cat IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R1C1.TFW
4.00000000000000
0
0
-4.00000000000000
1.00000000000000
1.00000000000000
$ cat IMG_PHR1A_MS_201509141606268_SEN_1627093101-002_R2C1.TFW
4.00000000000000
0
0
-4.00000000000000
1.00000000000000
-59999.00000000000000

Attachments (2)

2015-01-06_stitchedTogetherProperlyButNotInRightCRS.png (625.9 KB) - added by edjansing 19 months ago.
Properly stitched, not in CRS
2015-01-06_stitchedTogetherInCorrectlyButInRightCRS.png (836.4 KB) - added by edjansing 19 months ago.
Improperly stitched, but in EPSG:4326 CRS

Download all attachments as: .zip

Change History (19)

Changed 19 months ago by edjansing

Properly stitched, not in CRS

Changed 19 months ago by edjansing

Improperly stitched, but in EPSG:4326 CRS

comment:1 Changed 19 months ago by Even Rouault

Two things :

  • You should pass the 2 images in the same gdalwarp command "gdalwarp -rpc in1.tif in2.tif out.tif". If you do in several times, it will not increase the raster dimension & georeferenced extent of the first output.
  • You could try removing the .TFW files as I'm wondering if they don't confuse gdalwarp

comment:2 Changed 19 months ago by edjansing

Tried the following:

  • Passing both images into the same gdalwarp command. No improvement.
  • Passing both images into the same gdalwarp command, without the TFW files. No improvement.
  • Processing each image separately, without the TFW files. No improvement.

comment:3 Changed 19 months ago by edjansing

Used:

gdalwarp -rpc -dstnodata 0 -r cubic first.tif second.tif output.tif

comment:4 Changed 19 months ago by edjansing

Description: modified (diff)

comment:5 Changed 19 months ago by Even Rouault

Just to be sure: did you think of removing the output file or using -overwrite

comment:6 Changed 19 months ago by edjansing

I destroyed (rm output.tif) the output file after each try.

comment:7 Changed 19 months ago by Even Rouault

Cc: bishop added

Hum, there's something weird in the gdalinfo output of the 2 images. The RPC values are strictly the same so it means the files should overlap perfectly.

Ah I see in the file list that they refer to the same RPC_PHR1A_MS_201509141606268_SEN_1627093101-002.XML which must explain the behaviour you see.

CC'ing Dmitry as I think this is related to the MDReader.

comment:8 Changed 19 months ago by Bishop

The Pleiades/Airbus? don't provide the RPC for each tile. So trying to warp each separate tile (in the case of multiple tiles) is wrong. You need to create the VRT file for such tiled image and add RPC into Metadata section.

Unfortunately there is no support out of the box for Pleiades tiled images like is done for SPOT (http://gdal.org/frmt_various.html#DIMAP) or DigitalGlobe? (https://github.com/OSGeo/gdal/blob/trunk/gdal/frmts/til/tildataset.cpp), Even maybe correct me here.

In Pleiades imagery samples, that I got from Airbus site there were no tiled imagery, so I add RPC to metadata. It can be easily removed, but I prefer to treat this situation (tile with RPC of the whole image) as follows - some additional information.

comment:9 Changed 19 months ago by edjansing

"You need to create the VRT file for such tiled image and add RPC into Metadata section."

OK. Can you point me to a link, wiki, or some other doc to guide me through that process? If that's the case, I'll try it by hand and then, if it works, then I'll build a python script to do it automatically for all of the AB/P data we get.

Last edited 19 months ago by edjansing (previous) (diff)

comment:10 Changed 19 months ago by Bishop

Here it is: http://gdal.org/gdal_vrttut.html

Also you can try this command line utility: http://gdal.org/gdalbuildvrt.html and than change it in any text editor.

comment:11 Changed 19 months ago by edjansing

After reading the tutorial and the RPC22 document (I found online), I think I have a notion about what needs to happen here. I'll give it a whirl over the weekend and get back with you on success/failure.

Last edited 19 months ago by edjansing (previous) (diff)

comment:12 Changed 19 months ago by Bishop

I tested the <Metadata> tag - unfortunately I cannot add metadata to the RPC Metadata domain (http://gdal.org/gdal_datamodel.html) of vrt file. It seems to me that the only way is to export the result vrt to some raster format (geotiff, jp2, etc.) supported by mdreader and put near the RPC in sonamed file in some supported format (RPB, RPC, etc).

comment:13 Changed 19 months ago by Even Rouault

Dmitry, here's an example of a VRT with RPC : https://svn.osgeo.org/gdal/trunk/autotest/gcore/data/rpc.vrt if that was your question

comment:14 Changed 19 months ago by Bishop

Yes, exactly! Shame on me.

comment:15 Changed 19 months ago by edjansing

Worked like a charm! I am building a Python script to do it automatically. I'm willing to share it, provided you tolerate my amateurish coding style. :) I'm still new to Python.

comment:16 Changed 13 months ago by Even Rouault

Milestone: 2.0.22.0.3
Resolution: fixed
Status: newclosed

trunk r34365, branches/2.1 r34367, branches/2.0 r34368 "DIMAP/Pleiades metadata reader: take into tiling to properly shift RPC (#6293)"

comment:17 Changed 13 months ago by Even Rouault

Related enhancement: trunk r34369 "DIMAP: add support for tiled DIMAP 2 datasets (somehow related to #6293)"

Note: See TracTickets for help on using tickets.