Creation of GMLJP2 files with JP2OpenJPEG following INSPIRE TG on orthoimagery

[phasenohr]

It would be a very useful feature in GDAL to be able:

• to create GMLJP2 data sets which conform to the requirements and recommendations contained in the Annex E.3 of the INSPIRE Technical Guidelines on orthoimagery (document D2.8.II.3 Data Specification on Orthoimagery - Technical Guidelines available at ​http://inspire.jrc.ec.europa.eu/documents/Data_Specifications/INSPIRE_DataSpecification_OI_v3.0.pdf)
• to check the conformity of a GMLJP2 file against the specifications of Annex E.3 of the INSPIRE TG on orthoimagery.

[Jukka Rahkonen]

Are you aware of some issues with reaching INSPIRE requirements with OpenJPEG driver? Do you know how to fulfill them with the GDAL JP2KAK driver or with the kdu_compress demo utility? If you point where to look I can perhaps make some tests.

[phasenohr]

Replying to jratike80:

Are you aware of some issues with reaching INSPIRE requirements with OpenJPEG driver?

They may not necessarily be linked with the OpenJPEG driver. For example, currently it is not possible to create files with a number of bands different from 1 or 3 (#5695). The TG place additional constraints (identified as requirements or recommendations in the document) to the structure of the file: e.g. all bands must be of the same type, the alpha band must be used only as a binary transparency mask, no ROI allowed, precincts to be used. On top of that comes a specific requirement regarding the use of the JP2 markers (which may well be matching the way it is already done, I don't know.)

Do you know how to fulfill them with the GDAL JP2KAK driver or with the kdu_compress demo utility? If you point where to look I can perhaps make some tests.

I do not have any experience with kakadu. GDAL + OpenJPEG + INSPIRE is something I feel is important to make the creation of INSPIRE conformant orthoimages easier for a large community (no need for a 3rd party licence and some other INSPIRE themes propose the use of GMLJP2).

[Jukka Rahkonen]

By reading annex E.3 the technical requirements does not seem to contain anything special and I believe that OpenJPEG driver creates compliant JPEG2000 files with 1 and 3 bands. However, by reading the specifications it is not so easy to be sure. It would be very nice if the INSPIRE working group could make a set of official reference images for each type listed in the document:

"following types are addressed: bilevel, greyscale, palette-colour, full-colour images (known as RGB images) and multispectral images. So the image data of a JPEG 2000 file can contain either 1 (bilevel, greyscale and palette-colour), 3 bands (RGB) or more bands (multispectral images). An additional band for opacity may also be used."

First part of this ticket is that GDAL should be able to create INSPIRE compliant datasets (why to limit it to GMLJP2 when "plain JPEG2000" with GeoJP2 (tm) uuid box is as well accepted). As a ticket I think this is rather wide one. It would be best to investigate and find out cases when GDAL fails in making a compliant result and create one ticket per issue. This enhancement ticket could have links those well defined issues.

Another part of ticket suggest to make an INSPIRE compliance checker utility. I feel that, despite being useful for us who have to deal with INSPIRE, it does not belong to the scope of GDAL. Vast amounts of totally valid JPEG2000 images are not INSPIRE compliant. I would say that the INSPIRE orthoimagery working group should rather make such a tool (perhaps with GDAL) and make it downloadable from the INSPIRE web site ​http://inspire.ec.europa.eu/

What we could have is a Best practice document for INSPIRE compliant orthophotos in the GDAL wiki. Basically if would contain correct gdal_translate parameters for each JPEG2000 driver and image type: bilevel, greyscale, palette-colour, full-colour, and multispectral.

[Even Rouault]

trunk r28617 "OpenJPEG: implement INSPIRE_TG, PROFILE, JPX and GEOBOXES_AFTER_JP2C creation option (#5697)"

[Jukka Rahkonen]

May I ask if you are testing the right thing with # Error case: too big tile?

Inspire guidance says:

Internal tiling JPEG 2000 allows internal tiling within the codestream. Profile 1 of ISO 15444 - 1 already requires no tiling (i.e. the image = 1 tile) or tiling with tiles size bigger than 1024x1024 pixels. There is no further requirement

[Even Rouault]

Replying to jratike80:

May I ask if you are testing the right thing with # Error case: too big tile?

Inspire guidance says:

Internal tiling JPEG 2000 allows internal tiling within the codestream. Profile 1 of ISO 15444 - 1 already requires no tiling (i.e. the image = 1 tile) or tiling with tiles size bigger than 1024x1024 pixels. There is no further requirement

I believe it's an error in the TG. It should read "tiles size NO bigger than". Table A-45 of "​http://www.astro.cornell.edu/~carcich/LRO/jp2/ISO_JPEG200_Standard/INCITS+ISO+IEC+15444-1-2000+AM1-2002.pdf" shows a <= 1024 test. The purposes of the restricted profiles is to not require too much complication from "basic" readers, so asking for a minimum tile size doesn't make sense (although I'm not sure how a single tile >> 1024 would be much better...)

[Jukka Rahkonen]

All right. NITF, if I remember right, has also max. limit at 1024x1024. With Kakadu library the bigger tiles the better if reasonable progression order and good set of precincts are used. With lossy compression there tends to come artifacts at the seems of the tiles. However, some JPEG2000 libraries can be hopeless with big tiles or no tiles at all with a typical sized geospatial images.

[Even Rouault]

r28624 "OpenJPEG: add PRECINCTS creation option, and make it default to {512,512},{256,512},{128,512},{64,512},{32,512},{16,512},{8,512},{4,512},{2,512} (#5697)"

[Even Rouault]

r28628 "OpenJPEG: allow several quality values to be specified with QUALITY creation option. Add TILEPARTS, CODEBLOCK_WIDTH and CODEBLOCK_HEIGHT options (#5697)"

[Even Rouault]

r28629 "OpenJPEG: support writing images with a color table (#5697)"

[Even Rouault]

r28631 "OpenJPEG: support reading&writing color table with alpha component (#5697)"

[Even Rouault]

r28635 "OpenJPEG: support reading&writing datasets with unusual order of band color interpretation (#5697)"

[Even Rouault]

r28639 "OpenJPEG: cleanly reject datasets whose dimensions are greater than 31 bits (#5697)"

[Even Rouault]

r28647 "OpenJPEG: refuse creating file with more than 16384 bands, and be more restrictive relating to metadata to avoid creating .aux.xml file when not necessarily (#5697)"

[Even Rouault]

r28657 "OpenJPEG: do not expose overviews when there is a color table, since we need full lossless decoding to produce a meaningful image (#5697)"

[Even Rouault]

r28660 "OpenJPEG: add WRITE_METADATA and MAIN_MD_DOMAIN_ONLY creation options to write GDAL metadata, JP2 XML boxes or XMP box (#5697)"

[Even Rouault]

r28668 "GMLJP2: when parsing a GMLJP2 box, accept srsName found on gml:RectifiedGrid if not found on origin.Point, so as to be compatible with the example of DGIWG_Profile_of_JPEG2000_for_Georeferenced_Imagery.pdf (#5697)"

[Even Rouault]

r28677 "OpenJPEG: support update mode for editing metadata and georeferencing (#5697)"

[Even Rouault]

r28681 "OpenJPEG: add support for reading/writing/updating IPR box (from/into xml:IPR metadata domain) (#5697)"

[Even Rouault]

r28684 "Add embryonic GDALGetJPEG2000Structure() and map it to SWIG as gdal.GDALGetJPEG2000Structure() and gdal.GetJPEG2000StructureAsString() (#5697)"

[Even Rouault]

r28691 "Enhance GDALGetJPEG2000Structure() output (WIP) (#5697)"

[Even Rouault]

r28694 "GDALGetJPEG2000Structure(): decode all sub-boxes of jp2h super-box (#5697)"

[Even Rouault]

r28695 "GDALGetJPEG2000Structure(): decode rreq box (#5697)"

[Even Rouault]

r28705 "GDALGetJPEG2000Structure(): dump codestream structure (#5697)"

[Even Rouault]

r28761 "dump_jp2.py: add a -dump_gmljp2 option (#5697)"

r28762 "Add a swig/python/samples/validate_jp2.py utility (#5697)"

[Even Rouault]

r28766 "Improve tests done by validate_jp2.py (#5697)"

[Even Rouault]

r28767 "GMLJP2/SRS export as GML: output XML definition of a SRS as a GML 3.1.1 compliant Dictionary (#5697)"

[Even Rouault]

r28768 "dump_jp2.py: add a -dump_crsdictionary option (#5697)"

[Even Rouault]

r28770 "OpenJPEG: in Inspire compatible mode, use 3 components per entry of color table (#5697)"

r28771 "validate_jp2.py: validate GeoJP2 vs GMLJP2 consistency, use Orthoimagery document to do extra validations, and other misc fixes (#5697)"

[Even Rouault]

r28773 "validate_jp2.py: be robust to failure of gdal.Open(), use JP2OpenJPEG driver preferably, check embedded GeoTIFF box raster dimensions (#5697)"

[Even Rouault]

r28774 "validate_jp2.py: correctly deal with multi-band OrthoimageryCoverage document (#5697)"

[Even Rouault]

r28777 "jp2openjpeg: validate files with color table and metadata (#5697)"

[Even Rouault]

r28783 "OpenJPEG: add a word of caution about compatibility problems with 1-bit alpha channel (#5697)" r28784 "validate_jp2.py: add a -datatype imagery|non_imagery option so as to know which bit-depths are allowed when doing Inspire checks (#5697)"

[Even Rouault]

r28830 "Add build_jp2_from_xml.py script to help producing corrupted JPEG2000 files; Add test_validate_jp2.py to test the validate_jp2 script (#5697)"

[Even Rouault]

r29210 "GMLJP2: write non null bounding box at root of FeatureCollection (#5697)"

[Even Rouault]

Milestone renamed