Adding to JP2KAK support for Kakadu -num_threads

[Coatman]

When writing a .jp2 file, GDAL's Kakadu-based JP2KAK plugin provides through jp2kakdataset.cpp a mechanism for users to specify parameters that control creating the .jp2 file, including

QUALITY   =n to specify the size   of the compressed   output .jp2
BLOCKXSIZE=n to specify the width  of the tiles in the output .jp2
BLOCKYSIZE=n to specify the height of the tiles in the output .jp2
LAYERS    =n to specify the number of layers    in the output .jp2

A complete list of all 15 supported parameters is available at ​http://www.gdal.org/frmt_jp2kak.html

Beginning with Kakadu version 5.1, released 6 March 2006, Kakadu has supported multi-threaded processing via -num_threads. For example, on computers with two or more processors, a compressed .jp2 will be translated to an uncompressed .tif, or to pixels for display, almost twice as fast with kdu_expand -i compressed_image_in.jp2 -o uncompressed_image_out.tif -num_threads 2. Performance on computers with only one processor does not improve.

If it can be accomplished without a major rewrite of jp2kakdataset.cpp, then I suggest consideration be given to adding in a future version support for multi-threaded Kakadu by passing to Kakadu -num_threads, and that the default be -num_threads 2.

[warmerdam]

I believe this will require quite a bit of restructuring of the driver.

[Coatman]

Kakadu's David Taubman offers this guidance.

Taking a quick look at the jp2kakdataset.cpp code, the main function which needs to be changed is JP2KAKCreateCopy_WriteTile. Basically, you need to instantiate a kdu_thread_env object outside this function, before the tile-by-tile loop which calls JP2KAKCreateCopy_WriteTile. You add threads to the kdu_thread_env object, using its member functions -- very simple. You then pass a pointer to this object into JP2KAKCreateCopy_WriteTile. I typically call that pointer env in all my demo apps and interface declarations. You can then pass the env argument to the various Kakadu functions which accept it as an option, such as the kdu_analysis() constructor, the push function, and so forth. Finally, you must execute a synchronization call on the kdu_thread_env object before you can close the tile or the tile processing engines, to wait for all work to complete. This is the basic structure.

Perhaps the best example to look into is to examine kdu_compress where you will find two functions that can be used to compress: one for single-threaded and the other for multi-threaded compression. They are very similar, so it should not be too hard to see what is going on by comparing them.

Unfortunately, the GDAL is using very low level Kakadu calls, since it is based on a very old version of kdu_compress. A better approach in general is to do everything through kdu_multi_analysis and kdu_multi_synthesis and then you can just cut and paste the code from kdu_compress (eliminating the complex incremental flushing code).

[Jukka Rahkonen]

Seems to be implemented in current JP2KAK driver: JP2KAK_THREADS=n: By default an effort is made to take advantage of multi-threading on multi-core computers using default rules from the Kakadu library. This option may be set to a value of zero to avoid using additional threads or to a specific count to create the requested number of worker threads.

Ticket can be closed.