GDAL JP2KAK use irreversible for Quality > 75

[Coatman]

JPEG2000 provides for reversible (lossless) compression and irreversible (lossy) compression. JPEG2000 reversible compression always achieves compression equivalent to JP2KAK Quality=65, or less. It is possible to create an irreversibly (lossy) compressed image of Quality=>75, but there is no good reason to do so. I suggest serious consideration be given to changing line 2234 of 1.6/gdal/frmts/jp2kak/jp2kakdataset.cpp, which is used to specify an irreversible (lossy) compression, from if( dfQuality < 99.5 ) to if( dfQuality < 75.0 ). I see no negative consequences to this enhancement.

[warmerdam]

The QUALITY is really a target compression ratio, and I would like to have the actual compression ratio come close to that requested, or be greater. But if we were to treat values from 75 to 99 as irreversable there is no way we can provide assurance of achieving the desired compression ratio.

[Coatman]

There seems to be a lack of understanding that reversible JPEG2000 compression creates a file no larger than Quality=65, and so there is never any good reason to use Quality=85 to create a larger file, that is not a 100% perfect copy of the original. There can be no justification for creating an irreversible lossy compressed image of Quality=85, when a reversibly compressed file is always no larger than Quality=65. PLEASE reconsider your initial reaction, which is not supported by the nature of JPEG2000 compression.

[warmerdam]

Greg,

You seem to have said that "a reversibly compressed file is always no larger than Quality=65".

Are you suggesting that it is always possible to compress an image at 65% of it's original size and still be able to recreate the original image exactly? That is manifestly incorrect and would violate information theory.

[Coatman]

YES, indeed! That is the very nature of JPEG2000! That is one reason people deliberately choose JPEG2000.

kdu_compress -i image.tif -o image.jp2 Creversible=yes

will always create a reversibly compressed file no larger than a 65% of the uncompressed file, from which the original image can be perfectly recreated. That is why it is called a reversible compression: it stores a perfect copy of the original, in less space than the original. I have done this over a million times, checked the results, and has always worked. Your statement "That is manifestly incorrect" is wrong.

[warmerdam]

Greg,

Ah, proof by having tried it many times!

Well, luckily it only takes one example to disprove you. ​http://home.gdal.org/tmp/hard_to_compress.tif

This is an uncompressed TIFF file, with an image that has high information density. I think you will find it difficult to significantly compress losslessly. I would that kdu_compress produced a file about 8% larger than the original.

I am not interested in whether or not jpeg2000 can "typically" achieve compression to 65% of the original losslessly. The point is that it cannot always do so.

[Coatman]

Your bogus image was not taken by a camera through a lens and proves nothing. This fake image is just noise, without any signal information. It is still true that for any natural image taken by a camera through a lens, JPEG2000 will create a reversibly compressed file only 65% the size of the uncompressed image. Why do you try to deny the well publicized basic tenants of JPEG2000 reversible compression? Why handicap new users to GDAL?

For more than a year, you have actively and deliberately, in a destructive way, prevented any improvement to the 7 year old code in jp2kakdataset.cpp that delivers at best only 1/4 the performance of the free Kakadu applications available for Linux, Mac OS X, Solaris, and Windows. You said at one time that I needed to offer you more money, and then you would do the work, but after I offered you more money than you asked me for, then you just refused to do anything. Why do you spend so much time denying GDAL users code that is less than 7 years old, when you have that code available to you? You advertise yourself as a Geospatial Programmer for Rent, but then you refuse funds, wait a year, and then announce privately that you had no plans to ever do any of the work. I do not see how any one benefits from you stalling for a year, then doing nothing, and saying that really you never had any plan to do any work on the project, apparently you just want to delay the project as much as you possibly could. You now refuse even TO fix the Quality setting in jp2kakadataset.cpp after extensive discussions have shown that the GDAL community woUld benefit from my suggestions. You have rejected, without any reason, the two suggestions I have to change only two lines of code that would improve JP2KAK. Why weren't you honest a year ago?

[warmerdam]

Greg,

I think you had best take your concerns up with the Project Steering Committee. I don't think I can continue to deal with you in a civil manner.

[Coatman]

I think it can be objectively said that over the past year you have deliberately acted to prevent the GDAL community from achieving the JPEG2000 performance available for free to all users of Linux, Mac OS X, Solaris, and Windows operating systems, for 3 years. After you said I needed to pay you more money, and then I reacted by offering you more funding that you had asked for, then you did nothing, stalled for months, and finally announced only today that you will not do the work, regardless of what you are paid.

I said yesterday that I have no choice but to call this unacceptable matter to the attention of the Committee. Let's move forward with that, to get you over ruled as soon as possible.

[nowakpl]

invalid/wontfix imo, if anything the docs could be changed to mention that "at high rates lossless compression might produce better results for typical image data"

[Even Rouault]

Commited in r17933 a variation of Adam's proposal to improve the doc and give a hint that lossless compression might be more efficient than lossy compression for high QUALITY values