JPEG2000 Driver is slow because it reads whole image as one big block

[mempko@…]

The JPEG2000 driver is flawed because it reads a JPEG2000 image as one big block. It allocates jasper matrices the size of an image for each band and stores them in memory until the bands are completely read. If I have a 8000x8000 pixel jp2 file with three bands, i have 8000x8000x3 = 192,000,000 doubles stored in memory!!! This can be fixed by reading the jp2 in much smaller blocks (256x256?) and reading performance is significantly improved for instances where you need only a small portion of the jpeg. Also memory is also significantly reduced during reading. If you want I can send my fix to you and you warmerdam and you can review it.

[warmerdam]

Mempko, 

Andrey Kiselev is the author of the JPEG2000 (JasPer) driver, so I'm passing
this on to him.  I think you can go ahead and attach patches for him to review.

I was aware of the limitations of the jpeg2000 driver, but I was not aware
there was a meaningful alternative.

Best regards,

[mempko@…]

Reads JP2 files in 256x256 blocks instead of one large block.

[mempko@…]

I was working on server code where requests would only ask for a portion of a jpeg2000 file to be viewed. I found the old version of the driver too slow because it decompressed the jp2 file as one big block. The patch I attached improved performance of the jpeg2000dataset significantly (especially for cases where only a small portion of the image was viewed). It also improved memory usage of the driver. The patch attached can be improved further by having the driver figure out the optimal block size depending on the image it is reading. Like using native tile sizes of the jpeg2000 image.

[dron]

I have applied proposed changes. I don't remember why it was chosen to read the whole image in memory, but there were reasons. I hope now we can operate with block of smaller size. I am not sure that it is possible to determine "natural" block size of encoded image, because JasPer interface does not have ability to get low-level image parameters.

Mempko, please, use uniform diff next time (diff with -u option), otherwise it is hard to review and apply the patch.

Thanks,
Andrey

[mempko@…]

Sorry about the patch. I will submit a unified version next time. The problem with the patch above is that it is defaulted to 256x256 blocks. I am not sure if this is an issue with my patch, or jasper in general, but on some of my test images, I am getting warped pixels at the very right and sometimes bottom edge of the image. The warping is right on the edge. If it is because I am requesting the 256x256 blocks, then jasper should handle rendering excess blocks properly but now I am not quite sure. Japser may be putting junk pixels where no pixels should go. Have you gotten any warping on the right or bottom edge in any of your tests Kiselev? I would like to know if it is my software, or a problem with jasper in general.

[dron]

No, I haven't noted that, but my set of JPEG2000 samples is limited. Is it possible for you to share the file which demonstartes that problem?