Opened 13 years ago

Closed 13 years ago

Last modified 13 years ago

#2930 closed enhancement (fixed)

bit mask performance

Reported by: jlacroix Owned by: jlacroix
Priority: normal Milestone: 6.0 release
Component: MapServer C Library Version: unspecified
Severity: normal Keywords:
Cc: pramsey, warmerdam, dmorissette

Description

With really large shapefiles, some profiling revealed that a lot of call are made to msGetNextBit by either msShapeFileLayerNextShape and msFilterTreeSearch. We acknowledge that the bit mask is not the optimal solution for very large shapefile (>3M features in our case).

Currently we use a char* for the bit mask. We could use a 32bit bitmask instead to gain even more time.

Change History (6)

comment:1 Changed 13 years ago by jlacroix

Type: defectenhancement

We will convert the mapbits.c function to work with 32 bit integers instead of chars in the bit array and while we're at it we will create a new ms_bitarray typedef for the bit array argument of all the ms*Bit() functions.

comment:2 Changed 13 years ago by warmerdam

Cc: warmerdam added

comment:3 Changed 13 years ago by dmorissette

Cc: dmorissette added

comment:4 Changed 13 years ago by jlacroix

Fixed in r8741

Added the ms_bitarray typedef to use as the bitmask.

Leaving open to fix one other item: When a shapefile extent is all included in a map, we loop through all bit and call msSetBit() for each bit in the bitmask. By adding a msSetAllBits() function we would optimize this process.

comment:5 Changed 13 years ago by jlacroix

Resolution: fixed
Status: newclosed

msSetAllBits added in r8755

Using memset will probably be way faster than calling msSetBit for each feature in the shapefile.

Marking as Fixed

comment:6 Changed 13 years ago by tomkralidis

FYI small change in r8774 to fix warning.

Note: See TracTickets for help on using tickets.