Load MGP file failed with huge template maps

[christinebao]

Report from Autodesk QA:
In Site Admin load the attached MGP file and get error: An exception occurred in DB XML component. Error: DB_BUFFER_SMALL User memory too small for return value“
delete 3-4 10 Folders with MapGuide Administrator 2011 from the project the following message: „An exception occurred in DB XML component. Error: Not enough space“.

[christinebao]

The testing MGP file is too large to be attached. However, it has nothing to do with testing data. You can reproduce the defect by creating a MGP file with a large resource tree, and it cannot be loaded.

[christinebao]

Tech assessment:

Cause of the defect:
There are 6502 resources in the package, that means it requires 6502 pairs of DeleteResource/SetResource to finish the upload package process, and it must be run in a transaction. However the DBEnvironment set max transaction number to 1000, and the default lock number is 1000, which can not support 6502 pairs of operations, thus cause the defect.
To fix the defect, the number of transaction and lock should be increased. By testing the following number works:
max lock: 20000
Every coin has two sides. Increasing the number of transaction and lock will cause the pre-alloc memory for lock table to increase. It's the disadvantage of this fixing.

Fixing:
Increasing the number of transaction and locks.
DbEnvironment.h
static const u_int32_t MG_MAX_LOCKS = 20000;

DbEnvironment.cpp

m_dbEnv.set_tx_max(MG_MAX_TRANSACTIONS);
m_dbEnv.set_lk_max_lockers(MG_MAX_LOCKS);
m_dbEnv.set_lk_max_locks(MG_MAX_LOCKS);
m_dbEnv.set_lk_max_objects(MG_MAX_LOCKS);

Better fixing:
20000 is too much for normal operation. It's better to move the setting to server.ini to make it configurable. So user can increase it (in this case), or decrease it when not need to.
There are many configuration in DbEnvironment.h should move to server.ini. It's suggested to do them together.

Reference document:

1. A similiar issue: ​http://forums.oracle.com/forums/thread.jspa?threadID=1017325&tstart=150[[BR]]

BDB developer replies that if "Error: DB_BUFFER_SMALL: User memory too small for return value" returns, it should be a bug in BDB. However 2.5.16 (our current version) still have this issue in certain cases, and the BDB developer's suggestion is to increase cache size, locks etc.

2. set_lk_max_lockers API:
   

​http://pybsddb.sourceforge.net/api_c/env_set_lk_max_lockers.html[[BR]]

Quoted: Set the maximum number of simultaneous locking entities supported by the Berkeley DB lock subsystem. This value is used by DB_ENV->open to estimate how much space to allocate for various lock-table data structures. The default value is 1000 lockers.

3. set_lk_max_locks API:
   

​http://pybsddb.sourceforge.net/api_c/env_set_lk_max_locks.html[[BR]]

Quoted: Set the maximum number of locks supported by the Berkeley DB lock subsystem. This value is used by DB_ENV->open to estimate how much space to allocate for various lock-table data structures. The default value is 1000 locks.

4. set_lk_max_objects API:
   

​http://pybsddb.sourceforge.net/api_c/env_set_lk_max_objects.html[[BR]]

Quoted: Set the maximum number of simultaneously locked objects supported by the Berkeley DB lock subsystem. This value is used by DB_ENV->open to estimate how much space to allocate for various lock-table data structures.

[christinebao]

Attach patch ​http://trac.osgeo.org/mapguide/attachment/ticket/1779/Fix1779.patch for fixing this ticket.

[christinebao]

Submit r6067 to trunk to fix this ticket.

[christinebao]

Submit r6068 to replace tab with space for serverconfig.ini.