Changes between Version 3 and Version 4 of PerformanceEnhancements2

Timestamp:

Mar 29, 2010, 2:51:04 AM (14 years ago)

Author:

simonp

Comment:

--

PerformanceEnhancements2

@@ -47 +47 @@
 The net result of these two fixes is much faster load, harvest, reindex and massive operations in !GeoNetwork. For example, in one case doing a file system harvest of 20,000 records was taking 10-12 hours without these modifications. With the modifications described in this proposal, the same harvest now takes approx 30 minutes.  
 
-Addressing the startup delay issue: when a profiler is set up to watch the !DataManager init method, it was found that an incredible amount of time was being spent in processing the list of JDOM Elements returned from the database. Changing the code to retrieve a number child Element speeds up this code approx 10x. eg. Startup delay for !GeoNetwork with approximately 160,000 records drops from approx 500-600 seconds to 50 seconds. Also, the memory requirements of this method were reduced by using Lucene search selectors on the index to speed up retrieval of the _changeDate field. 
+Addressing the startup delay issue: when a profiler is set up to watch the !DataManager init method, it was found that an incredible amount of time was being spent in processing the list of JDOM Elements returned from the database. Changing the code to retrieve a number child Element speeds up this code approx 10x. eg. Startup delay for !GeoNetwork with approximately 160,000 records drops from approx 500-600 seconds to 50 seconds. Also, the memory requirements of this method were reduced by returning just the contents of the index _changeDate field and speed was helped a little by using Lucene field selectors to retrieve only the _changeDate field from the Documents in the index. 
 
 === Backwards Compatibility Issues ===