Dictionary I/O under Windows is slow

[Norm Olsen]

Dictionary I/O, especially dictionary scans required for WKT processing is rather slow under Windows using Microsoft CRT. (Not an issue under Linux.) A directory scan, as performed by CSllCsFromDt for example, is about 10 times slower than when performed using STDIO directly. Tests using a 500 scan benchmark show the following timing in seconds:

OS/Compiler	IoUtil	STDIO
Win7:2008	22.109	2.984
Win7:2010	21.985	2.781
Win7:2012	22.906	2.797
CentOS:gcc	4.440	1.120

Not sure how this affects performance in other areas.

This is not a significant performance issue during normal operation where dictionary scans are unusual. However, in test modules where the entire dictionary is converted to WKT, the execution time becomes hours when it used to be 2 or 3 minutes.

Unless corrected, avoid the WKT and EPSG tests built into csTestCpp (tests I thru N) on Windows. For an undetermined reason, running a complete csTestCpp becomes an overnight chore when compiled and linked with VC++ 2012.

[Norm Olsen]

COde used to acquire the timing data.

[Norm Olsen]

Added source code of the timing mechanism used to produce the timings recorded above.

[Norm Olsen]

This report is valid, but for Windows only. The culprit has been isolated to the CS_fseek call on line 347 of the csIoUtil.hpp module (references are as of revision 2753). This function call, when inside the CS_DefinitionRead template at its current position, has been observed to consume from 6, up to 100, times more CPU time than the remainder of the function. During investigation of this issue, the CS_fseek call was moved from the referenced template and placed immediately prior to the CS_csrd function call at line 631 in CS_csprm.c. This produced normal performance; i.e. six to 100 times faster. I haven't a clue as to what could be causing this issue.

As this performance issue is evident (currently) only in a testing environment, and since there is no problem when compiled and run under Linux/UNIX, the code is left as is for now. If anyone has a possible explanation for the above described behavior, this Trac Ticket will be revisited. For now, it is closed as a "WontFix" issue as the desired synchronization may be critical in some applications/environments.