Changes between Initial Version and Version 1 of MapGuideRfc36


Ignore:
Timestamp:
09/16/07 14:31:00 (17 years ago)
Author:
stevedang
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • MapGuideRfc36

    v1 v1  
     1= !MapGuide RFC 23 - Improve EnumerateResources API's performance =
     2
     3This page contains a change request (RFC) for the !MapGuide Open Source project. 
     4More !MapGuide RFCs can be found on the [wiki:MapGuideRfcs RFCs] page.
     5
     6
     7== Status ==
     8 
     9||RFC Template Version||(1.0)||
     10||Submission Date||September 16, 2007||
     11||Last Modified||Steve Dang [[Timestamp]]||
     12||Author||Steve Dang||
     13||RFC Status||draft||
     14||Implementation Status||pending||
     15||Proposed Milestone||2.0||
     16||Assigned PSC guide(s)||(when determined)||
     17||'''Voting History'''||July 24, 2007||
     18||+1||||
     19||+0||||
     20||-0||||
     21||-1||||
     22 
     23== Overview ==
     24
     25This RFC describes a new !EnumerateResources API that can be made much faster than the existing !EnumerateResources API when working with repositories that have a large number of resources.
     26
     27== Motivation ==
     28
     29We are finding that when the existing !EnumerateResources operation is executed on a repository that has a large number of resources, it is too slow, and may result in an Out Of Memory exception.
     30
     31For example in this scenario: If the root folder has 1,000 folders in which each contains 1,000 documents, then for the request where resource is ”Library://” and depth is 1, the operation may take about 20 minutes to complete. This is unacceptable and makes client programs that need to list contents of the repository unusable.
     32
     33== Proposed Solution ==
     34
     35The problem with the !EnumerateResources API is that it needs to enumerate all of the child resources in a folder just to count them, even though they are not being returned. So for the above example, !EnumerateResources will need to compute the number of children for 1,001 folders (the root folder plus 1,000 sub-folders), and to evaluate the user’s permissions on 1,001,001 resources (the root folder plus 1,000 sub-folders plus 1,000 X 1,000 documents), even though the user only gets information on the 1,000 top level folders in the result.
     36
     37We propose a new !EnumerateResources API that can be set to not return the number of resources in leaf level folders. For our example above, this means that the operation will only need to evaluate user’s permissions on 1,001 resources (the root folder plus 1,000 sub-folders). This will be much faster.
     38
     39The new !EnumerateResources API is described/documented as follows:
     40
     41    //////////////////////////////////////////////////////////////////////////////////////////////////////
     42    /// \brief
     43    /// Enumerates the resources in the specified repository.
     44    ///
     45    /// \remarks
     46    /// You can enumerate all types or just a selected type. You can
     47    /// also choose what depth in the repository to examine.
     48    /// This method only works on "Library" repository.
     49    /// If you specify a repository that is not supported, this method
     50    /// will throw an InvalidRepositoryType exception.
     51    ///
     52    /// <!-- Syntax in .Net, Java, and PHP -->
     53    /// \htmlinclude DotNetSyntaxTop.html
     54    /// MgByteReader EnumerateResources(MgResourceIdentifier resource, int depth, string type, bool computeChildren);
     55    /// \htmlinclude SyntaxBottom.html
     56    /// \htmlinclude JavaSyntaxTop.html
     57    /// MgByteReader EnumerateResources(MgResourceIdentifier resource, int depth, String type, boolean computeChildren);
     58    /// \htmlinclude SyntaxBottom.html
     59    /// \htmlinclude PHPSyntaxTop.html
     60    /// MgByteReader EnumerateResources(MgResourceIdentifier resource, int depth, string type, bool computeChildren);
     61    /// \htmlinclude SyntaxBottom.html
     62    ///
     63    /// \param resource (MgResourceIdentifier)
     64    /// Resource identifier specifying the resource to
     65    /// enumerate. This can be a document or a folder.
     66    /// \param depth (int)
     67    /// Recursion depth, relative to the specified resource.
     68    /// <ul>
     69    ///     <li>If the resource is a document, depth must be set to
     70    ///          0.
     71    ///     </li>
     72    ///     <li>If the resource is a folder:
     73    ///     <ul>
     74    ///         <li>If the depth is equal to 0, only information about
     75    ///              the specified folder is returned.
     76    ///         </li>
     77    ///         <li>If the depth is greater than 0, information about
     78    ///              the folder and its descendants up to the specified
     79    ///              depth are returned.
     80    ///         </li>
     81    ///     </ul>
     82    ///     <li>If the depth is -1, information about the folder
     83    ///          and all its descendants is returned.
     84    ///     </li>
     85    /// </ul>
     86    /// \param type (String/string)
     87    /// Type of the resource to be enumerated. (Case
     88    /// sensitive.) See \link MgResourceType MgResourceType \endlink for valid
     89    /// types. If the type is a folder, you must include the
     90    /// trailing slash.
     91    /// \n
     92    /// Or, this can
     93    /// be set to null, in which case information about all
     94    /// resource types is returned.
     95    /// \param computeChildren (boolean/bool)
     96    /// Flag to determine whether or not the number of children of the folder
     97    /// resource at the specified depth should be computed.
     98    /// <ul>
     99    ///     <li>If it is true, then the number of children of the folder
     100    ///          resource at the specified depth will be set to a computed value (>= 0).
     101    ///     </li>
     102    ///     <li>If it is false, then the number of children of the folder
     103    ///          resource at the specified depth will be set to -1.
     104    ///     </li>
     105    /// </ul>
     106    ///
     107    /// \return
     108    /// Returns an MgByteReader object containing a description of
     109    /// the resources in XML format using the \link ResourceList_schema ResourceList \endlink
     110    /// schema.
     111    ///
     112    /// <!-- Example (PHP) -->
     113    /// \htmlinclude PHPExampleTop.html
     114    /// These examples assume that <c>$resourceService</c> has
     115    /// already been initialized.
     116    /// \code
     117    /// // Enumerates everything in the library
     118    /// $resourceID = new MgResourceIdentifier("Library://");
     119    /// $byteReader = $resourceService->EnumerateResources($resourceID, -1, "", true);
     120    ///
     121    /// // Enumerates everything in Geography
     122    /// $resourceID = new MgResourceIdentifier("Library://Geography/");
     123    /// $byteReader = $resourceService->EnumerateResources($resourceID, -1, "", true);
     124    ///
     125    ///  // Enumerates all maps in the library
     126    /// $resourceID = new MgResourceIdentifier("Library://");
     127    /// $byteReader = $resourceService->EnumerateResources($resourceID, -1, "MapDefinition", true);
     128    ///
     129    /// // Enumerates all folders in the library
     130    /// $resourceID = new MgResourceIdentifier("Library://");
     131    /// $byteReader = $resourceService->EnumerateResources($resourceID, -1, "Folder", true);
     132    ///
     133    /// // Enumerates the folder Geography
     134    /// $resourceID = new MgResourceIdentifier("Library://Geography/");
     135    /// $byteReader = $resourceService->EnumerateResources($resourceID, 0, "Folder", false);
     136    ///
     137    /// // Enumerates maps one level below Geography
     138    /// $resourceID = new MgResourceIdentifier("Library://Geography/");
     139    /// $byteReader = $resourceService->EnumerateResources($resourceID, 1, "MapDefinition", false);
     140    ///
     141    /// // Enumerates a specific map
     142    /// // NOTE: In this case, depth must be set to 0
     143    /// $resourceID = new MgResourceIdentifier("Library://Geography/World.MapDefinition");
     144    /// $byteReader = $resourceService->EnumerateResources($resourceID, 0, "MapDefinition", false);
     145    /// \endcode
     146    /// \htmlinclude ExampleBottom.html
     147    ///
     148    /// \exception MgInvalidRepositoryTypeException
     149    /// \exception MgInvalidRepositoryNameException
     150    /// \exception MgInvalidResourcePathException
     151    /// \exception MgInvalidResourceNameException
     152    /// \exception MgInvalidResourceTypeException
     153    ///
     154    MgByteReader* EnumerateResources(MgResourceIdentifier* resource,
     155        INT32 depth, CREFSTRING type, bool computeChildren);
     156
     157Note that the performance of the !EnumerateResources API depends on the recursion depth and how effectively the tree hierarchy of resources is laid out, and that the client applications are encouraged to use the new API to take advantage of it.
     158
     159== Implications ==
     160
     161•    There is no compatibility issue associated with the new !EnumerateResources API (i.e. no change in the version of the operation).
     162•    Some indexing will be added to improve the performance of both old and new !EnumerateResources APIs.
     163•    The old !EnumerateResources API will be depreciated in the future.
     164 
     165== Test Plan ==
     166
     167Tests need to be done to verify:
     168
     169•    For the old API, it should work as before (and faster due to indexing).
     170•    For the new API, it should work the same as the old API and faster when the “computeChildren” flag is set to true and false respectively.
     171
     172== Funding/Resources ==
     173
     174Autodesk will provide funding and resources.