| 33 | | At the moment one must make several calls to a Geonetwork instance to ensure that the important functions are running and even that could not detect spurious or difficult to detect instabilities of Geonetwork. It would be useful to have a consistent way to both register and view such important characteristics like database connection, errors encountered, corrupt index. Failed logins, etc... |
| | 33 | |
| | 34 | It has been noted that many companies currently maintain forked instances of Geonetwork with customizations for their system. The customizations are typically one of the following types: |
| | 35 | * Xsl customizations |
| | 36 | * Java customizations |
| | 37 | * Javascript customizations |
| | 38 | |
| | 39 | The Javascript customizations are handled as best we can with the widget UI. The XSL customizations are an issue but Schemaplugins and User Formatters relieve this a little. However the java customizations typically require deep patches to the Geonetwork code which can be quite hard to migrate to new versions. |
| | 40 | |
| | 41 | A solution is to modify Jeeves and Geonetwork to be wired together via a dependency injection framework instead of many references between many objects as is done now. The current solution is very tightly coupled and as a result very inflexible. |
| | 42 | |
| | 43 | A converging issue is the desire to use spring security to manage security because it has much richer set of authentication options from what is in Geonetwork prior. |
| | 44 | |
| | 45 | Since Spring has a powerful dependency injection framework and Geonetwork already has several dependencies on Spring and (specifically Spring context). It makes sense to use the Spring dependency injection framework. |
| 44 | | A link will be made from the Admin/config.info page will be made to these servlets so a administrator can easily access this data. In a future implementation we can possible add a more attractive UI for viewing the information. |
| 45 | | All /monitor/* urls will be restricted by a Servlet-filter so that only administrators can access the information. |
| | 51 | * Using the Spring dependency injection framework it is simple to replace one implementation of an object with another. Many objects are already configurable like Resource providers and services but many are not. For example suppose you wanted to use a subclass of DataManager instead of the standard implementation. Currently that requires rebuilding Geonetwork. With spring that is a configuration detail. |
| | 52 | * The dependency injection also provides a simple plugin system. Currently Jeeves and sometimes other subsystems have plugins (like services and schedulers) and the code for parsing the xml, instantiating the classes injecting dependencies is all handled in the Geonetwork/Jeeves code. Spring takes care of this and provides a much more flexible and powerful plugin system than Jeeves. |
| | 53 | * In addition dependency injection makes it much easier to perform unit tests, for example a mock DataManager and Database Resource could be injected and a particular component or service could be tested with out requiring the full system to be running. |
| | 54 | * Spring already has the concept of "overrides" through inheritance and overriding beans thus making the need for a custom solution nearly obsolete |
| | 55 | * Spring provides some advanced features such as Aspect Oriented Programming. Some uses for this feature would be to inject an operation before and after any method in the system. For example one could add an operation each time a metadata is changed without needing to change the code of the core geonetwork at all. Another example would be to replace keep the implementation of DataManager but override a single method within DataManager. |
| | 56 | * Spring Security combined with dependency injection would allow access control on a per-method basis. For example one could designate specific methods that only Admin is permitted to execute and this can be done through configuration instead of code changes in the system. |
| | 57 | * With the addition of spring security into the system Geonetwork already has dependencies and even uses spring. Thus it makes sense to use more of its functionality. |
| | 58 | * Provides a migration path to other spring services like spring data for database access. |
| | 59 | * Reduces the amount of XML parsing done in Geonetwork (reducing the amount of code in Geonetwork) |
| | 60 | * Only one type of configuration (Geonetwork already has spring configuration for JZKit and spring security) |
| | 61 | * Spring developers (like those from Geoserver or a multitude of other projects) would have an easier time starting to work on Geonetwork. |
| 49 | | Some monitors I propose to make are: |
| | 65 | * Currently GetRecords only supports a few output formats, suppose you want to add more output formats. If the GetRecords, GetCapabilities, etc.. were injected it would be easy to intercept requests going to those services and handle extra formats or (in the case of GetCapabilties) modify the output of the service. |
| | 66 | * Currently if one want to have object available throughout the system (like DataManager or SearchManager) they have to be added to the Geonet object (or ServiceContext). However suppose one wants a set of services to be able to communicate together or share an object like a SearchManager then that requires substantial changes to the system. With Dependency injection it is again just a configuration detail (and of course implementation of the services and the object) and can be added to a release Geonetwork instance. |
| | 67 | * Making a statistics module would be much easier since it is very simple to be able to add wrap a method call in a method that will update the statistics. The wrapping method would have all the input and thus would have access to all the data required for calculating the search, edit, etc… statistics. |
| | 68 | * Suppose an application has extra restrictions on what users can access a certain metadata. With dependency injection the getMetadata method on DataManager or XmlSerializer have the appropriate methods wrapped by the dependency injection system to inspect each request and deny those that do not meet the new requirements. |
| | 69 | * Configuration files can have autocomplete if the spring eclipse extensions are installed. |
| 51 | | - Database Health Monitor - checks that the database is accessible |
| 52 | | - Index Health Monitor - checks that the Lucene index is searchable |
| 53 | | - Index Error Healther Monitor - checks that there are no index errors in index (documents with _indexError field == 1) |
| 54 | | - CSW !GetRecords Health Monitor - Checks that GetRecords does not return an error for a basic hits search |
| 55 | | - CSW !GetCapabilities Health Monitor - Checks that the GetCapabilities is returned and is not an error document |
| 56 | | - Database Access timer - Time taken to access a DBMS instance. This gives and idea of the level of contention over the database connections |
| 57 | | - Database Open Timer - Tracks the length of time a Database access is kept open |
| 58 | | - Database Connection Counter - Counts the number of open Database connections |
| 59 | | - Harvester Error Counter - Tracks errors that are raised during harvesting |
| 60 | | - Service timer - Track the time of service execution |
| 61 | | - Gui Services timer - Track the time of spend executing Gui services |
| 62 | | - XSL output timer - Track the time of output xsl transform |
| 63 | | - Log4j integration - monitors the frequency that logs are made for each log level so (for example) the rate that error are logged can be monitored. See http://metrics.codahale.com/manual/log4j |
| 64 | | - Webapp integration - monitors number of active requests, error codes returned and length of time requests take. See http://metrics.codahale.com/manual/webapps/ |
| | 71 | === Disadvantages: === |
| | 72 | * The XML configuration is not as nice of a DSL since for large part it is a generic Spring configuration. (This issues is made less severe since Geonetwork already has some spring security and the configuration actually is pretty close to the current configuration) |
| | 73 | * The old configuration files will no longer work with the new system. There will be a process that runs on start up that will convert the old configuration to the new configuration but it cannot be guaranteed to work in all cases and some systems may need manual configuration. |
| | 74 | * Current Geonetwork developers will have to learn how the new system works since it is a major architectural change. |
| 68 | | See below for an example of the JSON data accessible for the metrics |
| | 78 | The first iteration will have an architecture that is very similar to the current architecture with some notable changes. |
| | 79 | |
| | 80 | * There are still two main layers. Jeeves and Geonetwork. The Jeeves module still has the primary concepts like Service, ResourceManager, ServiceManager, etc… and Geonetwork is still the application. |
| | 81 | * Geonetwork will no longer be responsible for creating classes like the DataManager, SchemaManager, SearchManager etc… Spring will create all of those classes and wire them together. |
| | 82 | * Geonetwork will instead be an "Initializer" that is responsible for preparing the non Java dependencies. For example setting up the database. Setting certain environment variable/properties that are required by several subsystems. It will be a Java Bean and therefore any object that requires one of those initialization services before being capable of running will depend on that Bean. |
| | 83 | * All major components will be Java Beans created and wired together by the dependency injection framework. Spring will ensure that the dependencies are created in the correct order. |
| | 84 | * Although the Geonet object is not necessary from an architectural point of view it will remain as is for developers who are used to the current framework. |
| | 85 | * Service will not be deprecated yes but they will be slowly migrated to a new IService interface that only has the exec method (the init method will not exist) because the initialization method code is unnecessary in the dependency injection system. The configuration file can be used to choose an arbitrary init and destroy method. A benefit of this is that services can have dispose methods if they desire. |
| | 86 | * The destroy or shutdown methods for many of the objects will be removed and only the beans that need cleanup will have then which the dependency injection framework will execute instead of the Jeeves code. |
| | 87 | * Similarly the startup and initialization code of most objects will be invoked by the dependency injection framework instead of the chain of calls required now. |
| | 88 | |
| | 89 | === Configuration === |
| | 90 | |
| | 91 | The structure of the configuration will be kept as similar as possible with the current configuration while still being spring configuration files. The one special case will be the service definitions. A special namespace will be created so that the service definitions are exactly the same as they are now. Here are a few comparisons: |
| | 92 | |
| | 93 | New configuration of a service: |
| | 94 | {{{ |
| | 95 | <j:service id="home"> |
| | 96 | <output sheet="home.xsl"/> |
| | 97 | </j:service> |
| | 98 | }}} |
| | 99 | Old configuration of a service: |
| | 100 | {{{ |
| | 101 | <service name="home"> |
| | 102 | <output sheet="home.xsl"/> |
| | 103 | service> |
| | 104 | }}} |
| | 105 | As you can see the service configuration is almost identical. This is a special case because it is so common. A more typical example is of the general tag in the config.xml: |
| | 106 | |
| | 107 | New configuration of a service: |
| | 108 | {{{ |
| | 109 | <bean id="jeevesGeneralConfig" |
| | 110 | p:uploadDir="./data/tmp" |
| | 111 | p:debug="true" |
| | 112 | p:maxUploadSize="100" |
| | 113 | class="jeeves.config.GeneralConfig" /> |
| | 114 | }}} |
| | 115 | Old configuration of a service: |
| | 116 | {{{ |
| | 117 | <general> |
| | 118 | <profiles>user-profiles.xml</profiles> |
| | 119 | <!-- Size must be in megabyte (integer), 100MB by default --> |
| | 120 | <maxUploadSize>100</maxUploadSize> |
| | 121 | <uploadDir>./data/tmp</uploadDir> |
| | 122 | <debug>true</debug> |
| | 123 | </general> |
| | 124 | }}} |
| | 125 | An alternative way to specify the config element could be: |
| | 126 | {{{ |
| | 127 | <bean id="jeevesGeneralConfig" class="jeeves.config.GeneralConfig" > |
| | 128 | <property name="uploadDir" value="./data/tmp"/> |
| | 129 | <property name="debug" value="true"/> |
| | 130 | <property name="maxUploadSize" value="100"/> |
| | 131 | </bean> |
| | 132 | }}} |
| | 133 | The second version is typically used if comments about the purpose of a property is desired or if the property is a map or list of values. |
| | 134 | |
