Collection of changes originating from ​Nationaal Georegister

Date	2009/08/24
Contact(s)	​Heikki Doeleman
Last edited	Timestamp
Status	draft
Assigned to release	2.5
Resources	The work is done in the NGR project

Overview

This proposal combines various changes originally implemented in a GeoNetwork fork in the project ​Nationaal Georegister. All changes are already implemented and visible (if applicable) in ​Nationaal Georegister. The changes are otherwise unrelated, but they are not very big and I think it's more efficient to bundle them in this single proposal.

The proposed changes are:

1. Performance improvement to search
2. Persistent validation results
3. Organizations
4. Service monitoring
5. Tabbed view of Metadata
6. Local rating
7. Relevance as a percentage

Another change proposal (INSPIRE support) will be separately described by ​Francois and ​Heikki. Yet another change proposal (replacing InterMap by OpenLayers) is already described ​here.

Proposal Type

• Type: GUI Change, Core Change, Module Change
• App: GeoNetwork
• Module: Search engine, Data Manager, Group Manager, Rating

Links

• Email discussions: on geonetwork-devel: ​Proposals to include NGR functionality to GeoNetwork trunk

Voting History

Although presented together in one proposal, all 7 items will be put up for voting separately.

• None as yet

---

Motivations

The motivations for these changes are :

1. A better performance is preferred
2. More informative search results; motivate metadata providers to produce good quality metadata
3. The notion of Organization is lacking in GeoNetwork's domain model
4. More informative search results; motivate service providers to keep them up
5. Better usability
6. Local user feedback
7. Better usability

Proposal

1. Performance improvement to search
In current GeoNetwork, the search from the main page works as follows: the request with search parameters is sent to GeoNetwork and is turned into a JDOM structure by Jeeves; this request object is processed in XSLT (lucene.xsl), the result of which is a different JDOM structure; this in turn is further processed in Java to create the actual Lucene query.

This proposal removes the step of XSLT processing by lucene.xsl. The function of lucene.xsl is exactly re-built in Java (LuceneQueryBuilder.java). The resulting Lucene query is not changed in this proposal.
LuceneQueryBuilder.java was developed ​test-first and as a result, it's well covered by a unit test.

A second performance gain is effected by a change to retrieving the regions data in the search service. In standard GeoNetwork, the regions are retrieved from the DB at each search request; in this proposal, it happens only once, after which the regions data is kept in memory for application lifetime.

Performance metrics from ​YourKit Java Profiler show that this change leads to a 90% performance increase of the search service as such (excluding the creation of the search results page).

2. Persistent validation results
This proposal is to run validation (XSD+Schematron) every time a metadata is inserted or changed. The result of validation is persisted in the DB. This way we can always show the current "validation status" of metadata, e.g. in the search results page.

Validation status is either "valid", "not valid", or "not determined". This last status is the initial validation status when validation has not yet run.

A validation report is also stored in the DB. It contains any XSD or schematron validation messages, a reference to the schematron file that was used, a timestamp, and if applicable a message saying that the metadata profile could not be determined.

An icon indicating the validation status, that links to the validation report if the metadata is not valid, can be included in the search results page an/or in the show metadata page. This indicates something about the quality of the metadata to users, and may encourage metadata providers to produce more valid metadata.

TODO add screenshot

3. Organizations
In GeoNetwork there is no concept of "Organization". In ​NGR and in ​SwissTopo the concept was needed and has been implemented as a small addition to the existing notion of "Group". It would be better, though, to not mix up those two concept and implement Organization as a first-class citizen in GeoNetwork's domain model.

An Organizations page lists all the organizations in your GeoNetwork node and provides a quick search link to search metadata by organization.

The class diagram below sketches this domain model. Of course it may become more advanced in future, depending on community needs -- for example, a user might be part of more than one organizations but have different roles in them.

4. Service monitoring
In this proposal, services (WxS) that are advertised in the metadata are continuously monitored to see if they're up. The current implementation in ​NGR just pings them, although more sophisticated checking is envisioned. The results are displayed as a percentage that represents the uptime over a certain period of time. This percentage is displayed in the search results page. The frequency of the monitoring calls and the period over which the percentage is calculated are configurable in the Admin interface.

The uptime results for services are also claculated for all the services provided by one organization. On the Organizations page, this is displayed. TODO add screenshots

5. Tabbed view of Metadata
Metadata can be very big, resulting in a display where it's not easy to find what you're looking for and that has not much structure. In ​NGR this is tackled by displaying sections of the metadata in separate tabs, creating a much more user-friendly display.


TODO compare with BlueNetMEST

6. Local rating
In GeoNetwork, users can rate metadata. The user's rating is broadcast to all GeoNetwork nodes known to the one where the rating happens, and the results are thus shared between nodes. In ​NGR this does not happen and the rating is simply kept and calculated on only the user ratings on the GN node itself. This proposal wants to make this "local" rating behaviour an option to standard GeoNetwork, configurable through the Admin interface.

7. Relevance as a percentage
In standard GeoNetwork search results can be ordered by relevance. This is based on the relevance score that the Lucene search engine gives to each item in the search results. However it is not shown to the user what the relevance score for each result *is*.
In ​NGR the relevance score is displayed with each search result. Because the Lucene score is an unattractive float between 0 and 1, the relevance score is normalized such that the highest score from the search results is set to 100, and all other results' scores are calculated relative to that.

Backwards Compatibility Issues

1. none
2. a database change is required. SQL script to update existing DBs will be available
3. a database change is required. SQL script to update existing DBs will be available
4. a database change is required. SQL script to update existing DBs will be available
5. none
6. none
7. if implementations use the raw Lucene score for anything, that will be broken (not in standard GeoNetwork)

Risks

Participants

• As above