Harvesting History

Date	2011/03/28
Contact(s)	Simon Pigot, Jose Garcia
Last edited	2011/03/28
Status	Complete
Assigned to release	2.7.0
Resources	Available

Overview

Currently not possible to see a history of the results from a harvester - only the results of the last harvest are available. This proposal will add:

• a HarvestHistory table to the database
• record results of each harvest in the HarvestHistory table
• add user interfaces to the Harvest Manager interface for review of results from individual harvesters or all harvesters

Proposal Type

• Type: Harvester Manager GUI Change, Change to Core Harvester code, database addition
• Module: Harvester, Database

Voting History

• Proposed for voting at irc meeting on 2011-08-02 (see ​http://irclogs.geoapt.com/geonetwork/%23geonetwork.2011-08-02.log).

---

Motivations

Allow administrators to review the history of a harvester or all harevsters.

Proposal

Database changes: add a HarvestHistory table with the following structure:

CREATE TABLE HarvestHistory
  (
    id             int,
    harvestDate    varchar(30),
    harvesterUuid  varchar(250),
    harvesterName  varchar(128),
    harvesterType  varchar(128),
    deleted        char(1) default 'n' not null,
    info           longvarchar,
    params         longvarchar,

    primary key(id)

  );

CREATE INDEX HarvestHistoryNDX1 ON HarvestHistory(id);
CREATE INDEX HarvestHistoryNDX2 On HarvestHistory(harvesterUuid);
CREATE INDEX HarvestHistoryNDX3 ON HarvestHistory(harvestDate);

The info field is an XML string of the results of the harvest. The params field is the parameters of the harvester. The params field is present so that a harvester that has been deleted can be restored (but this is not yet implemented). XML strings are used for the results (info field) because the different harvesters do not share many result fields in common. The info field could also be an XML document describing a stack trace if a particular harvest failed.

As you can see, this table holds the history of all harvesters: this includes those that are currently defined in the Harvester Manager and those that have been deleted.

Here is what the Harvester Manager interface looks like with the new History mechanisms:

Note the addition of the 'History' button - this shows the history of all harvesters (including those that have been deleted). Each harvester has a 'History' link which shows the individual history of that particular harvester.

Here's what the history interface looks like for one particular harvester:

Note that if a harvest fails, then only the error message will be displayed by default. Additional information can be displayed by clicking on the down arrow next to the error message as shown in the following screenshot:

Here's what the full harvester history looks like:

The entries on this screen can be sorted by harvest date-time or by type and harvest date-time as the following screen shot shows:

In both the history of one harvester and the history of all harvester screens the user can select any or all of the history entries and delete them.

Backwards Compatibility Issues

Creation of the HarvestHistory table will need to be added to database migration scripts.

New libraries added

None required.

Risks

None known.

Participants

• Simon Pigot, Jose Garcia