Harvesting History

Date	2011/03/28
Contact(s)	Simon Pigot, Jose Garcia
Last edited
Status	Complete
Assigned to release	2.7.0
Resources	Available

Overview

Currently not possible to see a history of the results from a harvester - only the results of the last harvest are available. This proposal will add:

• a HarvestHistory table to the database
• record results of each harvest in the HarvestHistory table
• add user interfaces to the Harvest Manager interface for review of results from individual harvesters or all harvesters

Proposal Type

• Type: Harvester Manager GUI Change, Change to Core Harvester code, database addition
• Module: Harvester, Database

Voting History

• Not proposed for voting yet

---

Motivations

Allow administrators to review the history of a harvester or all harevsters.

Proposal

Database changes: add a HarvestHistory table with the following structure:

CREATE TABLE HarvestHistory
  (
    id             int,
    harvestDate    varchar(30),
    harvesterUuid  varchar(250),
    harvesterName  varchar(128),
    harvesterType  varchar(128),
    deleted        char(1) default 'n' not null,
    info           longvarchar,
    params         longvarchar,

    primary key(id)

  );

CREATE INDEX HarvestHistoryNDX1 ON HarvestHistory(id);
CREATE INDEX HarvestHistoryNDX2 On HarvestHistory(harvesterUuid);
CREATE INDEX HarvestHistoryNDX3 ON HarvestHistory(harvestDate);

The info field is an XML string of the results of the harvest. The params field is the parameters of the harvester. The params field is present so that a harvester that has been deleted can be restored. XML strings are used for the results (info field) because the different harvesters do not share many result fields in common. The info field could also be an XML document describing a stack trace if a particular harvest failed.

Here is what the Harvester Manager interface looks like with the new History mechanisms:

Image wiki:HarvestHistory/HarvestManager.png

As you can see, this table holds the history of all harvesters: this includes those that are currently defined in the Harvester Manager and those that have been deleted.

Backwards Compatibility Issues

Creation of the HarvestHistory table will need to be added to database migration scripts.

New libraries added

None required.

Risks

Participants

• Simon Pigot, Jose Garcia