Changes between Initial Version and Version 1 of CSW202Improvements


Ignore:
Timestamp:
Jan 15, 2009, 5:46:48 AM (16 years ago)
Author:
Fxp
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • CSW202Improvements

    v1 v1  
     1
     2== Context ==
     3!GeoNetwork CSW current implementation has some known issues or missing functionnalities. The purpose of this document is to analyse the missing or incomplete elements in !GeoNetwork according to CSW ISO 2.0.2 specification. The work will be based on current !GeoNetwork trunk and focus on apply patch to !GeoNetwork project and GeoSource.
     4
     5----
     6
     7== OGC-CSW Operation Analysis ==
     8=== Reference document ===
     9
     10 * Specifications:
     11  * http://portal.opengeospatial.org/files/?artifact_id=21460
     12  * INSPIRE Service attachment:INSPIRE-CSW-Synthese.doc
     13 * Implementing/compliant tools :
     14  * http://www.mdweb-project.org/
     15  * http://geonetwork-opensource.org/
     16  * http://www.geocatalogue.fr/
     17  * http://www.conterra.de/en/products/sdi/terracatalog/index.shtm
     18 * GeoNetwork:
     19  * trunk source code
     20  * Archie Warnock implementation & documentation : http://trac.osgeo.org/geonetwork/wiki/CSW
     21  * Patch provided by Zhuahua Zha on CSW-T operations
     22
     23=== !GetCapabilities operation ===
     24
     25 * Automatically added !GetRecords search parameters in !GetCapabilities operation
     26 * Changed !GetCapabilities properties for OGC Testsuite compliance
     27  * Added Constraint name="!IsoProfiles"
     28  * Added gmd:MD_Metadata Typenames
     29  * Added support of sections parameter
     30
     31=== !DescribeRecords operation ===
     32 * Fixed !DescribeRecord operation.
     33 * Make OGC testSuite successful : added three !SchemaComponent, modified default results ...
     34 * Added profil support
     35
     36=== !GetRecords operation ===
     37==== CSW Search parameters ====
     38
     39Here is a summary of what should be achieve on Geonetwork trunk CSW in order to respectuful of the 2.0.2 CSW specification.
     40
     41===== Issues =====
     42  * How to handle '''namespace''' properly ? GeoNetwork will be as much flexible as possible and compliant to the specification regarding namespace handling.
     43   1. remove namespace from mapping (dc, csw, apiso ...)
     44   2. mapped all properties with apiso namespace and dc (ie !SupportedISOQueryables, !SupportedDublinCoreQueryables)
     45   3. handled properties without dealing with namespace (ie whatever we provide : dc, csw, apiso... csw should be able to perform operation and to handle search parameters)
     46
     47  * All GeoNetwork search parameters will be supported in CSW search (even if not list in the GetCapabilities documents). Eg. any = africa will work and AnyText = africa also.
     48
     49  * Updated CSW202 schema to make validate option working.
     50
     51  * Added missing elements in brief/summary/full response and reorder misplaced elements.
     52
     53  * Inconsystency concerning '''case sensitivity''' between  :
     54   * Case sensitivity for common queryables must be the same as defined in the base specification (e.g. 'apiso:title') ''[§8.2.2.1.1, OGC 07-045, page 71], [§4.7.2 CSW spec / INSPIRE], [§7.2.4, OGC 07-045, page 46], [§3.2.2 CSW spec / INSPIRE]''
     55   * Parameter names in all KVP encodings must be handled in a case insensitive manner while parameter values shall be handled in a case sensitive manner [§4.2 CSW spec / INSPIRE].
     56
     57
     58  * '''Filter Encoding''': difference between !PropertyIsEqualTo & !PropertyIsLike : first one has an attribute "matchCase" defined to true by default, !IsLike property has no case constraint. Mainly because the fisrt one is !BinaryComparisonOpType compare to !PropertyIsLikeType.
     59
     60
     61  * Coupound parameters :
     62   * '''Degree & Specification''' must be coupled because there are '''linked''' or compound attribute. Do we need to create a couple field name and handle rules in java to perform adequate search ? There is also the same problem with '''operatesOn (Identifier & Name)''' 
     63   * Should we add as !AdditionalQueryables all '''compound elements''' such as CRS, !SpatialResolution ... as concat field in lucene
     64
     65  * Spatial filter only applied to !LatLongBoundingBox : not to any kind of gmd:extent ?
     66
     67  * !OperatesOnWithOpName replaced by !OperatesOnName (probably a typo in OGC 07-045 example).
     68
     69===== Core queryable properties =====
     70
     71 * Type : SupportedISOQueryables ??
     72
     73|| '''Name''' || '''Missing''' || '''Must be modified''' || '''Comments''' ||
     74||Subject || No || Yes || must be mapped to keyword AND topicCategory (only mapped with keyword for now) ||
     75||Title || No || No || - ||
     76||Abstract || No || No || - ||
     77||!AnyText || No || No || - ||
     78||Format || No || No || - ||
     79||Identifier || No || Yes || must be mapped to FileIdentifier (fileId in lucene index)||
     80||Modified || No || Yes || modified currently mapped to changeDate which is only indexed for dc schema in geonetwork. Modified must be mapped to dateStamp.Date or DateTime in ISO schema ??? ||
     81||Type || No || No || - ||
     82||!WestBoundLongitude || '''Yes''' || - || -> westBL in lucene index, check spatial search in lucene since Jesse's work ||
     83||!SouthBoundLongitude || '''Yes''' || - || -> southBL in lucene index, check spatial search in lucene since Jesse's work ||
     84||!EastBoundLongitude || '''Yes''' || - || -> eastBL in lucene index, check spatial search in lucene since Jesse's work ||
     85||!NorthBoundLongitude || '''Yes''' || - || -> northBL in lucene index, check spatial search in lucene since Jesse's work ||
     86||Authority || '''Yes''' || - ||  crs field in lucene must be modified, currently corresponding to codeSpace and code concatenation. Index fields authority, crsCode, crsVersion created. '''Should we add a CRS property in CSW parameters ??? In this case should we concat the 3 properties??''' ||
     87||ID||'''Yes'''||-|| e.g. Authority ||
     88||Version||'''Yes'''||-|| e.g. Authority ||
     89
     90===== Additional queryable properties =====
     91
     92====== common additional queryable properties  ======
     93
     94 * Type : SupportedISOQueryables
     95
     96|| '''Name''' || '''Missing''' || '''Must be modified''' || '''Comments''' ||
     97|| !RevisionDate || '''Yes''' || -  || -> revisionDate (lucene index) ||
     98|| !AlternateTitle || No || No || - ||
     99|| !CreationDate || No || No || - ||
     100|| !PublicationDate || '''Yes''' || - || -> publicationDate (lucene index) ||
     101|| !OrganisationName || No || No || - ||
     102|| !HasSecurityConstraints || No || No || - ||
     103|| Language || No || No || - ||
     104|| !ResourceIdentifier || No || Yes || currently dc:identifier in Geonetwork ||
     105|| !ParentIdentifier || No || No || - ||
     106|| !KeywordType || No || No || - ||
     107
     108====== additional queryable properties for dataset, datasetcollection, application ======
     109
     110 * Type : SupportedISOQueryables
     111
     112|| '''Name''' || '''Missing''' || '''Must be modified''' || '''Comments''' ||
     113|| !TopicCategory || No || Yes || topicCat lucene index should map the codelist?? ||
     114|| !ResourceLanguage || No || Yes || currently !DatasetLanguage in !Geonetwork ||
     115|| !GeographicDescriptionCode || No || No || - ||
     116|| Denominator || No || No || - ||
     117|| !DistanceValue || No || No || - ||
     118|| !DistanceUOM || No || No || - ||
     119|| !TempExtent_begin || No || No || - ||
     120|| !TempExtent_end || No || No || - ||
     121
     122======  additional queryable properties for services ======
     123
     124 * Type : SupportedISOQueryables
     125
     126|| '''Name''' || '''Missing''' || '''Must be modified''' || '''Comments''' ||
     127|| !ServiceType  ||  '''Yes''' || -  || should map a Codelist? add serviceType to lucene index  ||
     128|| !ServiceTypeVersion ||   '''Yes''' || -  || should also map a Codelist? add serviceTypeVersion to lucene index  ||
     129|| Operation ||  Yes || - || add operation to lucene index ||
     130|| !GeographicDescriptionCode || No  || No  || e.g. !GeographicDescriptionCode for dataset ||
     131|| !OperatesOn ||  '''Yes''' || -  || add OperatesOn to lucene index, should map MD_DataIdentification.citation.CI_Citation.identifier but in Geonetwork/Geosource, identifier is store in '''uuidref attribute''' of MD_DataIndetification ||
     132|| !OperatesOnIdentifier ||  '''Yes''' || -  || add !OperatesOnIdentifier to lucene index  ||
     133|| !OperatesOnName ||  '''Yes''' || -  || add !OperatesOnName to lucene index  ||
     134|| !CouplingType ||  '''Yes''' || -  || add !CouplingType to lucene index, should map to a tag code under SV_CouplingType, there is no '''code tag''' under GN!!  ??? ||
     135
     136====== INSPIRE additional queryable properties ======
     137
     138 * Type = !AdditionalQueryables
     139
     140|| '''Name''' || '''Missing''' || '''Must be modified''' || '''Comments''' ||
     141|| Degree || '''Yes''' || - || add Degree to lucene index ||
     142|| !SpecificationTitle|| '''Yes''' || - || add !SpecificationTitle to lucene index ||
     143|| !SpecificationDate || '''Yes''' || - || add !SpecificationDate to lucene index ||
     144|| !SpecificationDateType || '''Yes''' || - || add !SpecificationDateType to lucene index ||
     145|| !AccessConstraints || '''Yes''' || - || add !AccessConstraints to lucene index ||
     146|| !OtherConstraints || '''Yes''' || - || add !OtherConstraints to lucene index ||
     147|| !Classification || '''Yes''' || - || add Classification to lucene index ||
     148|| !ConditionApplyingToAccessAndUse|| '''Yes''' || - || add !ConditionApplyingToAccessAndUse to lucene index ||
     149|| !MetadataPointOfContact || '''Yes''' || - ||  '''it shall include an email contact??'''  ||
     150|| Lineage || '''Yes''' || - || add Lineage to lucene index ||
     151
     152===== Spatial Request & Filter Encoding =====
     153
     154 * Added support for spatial request [https://project.camptocamp.com/doc/brgm_geosourcev2/CSW/FE More...]
     155
     156===== CQL =====
     157 * Migrate from zing Parser to GeoTools CQL parser.
     158
     159=== !GetRecordById ===
     160
     161=== !GetDomain ===
     162 * OnGoing
     163
     164=== Harvest ===
     165 * OnGoing
     166
     167=== Transaction ===
     168 * OnGoing
     169
     170
     171----
     172== OGC-CSW Configuration system ==
     173
     174The following CSW options could be set up in config-csw.xml in WEB-INF directory.
     175 * List of parameters available in GetRecords operations
     176{{{
     177<geonet>
     178        <operations>
     179                <operation name="GetRecords">
     180                        <parameters>
     181                                <!-- - - - - - - - - - - - - - -->
     182                                <!-- Core queryable properties -->
     183                                <!-- - - - - - - - - - - - - - -->
     184                                <parameter name="Subject" field="subject" type="SupportedISOQueryables" />
     185                                <parameter name="CswParameterName" field="LuceneFieldKey" type="CswType" />
     186}}}
     187 * Profil support in CSW for DescribeRecord operation
     188{{{
     189                <operation name="DescribeRecord">
     190                        <!-- schema attribute must defined an existing schema file name -->
     191                        <!-- located at /web/geonetwork/xml/validation/csw/2.0.2/ -->
     192                        <typenames>
     193                                <typename namespace="http://www.isotc211.org/2005/gmd"
     194                                    prefix="gmd" name="MD_Metadata" schema="identification.xsd" />
     195}}}
     196The following CSW options could be define in the administrator interface:
     197 * Contact for the CSW service
     198
     199
     200
     201----
     202== OGC-CSW Iso Profil management ==
     203 * Add the profil for DescribeRecord operation in config-csw.xml
     204 * Add schema to xml/validation/csw202_apiso100/csw/2.0.2
     205 * Add profil-brief.xsl, profil-summary.xsl, profil-full.xsl in xml/csw/schemas/iso19139 directory for GetRecords output.
     206
     207----
     208== OGC-CSW TestSuite ==
     209
     210=== Teamengine ===
     211 * Download teamengine : http://sourceforge.net/project/showfiles.php?group_id=172853&package_id=197871&release_id=589119
     212 * Testsuite CSW 2.0.2 : http://www.gdi-de.org:8080/teamengine/
     213 * Testsuite doc : https://project.camptocamp.com/doc/c2c/ogc/testsuite
     214 
     215=== Results ===
     216
     217 * [https://project.camptocamp.com/doc/brgm_geosourcev2/testSuiteResults OGC test suite results]
     218
     219 * Make Test csw:level1.1 passed by modifying GetRecords-InvalidRequest operation : Typenames must be an attribute of the query element. (but in specificiation, should be csw:Record by default ??)
     220  * Added private checkTypenames method in GetRecords class
     221
     222 * csw:level1.2 / !DescribeRecord-ValidResponseStructure
     223  * Regarding the BRGM specification : The !DescribeRecordResponse must include three "!SchemaComponent" elements :
     224   * identification.xsd for data (ISO19115/19139)
     225   * serviceMetadata.xsd for service (ISO19119)
     226   * record.xsd for CSW/DC definition
     227  * Regarding OGC spec (07-045) : The !DescribeRecordResponse must include two "!SchemaComponent" elements :
     228   * identification.xsd for data (ISO19115/19139)
     229   * serviceMetadata.xsd for service (ISO19119)
     230  * Currently in Geonetwork; only one "!SchemaComponent" element. And giving csw-2.0.2.xsd as response, which is quite similar to record.xsd but is not.. Is there a reason for that one to differ ?
     231  * Should we add "application" and "datasetcollection" in schemaComponent file mapping ???
     232  * Fix namespace for !TypeName parameter : The value of !TypeName has to be qualified by a namespace or should be the default document namespace. There is currently an exception if any namespace sent in request.
     233
     234 * csw:level1.3 / csw.GetCapabilities.document: 
     235  * Is there a reason to comment NullCheck !ComparisonOperator in GN ?
     236  * Regarding this test, 7 commons parameters are expected, idem -> what to do with common constraints (i.e. IsoProfiles, postEncoding)
     237{{{
     238...
     239 <xsl:if test="not($response/csw2:Capabilities/ows:OperationsMetadata/ows:Operation/ows:Parameter[@name='typeName'])">
     240        <message>FAILURE: the second assertion failed (9)</message>
     241        <fail/>
     242      </xsl:if>
     243
     244...
     245}}}
     246----
     247
     248== OGC-CSW Interoperability Matrix ==
     249
     250|| || '''!GeoNetwork 2.2''' || '''!GeoNetwork trunk csw''' || '''!GeoSource v2''' || '''!MdWeb''' || '''geocatalogue.fr''' || '''!ConTerra catalogue ?''' ||
     251|| '''!GeoNetwork 2.2''' || || || || || || ||
     252|| '''!GeoNetwork trunk csw'''  || || || || || || ||
     253|| '''!GeoSource v2'''  || || || || || || ||
     254|| '''!MdWeb'''  || || || || || || ||
     255|| '''geocatalogue.fr'''  || || || || || || ||
     256|| '''!ConTerra catalogue ?''' ||  || || || || || ||
     257
     258 * Test methodology:
     259  * GeoNetwork & Geosource : Harvesting mechanism
     260  * MdWeb : Distributed search mechanism
     261
     262=== Test URL ===
     263
     264 * CSW 2.0.2 MDweb2 : http://demo20.mdweb-project.org/MDweb2/WS/csw
     265 * http://demo.geomatys.fr/constellation/WS/csw?service=CSW&version=2.0.2&request=GetCapabilities
     266 * CSW 2.0.2 GT : http://www.fao.org/geonetwork/srv/en/csw?request=GetCapabilities&version=2.0.2&service=CSW
     267
     268=== Test User Interface ===
     269Creation of a web interface for testing operation using POST method.
     270 * http://brgm-geosourcev2.demo-camptocamp.com/geonetwork_csw/srv/en/test.csw
     271
     272----
     273== OGC-CSW extension in GeoNetwork ==
     274=== GetRecords operation ===
     275 * Added resultsType="results_with_summary" for narrow your search option (see geocat.ch sandbox)
     276 * Added outputSchema="own" to return metadata record in their own format (dc, fgdc, iso, ...)
     277
     278=== Added gml:id attribute support ===
     279 * Allow use of gml:id in !SpatialFilter to reduce big geometry transfert between client and server:
     280{{{
     281<ogc:Intersects>
     282  <ogc:PropertyName>ows:BoundingBox</ogc:PropertyName>
     283  <gml:MultiPolygon xmlns:gml="http/www.opengis.net/gml" gml:id="kantone:2"/>
     284</ogc:Intersects>
     285}}}
     286 * Require that the server knows some well know geographic features.
     287
     288
     289=== output/format ===
     290 * csw.pdf : GetRecords operation results in pdf format.
     291 * csw.txt : GetRecords operation results in txt format.
     292 : to be defined  rss