Maestro - LayerDefinition Inefficiencies with Large Schemas

[crispinatime]

Take the following scenario:

1. Create a feature source connection to a scheme with lots of feature classes (eg a SHP folder connection with many SHP files)

2. Create a layer definition to a feature class

3. Save the layer definition

Observing the Maestro request log shows a OPERATION=GETIDENTITYPROPERTIES request for every class in the schema.

It's OK (slightly slow) for SHP folders but for some Oracle databases has been seen to timeout (and looks an unnecessary overhead).

[jng]

I'd like you to keep an eye on this ticket when I release 2.6 Beta 1 which will be built against the 3.9 branch of FDO, that just had this interesting change land in for the King Oracle provider (​http://trac.osgeo.org/fdo/changeset/7062)

Basically I'm currently laying the blame on the King Oracle provider and I'd like you to confirm or deny this when I release 2.6 beta 1

[crispinatime]

OK - the changes to Oracle FDO may help with the timeout - nice to see updates to that provider. For this ticket, is the GETIDENTITYPROPERTIES on every class a problem with Maestro .LayerDefinition management or the underlying FDO provider schema handling?

[jng]

GETIDENTITYPROPERTIES requires a class definition to pluck from.

GetClassDefinition should be fast if the provider supports GetSchemaNames, GetClassNames and friends.

If the provider doesn't (and I believe this is the case with King Oracle) it has to do a full schema description just to find the matching class definition to pluck the identity properties from.

Why it's happening on save is one part because of #2364 (which you also submitted) and the other part because of King Oracle's DescribeSchema inefficiency.

[crispinatime]

Thanks - it' also seen on SHP which I thought played nicely - but being file-based is often pretty quick even if there is brute-force processing. (easy to replicate using Windows copy/paste on a set of SHP files within a folder to create duplicates / copies and then creating a directory-level SHP connection)

[jng]

SHP suffers from the same problem as King Oracle. It doesn't support GetClassNames, GetSchemaNames and DescribeSchema with class hint, so the *full* schema is fetched even if you're asking for one class.

[jng]

Ok after a review of the validation code. I've come to the conclusion that the identity property check when validating a Feature Source is really a superfluous validation check that doesn't have to be done (it only triggers an informational result, and not a warning or error).

It's only an issue when a referencing layer is added to a Map Definition and is marked as selectable. Such cases are now being checked at the Map Definition validation level as of r8029.

But more importantly, when saving a Layer Definition it should only be doing a shallow validation of the Layer Definition itself and not any upstream dependent resources. This was fixed in r8028

So I've decided to remove this identity property check from the Feature Source validator for r8030.

If these 3 changes as a whole solves your issue, I guess you can close this ticket without having to wait for the release of 2.6 beta 1.

[crispinatime]

This looks good - there are about 7 'GETLEGENDIMAGE' requests per opening of a .LayerDefinition... shall I raise a separate ticket?

[jng]

Those GETLEGENDIMAGE requests are style previews from the new grid-based layer style editor. This is normal.

[crispinatime]

Seven still sounds a lot but until I look at source myself I'll bow to your internal knowledge... closing ticket, thanks.