ConfigOverride Proposal

[jesseeichar]

In order to permit deployment of a Geonetwork instance to multiple environments and assist in development this proposal adds the ability to configure Geonetwork for multiple target deployment platforms (for example, test and production) with a minimum of duplication and allow configuration selection to be chosen via System properties or ServletConfig/Context parameters

see: ConfigOverride

[jesseeichar]

patch for trunk

[jesseeichar]

Renamed file to include trunk in name and Corrected bug declaring which override file to use

[jesseeichar]

Patch that applies to trunk

[jesseeichar]

patch that applies to 2.6.x

[jesseeichar]

I have now correctly tested the patches against both 2.6.x and trunk. The good patches are the last two:

​http://trac.osgeo.org/geonetwork/attachment/ticket/513/override-config-trunk.2.patch

​http://trac.osgeo.org/geonetwork/attachment/ticket/513/override-config-2.6.x.patch

[fxp]

Looks interesting Jesse.

Could it be possible to manage other types of configuration files which may differ between deployment ? I was thinking of:

• log4j.cfg
• javax.xml.transform.TransformerFactory
• sql files (to set system settings maybe also ?)
• print-config.yaml (not so important probably)

Is the file in input necessarily an XML document ?

I'm also not sure what's going to happen here (​http://trac.osgeo.org/geonetwork/wiki/proposals/ImprovedSecurityArchitecture) but new configuration files will be added and should be align to existing one probably.

[jesseeichar]

Other file formats could be handled but they need to be structured files if you want to be able to reliably add/remove/update. Also the substitution takes place after launch so the transformer factory parameter is likely not an easy one to update. However if the code is executed early enough it might work.

So the types of files that can be updated are:

xml, json, yaml, etc...

sql files will be quite hard to do so I don't think I will to do that one. But I think that I can add support for log4j as that is really common.

TransformerFactory is also hard but I can do a quick experiment to see if that is possible

print-config.yaml ... I think a better place would be to patch the print module so that the chosen config file can be controlled by system parameters. It already respects init params so that does not need to be updated.

In short I will add support for updating log4j before we apply this proposal.

[fxp]

Replying to jesseeichar:

Other file formats could be handled but they need to be structured files if you want to be able to reliably add/remove/update.

True. I don't ask you to add that kind of support. I was more thinking what is the best option for all (too many) configuration files we have.

2 others thoughts:

• what happens if you have 2 GeoNetwork webapps ? the jeeves.configuration.overrides.file system property is going to be shared by the two (because System.getProperty is the first one called) and they will share the configuration. For data and lucene directory (which could be initialized by system property), we added variables depending on the webapp name (see #352) - that's maybe not the best option anyway (for example if no webapp context is set).
• to better manage different environments configuration on the long-term, maybe we should think of moving all configuration options outside the WEB-INF folder and be able to load the conf using system or servlet property like you've done (but loading the full conf for the target environment instead of processing the default and overriding part of it) ?

[jesseeichar]

Replying to fxp:

Replying to jesseeichar:

Other file formats could be handled but they need to be structured files if you want to be able to reliably add/remove/update.

True. I don't ask you to add that kind of support. I was more thinking what is the best option for all (too many) configuration files we have. I think that if this strategy works well we could expand it to include other file types like json for the javascript configuration (which I think is a better strategy than config-gui since it doesn't require a restart for the changes to take place, but that is another discussion).

2 others thoughts:

• what happens if you have 2 GeoNetwork webapps ? the jeeves.configuration.overrides.file system property is going to be shared by the two (because System.getProperty is the first one called) and they will share the configuration. For data and lucene directory (which could be initialized by system property), we added variables depending on the webapp name (see #352) - that's maybe not the best option anyway (for example if no webapp context is set).

If you look at the proposal A system property does not have to be used. It is one of 3 options. The System property option is simply the easiest. We also have ServletConfig and ServletContext, which can be injected by the servletContainer. The ServletConfig is webapp specific so you could use that strategy if you have multiple installations in the same container.

• to better manage different environments configuration on the long-term, maybe we should think of moving all configuration options outside the WEB-INF folder and be able to load the conf using system or servlet property like you've done (but loading the full conf for the target environment instead of processing the default and overriding part of it)?

Well in someways I agree. I think it is important to control the configuration through the servlet container. But the problem with that is that there are LOTS of configuration parameters so at minimum you need a way to set the defaults so that each deployment you don't need to declare every parameter. That would not be any improvement over the current situation. Also if all configuration is outside of geonetwork how can you add new parameters without breaking an installation when they upgrade. It is bad enough that new parameters are always added to the setting file causing breakages (which is made acceptable by the migration scripts).

I think the solution of having the defaults written and committed to the main repository and providing a way to override them is in fact quite a maintainable solution.

[jesseeichar]

patch for 2.6.x with logging

[jesseeichar]

I have updated the patches with the ability to control logging as well as the configuration. see the proposal and javadocs for more details on that.

[jesseeichar]

patch for trunk with logging

[fxp]

(which I think is a better strategy than config-gui since it doesn't require a restart for the changes to take place, but that is another discussion).

You don't need to restart when updating config-gui.xml.

If you look at the proposal A system property does not have to be used. It is one of 3 options.

So we should probably apply similar strategy for existing properties. Thanks.

[jesseeichar]

If you look at the proposal A system property does not have to be used. It is one of 3 options.

So we should probably apply similar strategy for existing properties. Thanks.

I think this is out of scope for this proposal but I agree.

[craigj]

Updated patch including override for profile configuration and shared overrides applied to trunk in r7903.