artifact manager to replace download.osgeo.org/webdav/geotools

[jive]

Planet has informed us that they would like to retire repo.boundlessgeo.com (an artifactory instance) used by the GeoServer project. The GeoServer PSC would like to ask OSGeo if it is possible to host an artifact manager on OSGeo hardware?

As discussed in 2019 (see ​https://trac.osgeo.org/osgeo/ticket/2261) this is a good oppertunity to setup an artifact manager to replace ​http://download.osgeo.org/webdav/geotools/

There are two good options:

• ​https://jfrog.com/open-source/ - easy to manage
• ​https://www.sonatype.com/download-oss-sonatype - does slightly more

[jive]

This is a somewhat time sensitive activity as we would like to rescue the contents of repo.boundlessgeo.com (mostly prior release of GeoServer and GeoWebCache).

[jive]

As noted in email discussion these repository managers can be configured to manage maven, docker, node, etc... and may be of interest to a sider group of osgeo projects.

[wildintellect]

Possible: Yes How: we need some members to step up to set up a container dedicated to this purpose and hopefully have some expertise or at least time to vet the options.

[robe]

jive -- as a clean replacement sounds like might be better to go with jfrog since that is what repo.boundlessgeo.com is using.

What does sonatype do additional that jfrog doesn't that would be a possible need?

I'd be willing to take a stab at installing it -- you have details of how we can pull over the artifacts from boundless?

[jive]

Answering your question *What does sonatype do additional that jfrog doesn't that would be a possible need?*

Frog artifactory:

• ​https://jfrog.com/open-source/
• personal experience: easier administration screens to manage
• Only supports java projects (Maven, Gradle and Ivy)
• Used by repo.boundlessgeo.com (after a poor experience with cloud hosting)

Nexus:

• ​https://www.sonatype.com/download-oss-sonatype
• personal experience: more complicated administration screens
• This is the clear choice for OSGeo as it offers capabilities for more than just java projects (docker images, node packages, ruby, python, go, etc...)
• Used by eclipse foundation (with their strong emphasis on control),

I have personally used both as an administrator (not installing but managing).

Links:

• ​https://www.sonatype.com/nexus-repository-oss
• ​https://sonatype.drift.click/oss-vs-pro
• ​https://info.sonatype.com/nexus-vs-artifactory

[jive]

I think to copy the contents Tom Ingold did:

1. Set up repo as a cache of old repo
2. Have a script that accessed all the content via the new URL
3. Turn of the "cache" and keep the content

[jive]

So what happens next?

[robe]

jive,

I'm planning to setup osgeo3 this week and thinking I would set up a container for nexus and jfrog.

I'm thinking maybe it would be easier if I setup jfrog first that way you have a clean migration from boundless? and then we can do nexus and slowly port over?

I don't think it's necessary for all projects to use the same thing so if you feel more comfortable with artifactory I'd say you should use that.

[robe]

Jody - I setup nexus using the docker container from here ​https://hub.docker.com/r/sonatype/nexus3/ - I sent you the login details

​https://nexus.osgeo.org

It's running in a dedicated LXD OS container so we could put other dockery stuff in there you need.

It's running on the new osgeo3 lxd host I set up today.

[robe]

nexus supports LDAP so I configured it to use OSGeo LDAP and set jive, wildintellect, strk as an admins.

There is a setting in the LDAP config to use LDAP groups for role management. But I wasn't sure if we wanted to do that for the first go round.

[robe]

based on popular demand I have added

​https://repo.osgeo.org as an alias for ​https://nexus.osgeo.org

we can get rid of nexus but that's what the container is called and the config I have at moment and some other names were proposed like docker.osgeo.org, apt.osgeo.org etc.

[jive]

Thanks for the hard work, I am collecting notes on how to transfer content.

1. webdav

This ​script would like to be run on osgeo machine hosting webdav folder.

This replies on direct file system access and should be fairly quick.

• from: ​http://download.osgeo.org/webdav/geotools
• to: ​https://repo.osgeo.org/geotools-releases

Once content is migrated we can redirect ​http://download.osgeo.org/webdav/geotools/ --> repo.osgeo.org/release

2. repo.boundlessgeo.com to

This ​maven repository provisioner can be run from anywhere, and just uses maven API.

• from: ​https://repo.boundlessgeo.com/artifactory/release
• to: ​https://repo.osgeo.org/geoserver-releases

I expect this will be quite bandwidth intensive, as such I would prefer to run this script on osgeo hardware.

3. when complete

Update the builds for osgeo projects and communicate the change to downstream applications

[robe]

jive are you all set here? Last we discussed you had root access to nexus container and download container and can install anything you need on the nexus container.

I also want to do an upgrade of nexus install to deal with vulnerability that was announced a couple of days ago. I'm not sure you have any groovy scripts so may be a non-issue.

[robe]

I did the upgrade on nexus.

[jive]

Tom Ingold was able to transfer content from ​​https://repo.boundlessgeo.com/artifactory/release to Geoserver-releases (no idea why that ended up with a capital but it does not matter so much as it mostly an internal detail). The maven repository provisioner did not work, but he was able to figure some scripts out (wget, find, xargs and curl).

Initial testing found that some things were missed (apache beanutils) so some testing is needed.

[jive]

I have written up a wiki page ​https://wiki.osgeo.org/wiki/SAC:Repo describing how projects can join the repository.

We have our initial builds working from GeoTools, I think this issue can be closed!

[jive]

aside: I was able to complete migrating contents from webdav folder.

Should we make plans to shutoff that webdav folder, or redirect to ​https://repo.osgeo.org/repository/release/ ?

[robe]

Replying to jive:

aside: I was able to complete migrating contents from webdav folder.

Should we make plans to shutoff that webdav folder, or redirect to ​https://repo.osgeo.org/repository/release/ ?

Yes we should -- you going to be at next SAC meeting? ​https://wiki.osgeo.org/wiki/SAC_Meeting-2020-04 maybe we can have a working meeting

[strk]

Would users have a benefit from redirect ? If so please go for it !