Opened 2 years ago

Closed 2 years ago

#2261 closed task (fixed)

Move download.osgeo.org to OSGEO7

Reported by: robe Owned by: sac@…
Priority: normal Milestone: Sysadmin Contract 2019-I
Component: Systems Admin Keywords:
Cc:

Description

This is to be done as part of SAC 2019 contract to resetup download.osgeo.org as an LXD container on OSGeo7

Change History (13)

comment:1 Changed 2 years ago by strk

Please use a Milestone for this

comment:2 Changed 2 years ago by robe

Milestone: Sysadmin Contract 2019-I

strk sorry last time I didn't have rights to create milestones so didn't think I could. I guess I have those rights now, so created a new Milestone.

comment:3 Changed 2 years ago by robe

IN meeting now. Jodygarnett said geotools relies on webdav, so we'll need that on the new box.

I do have a new download countainer running Debian 9 (I couldn't get ldap to work properly on Debian 10 which is the only reason I didn't go with that).

I put macoosgeo (Homebrew bottle.download.osgeo.org) on that already since they were in a rush and I didn't want to put them on the old one and have them change things twice.

comment:4 Changed 2 years ago by robe

We had discussions about this at the SAC meeting today with

jodygarnett, markusN, jef, dbb expressing their needs and concerns.

Key takeaways from the meeting about new download

1) Make sure to maintain timestamps and permissions as part of the sync

2) Maintain sudos

3) The size of osgeo4w + qgis (both users of the NFS) together take up about 100GB, so we could move these back in the folder and dispense with the NFS

4) Geotools currently relies on WebDav? for pushing artifacts so will need that

OR - an interesting idea Jody Garnett brought up is they would like to eventually switch to using something like artifactory / nexus

https://jfrog.com/open-source/

and this might be something that would be useful for other projects.

With that said - might be good to keep download.osgeo.org simple with just the nginx I have installed and no WebDav? and setup a separate container for artifactory use that geotools can use and other projects can eventually take advantage of.

jodygarnett proposed a domain name of repos.osgeo.org

Last edited 2 years ago by robe (previous) (diff)

comment:5 Changed 2 years ago by robe

Resolution: fixed
Status: newclosed

did the final resync and changed the dns. So download.osgeo.org and upload.osgeo.org are officially on osgeo7 now.

comment:6 Changed 2 years ago by jive

Resolution: fixed
Status: closedreopened

Looks like we missed something, builds fail with:

Failed to read artifact descriptor for org.geotools.data:refdataset-1.0:jar:1.0.7: Could not transfer artifact org.geotools.data:refdataset-1.0:pom:1.0.7 from/to osgeo (http://download.osgeo.org/webdav/geotools/): Not authorized -> [Help 1]

Indeed navigating to http://download.osgeo.org/webdav/geotools/ is:

a) redirected to https://download.osgeo.org/webdav/geotools/ b) asked for authorization

comment:7 Changed 2 years ago by jef

Moved the basic auth into limit_except. Please verify that it works.

comment:8 Changed 2 years ago by robe

jive -- did you not get any of my emails? I sent to your gmail account.

I told you I could not implement webdav/ldap so I had set up an htaccess account for geotools. It does not use the ldap one as I don't know the password for that one.

Anyway if you can write to me at my lr at pcorp dot us account we can sort this out.

jef,

I don't seem to be able to get to https://download.osgeo.org/webdav/geotools/ or https://download.osgeo.org/geotools/ anymore. I had set up so webdav uses webdav and geotools is just for download

comment:9 Changed 2 years ago by robe

ah never mind https://dwonload.osgeo.org/geotools still seems to work.

comment:10 in reply to:  9 Changed 2 years ago by jive

Replying to robe:

ah never mind https://dwonload.osgeo.org/geotools still seems to work.

I am not sure what you mean? If I go to https://download.osgeo.org/geotools it results in a 404 not found.

Please note that the build systems use the following:

 <repositories>
  <repository>
    <id>osgeo</id>
    <name>Open Source Geospatial Foundation Repository</name>
    <url>http://download.osgeo.org/webdav/geotools/</url>
    <!-- release repository used by geotools (and third-party dependencies) -->
  </repository>
</repositories>

And there are hundreds of them around the world. If we wish to change (even from http --> https) we should coordinate.

So there are two issues:

a) making sure the contents are available at http://download.osgeo.org/webdav/geotools/

b) sorting up some way for build.geoserver.org and others with an OSGeo userid to deploy new content

Last edited 2 years ago by jive (previous) (diff)

comment:11 Changed 2 years ago by jive

We are committed to the public URL http://download.osgeo.org/webdav/geotools/ for consumption only.

The build servers (or deploying) can be a different endpoint and that is easy to change in one spot.

Here is the GeoTools? maven script describing distribution management for deploying (the credentials are kept in a settings.xml file on the build server or on the developers machine):

  <distributionManagement>
    <repository>
      <uniqueVersion>false</uniqueVersion>
      <id>osgeo</id>
      <name>Open Source Geospatial Foundation - Maven 2 repo</name>
      <url>dav:http://download.osgeo.org/webdav/geotools/</url>
    </repository>
    <snapshotRepository>
        <id>boundless</id>
        <uniqueVersion>false</uniqueVersion>
        <name>Boundless Snapshot Repository</name>
        <url>https://repo.boundlessgeo.com/snapshot/</url>
    </snapshotRepository>
  </distributionManagement>

comment:12 Changed 2 years ago by jive

Thanks for the help, end of the day:

The pull request to fix geotools deploy https://github.com/geotools/geotools/pull/2385

comment:13 Changed 2 years ago by jive

Resolution: fixed
Status: reopenedclosed
Note: See TracTickets for help on using tickets.