Create a geos and postgis docker repo on repo.osgeo.org

[robe]

This is mostly to replace the private registry strk is currently running.

We will start by using it to hold images for our geos/postgis bots and maybe eventually expand the use for other things.

These should be separate repositories/

[robe]

For permissions add robe, strk for starters. We'll add one for jenkins bot later.

[jive]

I setup the postgis one for your review:

• postgis-docker repository created, allowed anonymous access, but turned off v1 api access
• docker group now includes postgis-docker
• setup postgis-admin role giving it permissions for the postgis-docker repository
• robe and strk users have been granted postgis-admin role (they were both admins already but whatever)
• created a local user postgisbuild user with the above postgis-admin role, it uses robe's email for notifications (please adjust this user as needed for your jenkin). This mirrors what was done for geoserver jenkins.

[jive]

Please try out the above repository, and write up how it works for the wiki page!

General approach:

1. Setting up your osgeo userid credentials in ~/.docker/config.json
2. Push to ​https://repo.osgeo.org/repository/postgis-docker/
3. Please pull anonymously from ​https://repo.osgeo.org/repository/docker/

References:

• ​https://blog.sonatype.com/using-nexus-3-as-your-repository-part-3-docker-images
• ​https://help.sonatype.com/repomanager3/formats/docker-registry/pulling-images
• ​https://help.sonatype.com/repomanager3/formats/docker-registry/pushing-images
• ​https://help.sonatype.com/repomanager3/formats/docker-registry/authentication

[strk]

Instructions ​https://blog.sonatype.com/using-nexus-3-as-your-repository-part-3-docker-images

[strk]

Do we need a specific port ?

[strk@liz:~] docker login repo.osgeo.org
Username: strk
Password:
Error response from daemon: login attempt to https://repo.osgeo.org/v2/ failed with status: 404 Not Found

[robe]

strk that's what I was saying that's the same error I get.

All the videos I've seen they explicitly have docker registry run on a separate port. Because you can't give a path for login.

I think we could just setup another domain like docker.osgeo.org and have it connect to port 8083 or something in nexus. But wasn't sure if there was a way around that.

So I'm thinking we set docker to explicitly be on specific port (of course I would need to expose that too on the nexus docker (or maybe not maybe that could be a path)

and then docker.osgeo.org goes to that.

[robe]

aha here it describes the issue

​https://help.sonatype.com/repomanager3/formats/docker-registry/ssl-and-repository-connector-configuration

---

The docker client does not allow a context as part of the path to a registry, as the namespace and image name are embedded in the URLs it uses. This is why requests to repositories on the repository manager are served on a specific and separate port from the rest of the application instead of how most other repositories serve content via a path i.e. <nexus-hostname>/<repositoryName>/<path to content> .

---

[robe]

This one is interesting

​https://blog.sonatype.com/setting-up-a-docker-private-registry-with-authentication-using-nexus-and-nginx

It uses a single nginx proxy config, but if the agent is docker, then redirects to registry port, but don't see a way of getting around need to open an additional port on nexus docker container and if we have more than one docker reigstry I think we'll need a port for each so we should just put them in now.

[strk]

DISCLAIMER: I did not read the articles in those links

Are you saying that Nexus does not allow having different permissions for writing in different subdirs ?

I do like the idea of using docker.osgeo.org

[robe]

strk,

I don't understand the lingo of registry vs. folder etc.

My understanding is with registry which is the way I think jive has it set up -- each registry can completely manage their roles/etc. I think if it's a single registry you can't but maybe one registry is sufficient.

But anyway the whole path issue is more of a limitation in docker itself, not nexus. That docker registries need to authenticate at the root because the path is encoded in the tag. DISCLAIMER - I may not know what I am talking about.

At anyrate thinking of copying over nexus container on osgeo4 to experiment. Need a backup there anyway. I think even though in theory I can open up ports on a docker container running -- it's not supported, so I'd rather shut it down and start it up with many ports which means there would be like 5 minutes of downtime while we do this. If there is no way around this whole having to run in a port.

I would think with nginx -- we could point a path like docker.osgeo.org -> nexus.lxd:8081/docker

but I have not seen anyone doing that so maybe it's not doable.

[robe]

Okay I was able to successfully login on my dev container. Had to add Docker Bearer Token Realm (to the nexus -> Realm ) section, in addition to using a separate port.

I'm still unclear if we can get away with just a single port for all docker repositories. Still experimenting with that.

[robe]

found this thread which seems to do it without additional ports, just rewriting the docker calls

​https://stackoverflow.com/questions/47178055/nexus3-push-to-docker-group-repo

I'll give that a try in dev.

[robe]

okay tried in dev and worked, I put in place on repo.osgeo.org (without specifying any ports), just rewriting the path calls and was then able to log in with

docker login

​https://git.osgeo.org/gitea/sac/osgeo3/commit/c48afd1b84a1c1c85a831cfa6a51f291311d6f1d

But I haven't tried committing (and not sure what paths should be put in for push to differentiate the repos.

I was able to push in dev (but that was with port explicitly for postgis-docker, and then it appeared when I browsed both the docker and postgis-docker)

[robe]

Okay I think I got this working. I created a new docker.osgeo.org nginx config dedicated. I could put it all in the nexus one, but felt might be better to keep it separate. I'm planning to eventually take out all the /v2 stuff I put in on the nexus config.

So the way it works, all pushes must go thru the project repo

e.g postgis-docker.osgeo.org, geoserver-docker.osgeo.org, geos-docker.osgeo.org

But pulls go thru

docker.osgeo.org

To test I copied over strk's images:

docker pull docker.kbt.io/postgis/build-test:trisquel2
docker tag docker.kbt.io/postgis/build-test:trisquel2 postgis-docker.osgeo.org/postgis/build-test:trisquel2
docker push postgis-docker.osgeo.org/postgis/build-test:trisquel2

and that shows a new image in postgis-docker repository folder (and of course exposed in the docker group

committed at - ​https://git.osgeo.org/gitea/sac/osgeo3/commit/87932245f05841f0413053e5f824dc0cd5bfae46

So key area of nginx script looks like this sorry about the crappy indentation, going to fix that next

 location ~ ^/(v1|v2)/[^/]+/?[^/]+/blobs/ {
           if ($request_method ~* (GET) ){
                        rewrite ^/(.*)$ /repository/docker/$1 last;
                }
           if ($host = postgis-docker.osgeo.org ){
                rewrite ^/(.*)$ /repository/postgis-docker/$1 last;
            }
          if ($host = geos-docker.osgeo.org ){
                rewrite ^/(.*)$ /repository/geos-docker/$1 last;
          }
          if ($host = geoserver-docker.osgeo.org ) {
                rewrite ^(.*)$ /repository/geoserver-docker/$1 last;
          }


            rewrite ^/(.*)$ /repository/docker/$1 last;
        }

      location ~ ^/(v1|v2)/ {
                if ($request_method ~* (GET) ){
                        rewrite ^/(.*)$ /repository/docker/$1 last;
                }
                if ($host = postgis-docker.osgeo.org ) {
                        rewrite ^/(.*)$ /repository/postgis-docker/$1 last;
                }
                if ($host = geos-docker.osgeo.org ) {
                        rewrite ^/(.*)$ /repository/geos-docker/$1 last;
                }
                if ($host = geoserver-docker.osgeo.org ) {
                        rewrite ^/(.*)$ /repository/geoserver-docker/$1 last;
                }

                rewrite ^/(.*)$ /repository/docker/$1 last;
        }


    location / {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                #try_files $uri $uri/ =404;
                client_max_body_size 0;
                include /etc/nginx/proxy_protocol_params;
                #need to change this if using https on server and have a redirect
                proxy_pass http://nexus.lxd:8081;
                proxy_redirect off;
        }

[robe]

I'm going to close this out. I tested on one of the dronie agents by doing this

docker rmi docker.kbt.io/postgis/build-test:trisquel2  #if I don't do this then it just tags the pulled (smart enough to know it's already been pulled from another server)

docker pull docker.osgeo.org/postgis/build-test:trisquel2

I also setup geos-docker and pushed the docker.kbt.io/geos/build-test:alpine

jive - when you get the chance, can you confirm your group can push to

docker login geoserver-docker.osgeo.org
docker push ...

[jive]

We are just trying it out now (to store some cite testing images).

Q: Do you know how the docker hub ​https://hub.docker.com/u/osgeo is managed?