rsync issue on staging.grass.osgeo.org

[neteler]

In preparation of making the new GRASS GIS website mirror'able via rsync, I have added rsync support to ​https://staging.grass.osgeo.org but it apparently requires something to be done on the host server.

ssh grasslxd
...
neteler@grass:~$ rsync localhost::
GRASS GIS Website
grass-website      GRASS GIS Website

.. so far so nice.

But from external IP I only get this:

rsync staging.grass.osgeo.org::
download           The whole OSGeo Download Area
bottle             Homebrew packages

Nothing against those entries but I'd like to see the grass-website here :-)

[robe]

I'll look at this in a bit

[robe]

neteler,

Where are you mirroring too?

The way I rsync from one lxd host to say another host is this way. Since download.osgeo.org (aka hop.osgeo7.osgeo.org) is the proxy you'll need to copy keys there as well

Step 1

# in server you are using to mirror #generate a key for server you want to mirror to

ssh-keygen -t rsa #accept the defaults
#copy the public key ~/.ssh/id_rsa.pub add it to your hop.osgeo7.osgeo.org authorized_keys

Step 2 #log into ssh neteler@…

echo ssh-rsa --contents-public-key-here- grass@osgeo7 >> ~/.ssh/authorized_keys

Step 3 #in grass container in the account you will be using for rsync

echo ssh-rsa --contents-public-key-here- grass@osgeo7 >> ~/.ssh/authorized_keys

Step 4 in the server you want to use for mirror

Add an entry to your ssh config

**create in server you want to mirror content ~/.ssh/config file that has these lines**
    Host grass
        ProxyCommand ssh neteler@hop.osgeo7.osgeo.org -W %h:%p

Then to rsync you can do

#test inspect
rsync -avz --dry-run  \
    -e ssh root@grass:/var/www/grass-website \
     /backups/grass-website

#real run
rsync -avz  \
    -e ssh root@grass:/var/www/grass-website \
     /backups/grass-website

Other option is I could open a port in osgeo7 and allow grass to be directly reachable by that port. If we do that I would like to lock down grass so it's only accessible via keys and block off password access.

[neteler]

I am sorry, I should have mentioned that we wish to have a public mirror as on the old server (to be phased out): ​https://grass.osgeo.org/mirrors/

Would that be possible my opening the respective port?

[robe]

Yes I think jef had set up an rsync that was public on download for osuosl. I think it might have used http too. Didn't quite follow what he did, but that might work here.

I didn't quite follow what he was doing, but I think what he did might not have used the ssh port at all.

[robe]

neteler,

I see that jef set it up the way you did using rsync daemon that runs on port 873. I can't easily give you the same one without allocating another ip which would mean you can't use the same domain. Instead I mapped your 873 container port out to 50026.

So to do your rsync you have to specify the port. See if this works okay. I tested on one of my servers and see grass website listed now

rsync --port=50026 staging.grass.osgeo.org::

Gives me

GRASS GIS Website

grass-website   GRASS GIS Website

Feel free to reopen if this doesn't solve your issue.

[neteler]

Looks good, thanks for your efforts!

[neteler]

Just for the record: this is the command to mirror:

rsync -az --port=50026 staging.grass.osgeo.org::grass-website grass-website

(in the near future the DNS will be changed to grass.osgeo.org)