OSGeo7 Setting up backup

[robe]

We'll need to start setting up some sort of backup for OSGeo7.

There are 2 kinds of backups

1) Would be backup of key files on each container and system 2) Image snapshots

Doing image snapshots is surprisingly fast in lxd, but publishing as an image takes much longer and the tarball when you export can be big as the containers get bigger.

I think you can create a new container from an lxd snapshot even on another server, but I haven't explored that.

For the nextcloud setup I created the nextcloud server on my own local Ubuntu 18.04 lxd server, exported it as a tarball and restored it onto osgeo7. This I did just to confirm it does work.

So steps were:

lxc snapshot nextcloud-ubuntu nextcloud-ubuntu-20181223 #take a snapshot
lxc publish nextcloud-ubuntu/nextcloud-ubuntu-20181223 --alias nextcloud-ubuntu #publish as an image named nextcloud-ubuntu

#confirm got added to images list
lxc image list

#export as a tar ball named nextcloud-ubuntu-20181223.tar.gz
lxc image export nextcloud-ubuntu nextcloud-ubuntu-20181223

I figure maybe we can do a tar export once a month and a snapshot once a week? I'm not sure how much space snapshots take as given how quickly it created them I assume it must be some sort of diff.

[strk]

_where_ does "publish" publishes ?

[robe]

strk - That's a good question. It publishes locally on the same zfs storage drive as the containers I think.

I think there is a way to get it to publish to another machine, but I haven't read that far into it and not sure if that other machine has to be an lxd host as well.

[strk]

Why not integrating to the existing Bacula service ? ​https://wiki.osgeo.org/wiki/SAC:Bacula

[robe]

I assumed we would in some way just not sure how bacula works and how to set it up for backup

[strk]

I tried to log any knowledge built while setting up more backup things on that wiki page. I still feel not completely sure, but I guess we'd better use a single technology, and learn that, for all cases.

Good luck learning !

[wildintellect]

Bacula is a client/server system. Install the client on osgeo7, then configure the backup selections on the Server backup with a config file, including what files, how long to keep, etc...

[robe]

I have setup nextcloud-ubuntu, download, new-secure, nginx to do daily snapshots and to expire the snapshots 30 days after creation. The schedule snapshot I had done weeks ago but just put in the the expiry. We could set it up at the profile level, but I feel a lot of containers don't need frequent backups - e.g. collabora only changes when I do an upgrade. Dronie-server is mostly the sqlite database that has the history is worth recuring backup, though I guess snapthoting it daily won't hurt since the snapshots do a diff of previous snapshot so would be light anyway.

Here is an example of setting backup (this one is for new-secure)

#sets to snbapshot daily at 5:15 PM
lxc config set new-secure snapshots.schedule "15 17 * * *"

#set snapshots newly created to expire in 30 days (must conform to this regex) ^(\d+)(M|H|d|w|m|y)$
lxc config set new-secure snapshots.expiry "30d"

I want to discuss in our next meeting the bacula setup as I don't think all containers need to have a bacula-fd agent so thinking along the lines of copying say postgresql backups and so forth to osge7 host with a cron job (which I started creating) osgeo7 would have a bacula agent that picks these up.

[robe]

Moving this to contract 2

[robe]

I'm in middle of setting up script on osgeo4 that makes a snapshot of each of the key containers and copys them over in a stopped state.

[robe]

this was done a while ago