Opened 5 years ago

Closed 4 years ago

#2231 closed task (fixed)

OSGeo7 Setting up backup

Reported by: robe Owned by: robe
Priority: normal Milestone: Sysadmin Contract 2019-II
Component: SysAdmin Keywords:
Cc:

Description

We'll need to start setting up some sort of backup for OSGeo7.

There are 2 kinds of backups

1) Would be backup of key files on each container and system 2) Image snapshots

Doing image snapshots is surprisingly fast in lxd, but publishing as an image takes much longer and the tarball when you export can be big as the containers get bigger.

I think you can create a new container from an lxd snapshot even on another server, but I haven't explored that.

For the nextcloud setup I created the nextcloud server on my own local Ubuntu 18.04 lxd server, exported it as a tarball and restored it onto osgeo7. This I did just to confirm it does work.

So steps were:

lxc snapshot nextcloud-ubuntu nextcloud-ubuntu-20181223 #take a snapshot
lxc publish nextcloud-ubuntu/nextcloud-ubuntu-20181223 --alias nextcloud-ubuntu #publish as an image named nextcloud-ubuntu

#confirm got added to images list
lxc image list

#export as a tar ball named nextcloud-ubuntu-20181223.tar.gz
lxc image export nextcloud-ubuntu nextcloud-ubuntu-20181223

I figure maybe we can do a tar export once a month and a snapshot once a week? I'm not sure how much space snapshots take as given how quickly it created them I assume it must be some sort of diff.

Change History (12)

comment:1 by strk, 5 years ago

_where_ does "publish" publishes ?

comment:2 by robe, 5 years ago

strk - That's a good question. It publishes locally on the same zfs storage drive as the containers I think.

I think there is a way to get it to publish to another machine, but I haven't read that far into it and not sure if that other machine has to be an lxd host as well.

comment:3 by strk, 5 years ago

Why not integrating to the existing Bacula service ? https://wiki.osgeo.org/wiki/SAC:Bacula

comment:4 by robe, 5 years ago

I assumed we would in some way just not sure how bacula works and how to set it up for backup

comment:5 by strk, 5 years ago

I tried to log any knowledge built while setting up more backup things on that wiki page. I still feel not completely sure, but I guess we'd better use a single technology, and learn that, for all cases.

Good luck learning !

comment:6 by wildintellect, 5 years ago

Bacula is a client/server system. Install the client on osgeo7, then configure the backup selections on the Server backup with a config file, including what files, how long to keep, etc...

comment:7 by robe, 5 years ago

Milestone: UnplannedSysadmin Contract 2019-I

comment:8 by robe, 5 years ago

Owner: changed from sac@… to robe

comment:9 by robe, 5 years ago

I have setup nextcloud-ubuntu, download, new-secure, nginx to do daily snapshots and to expire the snapshots 30 days after creation. The schedule snapshot I had done weeks ago but just put in the the expiry. We could set it up at the profile level, but I feel a lot of containers don't need frequent backups - e.g. collabora only changes when I do an upgrade. Dronie-server is mostly the sqlite database that has the history is worth recuring backup, though I guess snapthoting it daily won't hurt since the snapshots do a diff of previous snapshot so would be light anyway.

Here is an example of setting backup (this one is for new-secure)

#sets to snbapshot daily at 5:15 PM
lxc config set new-secure snapshots.schedule "15 17 * * *"

#set snapshots newly created to expire in 30 days (must conform to this regex) ^(\d+)(M|H|d|w|m|y)$
lxc config set new-secure snapshots.expiry "30d"

I want to discuss in our next meeting the bacula setup as I don't think all containers need to have a bacula-fd agent so thinking along the lines of copying say postgresql backups and so forth to osge7 host with a cron job (which I started creating) osgeo7 would have a bacula agent that picks these up.

comment:10 by robe, 5 years ago

Milestone: Sysadmin Contract 2019-ISysadmin Contract 2019-II

Moving this to contract 2

comment:11 by robe, 5 years ago

I'm in middle of setting up script on osgeo4 that makes a snapshot of each of the key containers and copys them over in a stopped state.

comment:12 by robe, 4 years ago

Resolution: fixed
Status: newclosed

this was done a while ago

Note: See TracTickets for help on using tickets.