Opened 2 years ago

Closed 2 years ago

#2788 closed task (fixed)

old-web won't start Instance snapshot record count doesn't match instance snapshot volume record count

Reported by: robe Owned by: sac@…
Priority: normal Milestone: Sysadmin Contract 2022-II
Component: SysAdmin Keywords:
Cc:

Description

old-web is having the issue from past of bad snapshots and the newer lxd versions prevent containers from starting if they have them.

This issue I reported upstream and details were here - https://discuss.linuxcontainers.org/t/lxc-snapshot-and-lxc-start-error-instance-snapshot-record-count-doesnt-match-instance-snapshot-volume-record-count

The fix they implemented was to automatically fix these issues where there is no data loss (of backup snapshots). So for this one we have to do manually.

Affected sites are old.www.osgeo.org, fdo.osgeo.org, mapguide.osgeo.org.

Change History (1)

comment:1 by robe, 2 years ago

Resolution: fixed
Status: newclosed

I had deleted the links in nginx container /etc/nginx/sites-enabled/ (for these 3) so nginx wouldn't complain.

steps to fix, from osgeo7 host

lxd sql global "SELECT v.id, v.name FROM (SELECT vs.* FROM storage_volumes AS v INNER JOIN storage_volumes_snapshots AS vs ON v.id = vs.storage_volume_id WHERE v.name = 'old-web') AS v LEFT JOIN 
(SELECT vs.* FROM instances AS v INNER JOIN instances_snapshots AS vs ON v.id = vs.instance_id WHERE v.name = 'old-web') AS i ON i.name = v.name WHERE  i.name IS NULL;"

Returned:

+------+-------+
|  id  | name  |
+------+-------+
| 4719 | snap1 |

Then delete the orphan volume:

lxd sql global "DELETE FROM storage_volumes_snapshots WHERE id IN(4719)"

Then doing the below works

lxc snapshot old-web

lxc start old-web

Once this is started:

lxc exec nginx bash
ln -s /etc/nginx/sites-available/mapguide.osgeo.org /etc/nginx/sites-enabled/mapguide.osgeo.org
systemctl reload nginx

I'm not going to bother with fdo.osgeo.org or old.www.osgeo.org unless someone complains. Those two should not be used anymore. At a glance last update for fdo.osgeo.org was in 2015 and osgeo.org and www.osgeo.org have been moved for a while.

Note: See TracTickets for help on using tickets.