Opened 3 years ago

Closed 2 years ago

#2521 closed task (fixed)

Change secure to use dedicated cert and set up script to copy the cert

Reported by: robe Owned by: sac@…
Priority: normal Milestone: Sysadmin Contract 2021-II
Component: SysAdmin Keywords:
Cc:

Description

secure right now is using a letsencrypt wildcard cert which I never got around to changing to a ldap.osgeo.org cert. So I've been renewing it every 3 months which is not ideal.

My proposed:

Change to use ldap.osgeo.org Setup a script on nginx that when it renews it copies the script to secure container.

I do have a script already that copies the cert but it can be improved to not require manual stuff. Something a savyor network admin like strk can do inhis sleep.

I have what I am currently doing outlined here

https://git.osgeo.org/gitea/sac/osgeo7/wiki/NGinx-Proxy-container#user-content-generating-wildcard-cert-for-osgeo-org

Change History (14)

comment:1 by robe, 3 years ago

Milestone: UnplannedSysadmin Contract 2020-II

comment:2 by strk, 3 years ago

I'm a bit confused by this ticket. According to https://wiki.osgeo.org/wiki/SAC_Service_Status#NGINX_proxy the nginx container proxies all traffic, then why should the cert be installed in the underlying container ? Is ldap.osgeo.org out of the proxy ? What makes ldap.osgeo.org different from other containers ?

comment:3 by strk, 3 years ago

Answering myself: LDAP is different in that it's NOT using the certificate in a web server but a LDAP server. We could have nginx indeed respond to http requests for the LDAP server IP address for the sole purpose of doing http validation for letsencrypt. About the copy I wonder if we should be instead MOUNTING the certificate directory directly, instead of copying it ?

comment:4 by robe, 3 years ago

Mounting sounds like a cleaner approach. Only issue with that is it makes it less portable.

comment:5 by strk, 3 years ago

Ok I've implemented a copy script that sits in /root on osgeo7 which pulls keys from nginx and pushes them to secure, restarting slapd IFF any of those keys changed.

I've run the script manually for now. Running openssl s_client -connect localhost:636 -showcerts from the secure container shows the new certificate is in use:

Server certificate
subject=/CN=ldap.osgeo.org
issuer=/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3

Next: add a cronjob running the script

comment:6 by strk, 3 years ago

The cronjob should be written in the new ansible deployment playbooks: https://git.osgeo.org/gitea/sac/ansible-deployment

comment:7 by strk, 3 years ago

Cron job is now in place and deployed via ansible. It has to be tested that it works correctly, including when errors arise, as I'm not sure if a mail would ever be received on error (no MTA on osgeo7?)

comment:8 by strk, 3 years ago

Cron mails will arrive, as per #2536 - now it's to be tested if those mails are too noisy, and then if renewal is effective.

comment:9 by strk, 3 years ago

I've added the ldap.osgeo.org virtual host configuration to the ansible deployment repository with https://git.osgeo.org/gitea/sac/ansible-deployment/commit/971e9a6fa5661353337f39194557644ac1609b54

Didn't add more virtual hosts but we should, in a separate ticket.

Only things left out of ansible yet is:

  • certbot configuration for ldap.osgeo.org
  • LXC container configuration to ensure a device listening on ldap.osgeo.org is added to the 'nginx container

comment:10 by strk, 3 years ago

nginx container setup was added to ansible playbooks with https://git.osgeo.org/gitea/sac/ansible-deployment/commit/711a269093b1bb460a7180c5f5d39e2d6ade48fb

Only thin left out of ansible is now certbot configuration for ldap.osgeo.org

comment:11 by strk, 3 years ago

Milestone: Sysadmin Contract 2020-IISysadmin Contract 2020-III

Ticket retargeted after milestone closed

comment:12 by strk, 3 years ago

Milestone: Sysadmin Contract 2020-IIISysadmin Contract 2020-II

comment:13 by robe, 2 years ago

Milestone: Sysadmin Contract 2020-IISysadmin Contract 2021-II

Okay this is till not working so reopening it.

When I do this:

Check if secure's cert expire date

 openssl s_client -servername ldap.osgeo.org -connect ldap.osgeo.org:636 2>/dev/null | openssl x509 -noout -dates

I get:

notBefore=Sep 29 09:16:39 2021 GMT
notAfter=Dec 28 09:16:38 2021 GMT

When I check the recently received cert on nginx:

 openssl s_client -servername ldap.osgeo.org -connect ldap.osgeo.org:443 2>/dev/null | openssl x509 -noout -dates

I get

notBefore=Nov 28 19:31:56 2021 GMT
notAfter=Feb 26 19:31:55 2022 GMT

If I restart slapd on osgeo7-secure

systemctl restart slapd

It does not fix the issue.

So it seems the cronjob is not working.

The cronjob on osgeo7 looks

 sudo systemctl status cron

shows this:

Dec 05 12:30:01 osgeo7 CRON[25890]: (tech_dev) CMD (/usr/local/bin/copy_ldap_certs_to_secure.sh)
Dec 05 12:30:01 osgeo7 cron[4341]: sendmail: fatal: open /etc/postfix/main.cf: Permission denied
Dec 05 12:30:01 osgeo7 CRON[25889]: (tech_dev) MAIL (mailed 109 bytes of output but got status 0x004b from MTA
                                    )
Dec 05 12:30:01 osgeo7 CRON[25889]: pam_unix(cron:session): session closed for user tech_dev
Dec 05 13:17:01 osgeo7 CRON[7005]: pam_unix(cron:session): session opened for user root by (uid=0)
Dec 05 13:17:01 osgeo7 CRON[7009]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Dec 05 13:17:01 osgeo7 CRON[7005]: pam_unix(cron:session): session closed for user root
Dec 05 14:17:01 osgeo7 CRON[18083]: pam_unix(cron:session): session opened for user root by (uid=0)
Dec 05 14:17:01 osgeo7 CRON[18084]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Dec 05 14:17:01 osgeo7 CRON[18083]: pam_unix(cron:session): session closed for user root

comment:14 by robe, 2 years ago

Resolution: fixed
Status: newclosed

okay looks like a permission issue.

I can't run under the osgeo7 tech_dev account

  /usr/local/bin/copy_ldap_certs_to_secure.sh

Get this

Error: open /var/lib/snapd/hostfs/etc/cron.d/fullchain.pem.new: permission denied
tech_dev@osgeo7:/etc/cron.d$ sudo /usr/local/bin/copy_ldap_certs_to_secure.sh
fullchain.pem.new fullchain.pem.current differ: byte 34, line 2

But if I do

 sudo /usr/local/bin/copy_ldap_certs_to_secure.sh

It works. I'm gong to change the cronjob to do sudo.

strk feel free to change again if you don't feel that is right.

Note: See TracTickets for help on using tickets.