Write recommendation for mailing list configuration regarding DKIM/DMARC/SPF

[strk]

Given the amount of trouble mailing lists are having due to anti-spam measures it would be useful if the OSGeo SysAdmin commettee would publish recommendation about how to deal with them.

See ​https://lists.osgeo.org/pipermail/postgis-devel/2023-October/030125.html for some background info

DKIM specs: ​https://dkimcore.org/specification.html

DMARC specs: ​https://dmarc.org/resources/specification

SPF specs: ​http://www.open-spf.org/Specifications/

[strk]

A good explanation of the problem and different solutions is in Greg Troxel mail in the aforementioned thread: ​https://lists.osgeo.org/pipermail/postgis-devel/2023-October/030131.html

[jef]

See also #2985

[strk]

The test we're doing on postgis-tickets revealed some MUA fail to provide an indication of the message coming from a mailing list and give advice on how to unsubscribe. This would degrade the experience of some users and thus keeping the footer seem to still have a value.

[lnicola]

Some more thoughts on this, which I've mentioned in an email before:

• rewriting the sender is really annoying because mail clients tend to add the wrong name to the address book
• it looks like Mailman prefers to strip the incoming DKIM signature; ours preserves it, but adds a Sender header, which probably invalidates the signature
• stripping and re-signing is a valid option
• the postgresql.org list manager is able to pass DKIM through just fine, but is not adding a Sender header
• ​https://mail.python.org/archives/list/mailman-developers@python.org/thread/M3VI4MLIRB5DMWKMLEYUMFM67O5YDPVA/ is a Mailman discussion about the Sender header, which is interesting, but goes nowhere.
• we and postgresql.org send List-Unsubscribe headers, but some email clients don't like those
• ideally (under RFC 8058), we'd have a List-Unsubscribe-Post header and DKIM-sign both to support the one-click thingy; of course, we have no DKIM at all
• I'll take DKIM and the one-click thingy over any message footers

[gdt]

It is clear that we can't have a solution that makes everybody happy. List-Foo headers seem to be pretty clearly a standard, and if someone and their MUA can't deal with that, I don't think we should make everyone suffer from forged From: lines because of it.

I don't understand "stripping and resigning is a valid option". That requires a forged From:.

The Sender: situation is crazy. It is normal for mailinglists to add Sender:. So I see three options:

• Add Sender:. If a message to the list has DKIM covering Sender: and a DMARC policy, reject it, because that domain has declared that messages from its users may not appear on mailing lists.
• With DKIM/Sender and DMARC, skip adding Sender:
• Just don't add Sender either

The first option is righteous but unhelpful. Given the pg experience of not having problems from not adding sender, I think the third option is best.

[lnicola]

List-Foo headers seem to be pretty clearly a standard, and if someone and their MUA can't deal with that, I don't think we should make everyone suffer from forged From: lines because of it.

I opened a ticket with Fastmail for that. But the solution is probably to add List-Unsubscribe-Post (if even possible) and sign the one or two headers with DKIM. I'm not sure how hard that is to set up.

I think the third option is best.

That's my preference too.

[gdt]

How is a user not understanding that to unsubscribe from a list that they should look at headers and find List-Unsubscribe, something that fastmail should address? Are you saying that fastmail provides an MUA that fails to notice these headers and provide an unsubscribe button? Or something else?

(I read mail with emacs/gnus, and it doesn't have any automatic unsubscribe action feature that I know of. When I want off a list, I have no issue just looking at the headers and finding the List-Unsubscribe etc. header and sending a request or finding the mailman page.)

[lnicola]

Yes, they normally show an Unsubscribe link next to the subject, in some cases even when the headers are missing. I don't think we need to worry too much about broken MUAs unless we're doing something wrong.

[strk]

Another case of mailing list messages going to spam: #3016

[strk]

Removing the DKIM headers could be a good first step, and this recipe might be a correct one for mailman2 (and could be global too): ​https://mail.python.org/pipermail/mailman-users/2011-October/072304.html

[strk]

The recipe in the above link is alrady in place for OSGeo, since 2017 (thanks jef) but it's evidently not working as expected

[strk]

It was actually jmckenna adding the REMOVE_DKIM_HEADERS = Yes in May 2019, according to git log, not sure why that change included also a comment supposedly coming from 2017 by Jurgen (jef) --- I think it'll be good to put that configuration in ansible.

[strk]

The help string for that variable:

# Some list posts and mail to the -owner address may contain DomainKey or
# DomainKeys Identified Mail (DKIM) signature headers <http://www.dkim.org/>.
# Various list transformations to the message such as adding a list header or
# footer or scrubbing attachments or even reply-to munging can break these
# signatures.  It is generally felt that these signatures have value, even if
# broken and even if the outgoing message is resigned.  However, some sites
# may wish to remove these headers.  Possible values and meanings are:
# No, 0, False -> do not remove headers.
# Yes, 1, True -> remove headers only if we are munging the from header due
#                 to from_is_list or dmarc_moderation_action.
# 2 -> always remove headers.
# 3 -> always remove, rename and preserve original DKIM headers.
REMOVE_DKIM_HEADERS = No

The current value (YES == 1) is not enough to fix lnicola's case above: ​https://trac.osgeo.org/osgeo/ticket/3011#comment:7

[strk]

It's to be noted that DMARC moderation action can also be set globally rather than expecting each list owner to deal with it. Now the /etc/mailman/mm_cfg.py file is in ansible so further tweaks to it could be done there

[strk]

Configuration for avoiding to to set a "sender" seem to be per-list only, with the only global configuration allowing or forbidding the avoidance:

# RFC 2822 suggests that not adding a Sender header when Mailman is the agent
# responsible for the actual transmission is a breach of the RFC.  However,
# some MUAs (notably Outlook) tend to display the Sender header instead of the
# From details, confusing users and actually losing the original sender when
# forwarding mail.  By setting this variable to Yes, list owners will be
# given the option to avoid setting this header.
ALLOW_SENDER_OVERRIDES = Yes

[strk]

I've changed include_sender_header value to 0 for postgis-tickets mailing list, to test if option 3 in ​https://trac.osgeo.org/osgeo/ticket/3011#comment:8 is viable

[lnicola]

Having tested on postgis-tickets, it seems that disabling the addition of the Sender header is enough to keep DKIM signatures valid.

[strk]

Note I did not find a way to change that setting from the web UI, so I'm not sure how to make it a default also for newly created lists. The incantation I've used to change it for postgis-tickets was this:

list=postgis-tickets
config_list -o - ${list} | 
 sed 's/^include_sender_header =.*/include_sender_header = 0/' |
 config_list -i /dev/stdin ${list}

[gdt]

I think it's entirely unreasonable to remove DKIM headers. The problem is that DMARC expects DKIM to be present and valid, and the only non-problematic approach is to configure the list to not break DKIM (by not modifying the message).

strk's change to stop setting Sender seems good. One could just change the installed mailman code to make it default; it seems that with mail senders including Sender: in DKIM (which is IMHO a bug but we can't fix it), that it's no longer OK for lists to set it, in general.

[strk]

So to recap the current recommendation for mailing list would be to:

1. Set "Replace the From" to "no" ​https://lists.osgeo.org/mailman/admin/postgis-tickets/general

2. Disable "Reply-to" munging ​https://lists.osgeo.org/mailman/admin/postgis-tickets/general

3. Remove footer from the non-digest options ​https://lists.osgeo.org/mailman/admin/postgis-tickets/nondigest

4. Set include_sender_header to false ​https://trac.osgeo.org/osgeo/ticket/3011#comment:21

5. Set subject_prefix to the empty string (no prefix) ​https://lists.osgeo.org/mailman/admin/postgis-tickets/general

I guess next step would be writing a script to set the above configurations programmatically for any list, and try it against the next test list (sac?)

[strk]

It looks like the right place for writing the reccomendations would be the wiki page: ​https://wiki.osgeo.org/wiki/SAC:Mailing_Lists#Configuring_the_mailing_list

For now I've just added a link back to comment:23 but it would proabbly be better to copy all the content there instead.

[strk]

I've added an /osgeo/mailman-tools/recommended_setup script on osgeo6 to more quickly set the configuration of a mailing list. The script is deployed via ansible as of ​https://gitea.osgeo.org/gitea/sac/ansible-deployment/commit/7b621ce4c082e4d8f00035461f389b1e6947625c

The script does NOT yet deal with the footer but prints a link to deal with it yourself. As of today I've changed postgis-devel to use the new configuration and updated the wiki page