Opened 14 months ago

Last modified 13 months ago

#3011 new task

Write recommendation for mailing list configuration regarding DKIM/DMARC/SPF

Reported by: strk Owned by: sac@…
Priority: normal Milestone: Unplanned
Component: SysAdmin/Mailman Keywords: dkim, spf, dmarc
Cc: Jeff McKenna

Description

Given the amount of trouble mailing lists are having due to anti-spam measures it would be useful if the OSGeo SysAdmin commettee would publish recommendation about how to deal with them.

See https://lists.osgeo.org/pipermail/postgis-devel/2023-October/030125.html for some background info

DKIM specs: https://dkimcore.org/specification.html

DMARC specs: https://dmarc.org/resources/specification

SPF specs: http://www.open-spf.org/Specifications/

Change History (25)

comment:1 by strk, 14 months ago

A good explanation of the problem and different solutions is in Greg Troxel mail in the aforementioned thread: https://lists.osgeo.org/pipermail/postgis-devel/2023-October/030131.html

comment:2 by strk, 14 months ago

Component: SysAdminSysAdmin/Mailman
Keywords: dkim spf dmarc added

comment:3 by jef, 14 months ago

See also #2985

Last edited 14 months ago by jef (previous) (diff)

comment:4 by jef, 14 months ago

Summary: Write recommandation for mailing lits configuration reguarding DKIM/DMARC/SPFWrite recommendation for mailing lits configuration reguarding DKIM/DMARC/SPF

comment:5 by gdt, 14 months ago

Summary: Write recommendation for mailing lits configuration reguarding DKIM/DMARC/SPFWrite recommendation for mailing list configuration regarding DKIM/DMARC/SPF

comment:6 by strk, 13 months ago

The test we're doing on postgis-tickets revealed some MUA fail to provide an indication of the message coming from a mailing list and give advice on how to unsubscribe. This would degrade the experience of some users and thus keeping the footer seem to still have a value.

comment:7 by lnicola, 13 months ago

Some more thoughts on this, which I've mentioned in an email before:

  • rewriting the sender is really annoying because mail clients tend to add the wrong name to the address book
  • it looks like Mailman prefers to strip the incoming DKIM signature; ours preserves it, but adds a Sender header, which probably invalidates the signature
  • stripping and re-signing is a valid option
  • the postgresql.org list manager is able to pass DKIM through just fine, but is not adding a Sender header
  • https://mail.python.org/archives/list/mailman-developers@python.org/thread/M3VI4MLIRB5DMWKMLEYUMFM67O5YDPVA/ is a Mailman discussion about the Sender header, which is interesting, but goes nowhere.
  • we and postgresql.org send List-Unsubscribe headers, but some email clients don't like those
  • ideally (under RFC 8058), we'd have a List-Unsubscribe-Post header and DKIM-sign both to support the one-click thingy; of course, we have no DKIM at all
  • I'll take DKIM and the one-click thingy over any message footers
Last edited 13 months ago by lnicola (previous) (diff)

comment:8 by gdt, 13 months ago

It is clear that we can't have a solution that makes everybody happy. List-Foo headers seem to be pretty clearly a standard, and if someone and their MUA can't deal with that, I don't think we should make everyone suffer from forged From: lines because of it.

I don't understand "stripping and resigning is a valid option". That requires a forged From:.

The Sender: situation is crazy. It is normal for mailinglists to add Sender:. So I see three options:

  • Add Sender:. If a message to the list has DKIM covering Sender: and a DMARC policy, reject it, because that domain has declared that messages from its users may not appear on mailing lists.
  • With DKIM/Sender and DMARC, skip adding Sender:
  • Just don't add Sender either

The first option is righteous but unhelpful. Given the pg experience of not having problems from not adding sender, I think the third option is best.

comment:9 by lnicola, 13 months ago

List-Foo headers seem to be pretty clearly a standard, and if someone and their MUA can't deal with that, I don't think we should make everyone suffer from forged From: lines because of it.

I opened a ticket with Fastmail for that. But the solution is probably to add List-Unsubscribe-Post (if even possible) and sign the one or two headers with DKIM. I'm not sure how hard that is to set up.

I think the third option is best.

That's my preference too.

Last edited 13 months ago by lnicola (previous) (diff)

comment:10 by gdt, 13 months ago

How is a user not understanding that to unsubscribe from a list that they should look at headers and find List-Unsubscribe, something that fastmail should address? Are you saying that fastmail provides an MUA that fails to notice these headers and provide an unsubscribe button? Or something else?

(I read mail with emacs/gnus, and it doesn't have any automatic unsubscribe action feature that I know of. When I want off a list, I have no issue just looking at the headers and finding the List-Unsubscribe etc. header and sending a request or finding the mailman page.)

comment:11 by lnicola, 13 months ago

Yes, they normally show an Unsubscribe link next to the subject, in some cases even when the headers are missing. I don't think we need to worry too much about broken MUAs unless we're doing something wrong.

comment:12 by strk, 13 months ago

Another case of mailing list messages going to spam: #3016

comment:13 by strk, 13 months ago

Removing the DKIM headers could be a good first step, and this recipe might be a correct one for mailman2 (and could be global too): https://mail.python.org/pipermail/mailman-users/2011-October/072304.html

comment:14 by strk, 13 months ago

The recipe in the above link is alrady in place for OSGeo, since 2017 (thanks jef) but it's evidently not working as expected

comment:15 by strk, 13 months ago

Cc: Jeff McKenna added

It was actually jmckenna adding the REMOVE_DKIM_HEADERS = Yes in May 2019, according to git log, not sure why that change included also a comment supposedly coming from 2017 by Jurgen (jef) --- I think it'll be good to put that configuration in ansible.

comment:16 by strk, 13 months ago

The help string for that variable:

# Some list posts and mail to the -owner address may contain DomainKey or
# DomainKeys Identified Mail (DKIM) signature headers <http://www.dkim.org/>.
# Various list transformations to the message such as adding a list header or
# footer or scrubbing attachments or even reply-to munging can break these
# signatures.  It is generally felt that these signatures have value, even if
# broken and even if the outgoing message is resigned.  However, some sites
# may wish to remove these headers.  Possible values and meanings are:
# No, 0, False -> do not remove headers.
# Yes, 1, True -> remove headers only if we are munging the from header due
#                 to from_is_list or dmarc_moderation_action.
# 2 -> always remove headers.
# 3 -> always remove, rename and preserve original DKIM headers.
REMOVE_DKIM_HEADERS = No

The current value (YES == 1) is not enough to fix lnicola's case above: https://trac.osgeo.org/osgeo/ticket/3011#comment:7

comment:17 by strk, 13 months ago

It's to be noted that DMARC moderation action can also be set globally rather than expecting each list owner to deal with it. Now the /etc/mailman/mm_cfg.py file is in ansible so further tweaks to it could be done there

comment:18 by strk, 13 months ago

Configuration for avoiding to to set a "sender" seem to be per-list only, with the only global configuration allowing or forbidding the avoidance:

# RFC 2822 suggests that not adding a Sender header when Mailman is the agent
# responsible for the actual transmission is a breach of the RFC.  However,
# some MUAs (notably Outlook) tend to display the Sender header instead of the
# From details, confusing users and actually losing the original sender when
# forwarding mail.  By setting this variable to Yes, list owners will be
# given the option to avoid setting this header.
ALLOW_SENDER_OVERRIDES = Yes

comment:19 by strk, 13 months ago

I've changed include_sender_header value to 0 for postgis-tickets mailing list, to test if option 3 in https://trac.osgeo.org/osgeo/ticket/3011#comment:8 is viable

comment:20 by lnicola, 13 months ago

Having tested on postgis-tickets, it seems that disabling the addition of the Sender header is enough to keep DKIM signatures valid.

comment:21 by strk, 13 months ago

Note I did not find a way to change that setting from the web UI, so I'm not sure how to make it a default also for newly created lists. The incantation I've used to change it for postgis-tickets was this:

list=postgis-tickets
config_list -o - ${list} | 
 sed 's/^include_sender_header =.*/include_sender_header = 0/' |
 config_list -i /dev/stdin ${list}

comment:22 by gdt, 13 months ago

I think it's entirely unreasonable to remove DKIM headers. The problem is that DMARC expects DKIM to be present and valid, and the only non-problematic approach is to configure the list to not break DKIM (by not modifying the message).

strk's change to stop setting Sender seems good. One could just change the installed mailman code to make it default; it seems that with mail senders including Sender: in DKIM (which is IMHO a bug but we can't fix it), that it's no longer OK for lists to set it, in general.

comment:23 by strk, 13 months ago

So to recap the current recommendation for mailing list would be to:

  1. Set "Replace the From" to "no" https://lists.osgeo.org/mailman/admin/postgis-tickets/general
  1. Disable "Reply-to" munging https://lists.osgeo.org/mailman/admin/postgis-tickets/general
  1. Remove footer from the non-digest options https://lists.osgeo.org/mailman/admin/postgis-tickets/nondigest
  1. Set include_sender_header to false https://trac.osgeo.org/osgeo/ticket/3011#comment:21
  1. Set subject_prefix to the empty string (no prefix) https://lists.osgeo.org/mailman/admin/postgis-tickets/general

I guess next step would be writing a script to set the above configurations programmatically for any list, and try it against the next test list (sac?)

Last edited 13 months ago by strk (previous) (diff)

comment:24 by strk, 13 months ago

It looks like the right place for writing the reccomendations would be the wiki page: https://wiki.osgeo.org/wiki/SAC:Mailing_Lists#Configuring_the_mailing_list

For now I've just added a link back to comment:23 but it would proabbly be better to copy all the content there instead.

comment:25 by strk, 13 months ago

I've added an /osgeo/mailman-tools/recommended_setup script on osgeo6 to more quickly set the configuration of a mailing list. The script is deployed via ansible as of https://gitea.osgeo.org/gitea/sac/ansible-deployment/commit/7b621ce4c082e4d8f00035461f389b1e6947625c

The script does NOT yet deal with the footer but prints a link to deal with it yourself. As of today I've changed postgis-devel to use the new configuration and updated the wiki page

Note: See TracTickets for help on using tickets.