Discourse text/plain email use html entities, markdown and backslash-escapes for no reason

[strk]

I see backslashes in discourse emails, see ​https://matrix.to/#/!bzcNHWCfdsmgNjQSgV:osgeo.org/$7oGHRsTksBseRoGPDBE85zcI9Gxbe69YTe3y1N3Bibg?via=osgeo.org&via=matrix.org&via=dend.ro

I noticed those emails are multipart with TEXT and HTML versions, the broken one is the TEXT version which I instruct my MUA to prefer.

[strk]

See how bad it is:

[robe]

I assume this must be what you are getting via email?

Cause the mirror record looks fine - ​https://discourse.osgeo.org/t/osgeo-3039-problems-with-sending-overflow-attachments/4711

Maybe it's the setting on discourse to prefer html send doing it. It was set that way and I was tempted to change it to plain text, but there doesn't seem to be a setting for that per category, just a global one.

[robe]

The settings I am referring to are here - ​https://discourse.osgeo.org/admin/site_settings/category/email

Incoming email prefer html: yes

I should also add, the default behavior of discourse for

trim incoming emails Trim part of the incoming emails that isn't relevant.

Was set to true. Having this be true, made all trac emails have the body chopped off, cause I guess the ----- that trac uses in email was some sort of signaling to it to ignore the rest of the message.

I also had changed the osgeo trac mail encoding when I was troubleshooting the above issue to quoted-printable. I've changed that back to base64 to see if that fixes this issue.

[robe]

@strk are you seeing problems in other emails besides trac? I know there are issues with trac mail which I haven't figured out. I'd be more concerned if you are seeing issues with other mail.

[strk]

Yes, see the difference:

Mailman archive view of that same email: ​https://lists.osgeo.org/pipermail/sac/2024-January/016358.html:

Discourse view (colors, but useless IMHO): ​https://discourse.osgeo.org/t/email-flooding-cron-root-ldap-web-staging-bash-root-pull-repository-if-exists-sh/6508/4

I clearly don't need markdown in the text/plain version of the email from Discourse.

BTW: I receive a multipart/alternative both from Mailman and from Discourse (I guess that's how Vicky sent that mail), but the Mailman version of the text is correctly formatted.

[robe]

Ah I see what it's doing. Looks like it's trying to markdownify the email.

Maybe If we turn off prefer html incoming or something that will help.

#this by default is on. I just turned it off.

incoming email prefer html Use HTML instead of text for incoming email.

[strk]

I think my problem is with outgoing, not with incoming ?

[robe]

Well I didn't see a setting for outgoing so thought it maybe formats the incoming and sends it outgoing. So it's the incoming formatting that is causing the outgoing format to be screwy.

[strk]

The thing is that the original email is the same that goes into Mailman too, but following that path lands to me in perfect shape. I guess we should report this upstream. Now it would be nice if those who proposed Discourse in the first place took on some responsibility to make it behave...

[strk]

Upstream ticket (a topic in the upstream "bug" category): ​https://meta.discourse.org/t/html-entities-and-escape-characters-in-text-plain-outgoing-email-alternative/290637

[strk]

The upstream ticket is unanswered for 4 days now, I suspect neither Discourse dev nor Discourse proponent care about email mediated communication, making this problem's only possible fix being a "scratch your own itch" project.

I'd rather avoid Discourse for anything I rely upon (trac tickets being one of those things) so hopefully I can be selective about what I want or don't want to receive by mail from it.

[strk]

I've just received a new email and was formatted good:

It might have been fixed by upgrade to 3.2.0.beta5-dev ( #3113 ) I'm closing this as fixed

[strk]

And I'm reopening this because I just got a badly formatted email from Marting Spott:

[robe]

@strk,

It might have been fixed by my unchecking the "Category is mirror" checkbox on sac-tickets. It's probably broken again for trac tickets because I had to check that back, since it was causing issues with trac emails being received. Discourse started rejecting some with Reason "Title already used". I think it might have also broken the threading when I did that.

Anyrate if that is the case, you should not have issues in Testing category, as that is not a mirrored category or have issues in general or any of the real lists.

Hey maybe you can join pgrouting-develop which has moved to discourse and say something there.

[jef]

For the time being I now use this procmail recipe:

:0
* ^List-ID: OSGeo Discourse
{
        :0 c
        osgeo-discourse-orig

        :0 fb
        | sed -e '/Content-Type: text\/plain/,/^----/ { s/\\\(.\)/\1/g; s/&amp;/\&/g; s/&quot;/"/g; s/&nbsp;/ /g; s/&lt;/</g; s/&gt;/>/g; s/&#39;/'"'"'/g; }'

        :0
        * ^X-Discourse-Category: SAC/SAC
        osgeo-discourse-sac

        :0
        * ^X-Discourse-Category: OSGeo Forum Feedback
        osgeo-discourse-forumfeedback

        :0
        * ^X-Discourse-Category: pgRouting
        osgeo-discourse-pgrouting

        :0
        osgeo-discourse
}

from the few mails I got so far. At least the SAC tickets look ok in mutt.

[strk]

Thanks for the recipe, I'll give it a try. I just realized by having mailing list mode off in my preferences I'm not receiving any mail so didn't really have a chance to test it. This comment might send me the email as I now started "watching" the SAC tickets category.

[strk]

By watching the SAC tickets category I finally received the email from Discourse, 10 minutes after I received it directly from Mailman.

I confirm the procmail filter helps in making SAC tickets nicer in Mutt.