Opened 9 months ago
Last modified 6 weeks ago
#3104 reopened task
geoserver-users SF mailman to discourse?
Reported by: | jive | Owned by: | |
---|---|---|---|
Priority: | normal | Milestone: | Sysadmin Contract 2024-II |
Component: | SysAdmin | Keywords: | |
Cc: |
Description
The GeoServer team has expressed interest in using discourse; initially for the user-list content: https://sourceforge.net/p/geoserver/mailman/
Is it possible to move the geoserver-users email list from source forge mailman to discourse?
It looks like an export is available for admins: https://sourceforge.net/p/forge/documentation/Mailing%20List%20Archives/
But it says:
mbox archives contain only obfuscated email addresses
I wonder if it will work?!
Attachments (1)
Change History (28)
comment:1 by , 9 months ago
Milestone: | Unplanned → Sysadmin Contract 2024-II |
---|
comment:2 by , 9 months ago
I was unable to download following this link:
The error was:
The mailing-list mbox was not found.
I will ask another geoserver admin to try, and then we may have to open a source forge support ticket.
comment:3 by , 8 months ago
Andrea has been able to obtained the mbox download from SF so this activity can now be worked. Please contact Andrea for a file (from a few weeks ago) that can be used for testing.
We can try it out, and then communicate the transition to our community.
comment:4 by , 8 months ago
Anyone with admin rights can now download the mbox from the link above, fresh of the day, when the migration is ready to go.
comment:5 by , 8 months ago
@aaime
I do not have admin rights there.
I need someone to send me the mbox via mail. vicky at erosion.dev
comment:7 by , 8 months ago
I tried the import, basically getting errors on all mails
Short version:
... Failed to index message in /shared/import/data/geoserver-users/geoserver-users.mbox at lines 33-1899 ... Failed to index message in /shared/import/data/geoserver-users/geoserver-users.mbox at lines 1900-1953 ... Failed to index message in /shared/import/data/geoserver-users/geoserver-users.mbox at lines 1954-2056 ... Failed to index message in /shared/import/data/geoserver-users/geoserver-users.mbox at lines 2057-2118 ... Failed to index message in /shared/import/data/geoserver-users/geoserver-users.mbox at lines 2119-2220 ...
The error between each "Failed to index"
undefined method `sub' for :from:Symbol ["/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:91:in `block in index_emails'", "/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:158:in `block (2 levels) in all_messages'", "/ var/www/discourse/script/import_scripts/mbox/support/indexer.rb:190:in `block in each_mail'", "/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:210:in `block in each_line'", "/var/www/discourse/s cript/import_scripts/mbox/support/indexer.rb:210:in `each_line'", "/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:210:in `each_line'", "/var/www/discourse/script/import_scripts/mbox/support/ind exer.rb:187:in `each_mail'", "/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:150:in `block in all_messages'", "/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:143:in `foreach'" , "/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:143:in `all_messages'", "/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:67:in `index_emails'", "/var/www/discourse/script/imp ort_scripts/mbox/support/indexer.rb:25:in `block in execute'", "/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:22:in `each'", "/var/www/discourse/script/import_scripts/mbox/support/indexer.rb:2 2:in `execute'", "/var/www/discourse/script/import_scripts/mbox/importer.rb:43:in `index_messages'", "/var/www/discourse/script/import_scripts/mbox/importer.rb:27:in `execute'", "/var/www/discourse/script/import _scripts/base.rb:47:in `perform'", "script/import_scripts/mbox.rb:13:in `<module:Mbox>'", "script/import_scripts/mbox.rb:11:in `<module:ImportScripts>'", "script/import_scripts/mbox.rb:10:in `<main>'"]
Example problematic header:
From aurelio@...3... Mon Jan 19 10:58:27 2004 Message-ID: <001f01c3dec6$d49f0cb0$6801a8c0@...4...> From: =?iso-8859-1?Q?Aur=E9lio_Calegari?= <aurelio@...3...> To: <geoserver-users@lists.sourceforge.net> Date: Mon, 19 Jan 2004 17:00:04 -0300 MIME-Version: 1.0
An example of a header on an osgeo mailinglist
From sentto-3377054-1-991426944-postgis=postgis.refractions.net@returns.onelist.com Fri Jun 1 13:22:16 2001 Return-Path: <sentto-3377054-1-991426944-postgis=postgis.refractions.net@returns.onelist.com> Delivered-To: postgis@offsite.refractions.net Received: from fj.egroups.com (fj.egroups.com [64.211.240.231]) by offsite.refractions.net (Postfix) with SMTP id 4C17A43490 for <postgis@postgis.refractions.net>; Fri, 1 Jun 2001 13:22:15 -0700 (PDT) X-eGroups-Return: sentto-3377054-1-991426944-postgis=postgis.refractions.net@returns.onelist.com
Probably the aurelio@...3...
From aurelio@...3... Mon Jan 19 10:58:27 2004
Doing a grep '^From ' geoserver-users.mbox
From andrea.aime@...84... Fri Sep 27 19:40:34 2013 From mihail12@...84... Sun Sep 29 17:34:14 2013 From peter.vonminden@...2631... Mon Sep 30 02:45:30 2013 From andrea.aime@...84... Mon Sep 30 06:13:50 2013 From maurobartolomeoli@...84... Mon Sep 30 07:02:51 2013 From paulius@...4984... Mon Sep 30 07:36:57 2013 From alessio.fabiani@...84... Mon Sep 30 07:44:47 2013 From paulius@...4984... Mon Sep 30 08:01:16 2013 From andrea.aime@...84... Mon Sep 30 08:09:38 2013
comment:8 by , 8 months ago
Hi Vicky
I have used sed to make the anonymised email addresses @...\d+... into a valid address, to see if this solves the import problem. Could you please try again with the file link that I emailed you?
Peter
comment:9 by , 7 months ago
The mailbox has been successfully created on staging for demonstration purposes
https://discourse.staging.osgeo.org/c/geoserver-users/12
TODO to finish up the demo:
- create corresponding group
- add suitable description
- create the admins
Note: the demo does not include sending/receiving mails that feature is blocked on staging.
follow-up: 11 comment:10 by , 6 months ago
In trying out staging I had some questions - https://discourse.staging.osgeo.org/t/about-the-geoserver-users-category/14078
I seem to be unable to edit the category description?
comment:11 by , 6 months ago
Replying to jive:
In trying out staging I had some questions - https://discourse.staging.osgeo.org/t/about-the-geoserver-users-category/14078
I seem to be unable to edit the category description?
We were discussing that yesterday. It's a limitation of discourse that only discourse admins can edit category information metadata, though group admins can edit most metadata about a group.
Not sure if there is a workaround for that.
comment:12 by , 5 months ago
The limitation is fine, we should document that the group description needs to be provided when requesting a new category.
Please use the text from the staging environment above (cut and paste for the links to work).
Welcome to the discussion category for GeoServer user community. This category is administered by the GeoServer PSC. * As an Open Source Geospatial Foundation project we require all participants follow our [code of conduct](https://github.com/geoserver/geoserver/blob/main/CODE_OF_CONDUCT.md). * This is a public channel and not to be used for the discussion of security vulnerability and exploits. See [security policy](https://github.com/geoserver/geoserver/security) for instructions on responsible disclosure. * Being an open community requires public communication, allowing questions to be shared. Chances are someone else has the same question and we all benefit from public discussion. * The availability of a public forum means that sending email to an individual developer is discouraged. If you are unable to communicate in public [commercial support providers](https://geoserver.org/support/) are listed on our website. * This forum is for use of the GeoServer software, development chatter is [directed elsewhere](https://geoserver.org/devel/). To use this category as a mailing list first register and then use geoserver-users@discourse.osgeo.org for communication.
The GeoServer proposal is now approved.
Let me know and I can use the link above to provide a new mbox dump.
comment:13 by , 5 months ago
@jive,
For what you have done above, I think that is generally done as a post on the category and then pinned so it doesn't fall off the top. That is something you can do yourself.
Kind of like what we have for pgrouting-dev category - https://discourse.osgeo.org/c/pgrouting/pgrouting-dev/17
The about post stays at the top.
The metadata I was talking about is like the logo and a very brief description, kind of like you see on QGIS-fr
https://discourse.osgeo.org/c/qgis/qgis-fr-user/5
and that shows in the qgis category screen - https://discourse.osgeo.org/c/qgis/11
follow-up: 15 comment:14 by , 3 months ago
Okay it is time to go, we have some instructions outlined, and once the discourse channel is ready we will take screen shots, update our website communication information, and share with our users.
Q: is geoserver
/geoserver-users
subcategory necessary to can the structure be geosever
/ users
? and be clear enough?
Based on discussion I assume if the police changes to introduce a "projects" or "osgeo projects" top-level category that things will shuffle around a bit on us.
comment:15 by , 3 months ago
Replying to jive:
Okay it is time to go, we have some instructions outlined, and once the discourse channel is ready we will take screen shots, update our website communication information, and share with our users.
geoserver
/user
Notes:
- You can let me know if
geosever
/geoserver-users
is required for disambiguation
I think you can do geoserver/users if you want. I think the main impact I would be worried about is in the email subject line.
Right now we have that set to: %{optional_cat}%{topic_title}
I had thought that optional_cat was just the category name, but now I realize it includes the parent category so maybe just having it called users is better cause then the subject line would be shorter.
https://meta.discourse.org/t/superseded-customize-the-subject-format-for-standard-emails/20801
- I assume if the police changes to introduce a "projects" or "osgeo projects" top-level category that things will shuffle around a bit on us.
I haven't looked at the email subject line of a category 3 levels deep - we do have https://discourse.osgeo.org/c/local-chapters/gfoss-it/33 which is 3 levels deep. As stated this is something discourse discourages and I had changed the config to allow it cause I figured we'd need it for what you are describing. Again I'd be concerned of the subject line.
Speaking of which all the mirrors I set up for sac, I put SAC on them just to replicate how the mailman mailing lists works, but probably a good test to take that off. So I'll do that now and see how the subject lines look.
follow-up: 17 comment:16 by , 3 months ago
Thanks for experimenting with category names, advise what to do and I will update the geoserver and geonetwork planning accordingly.
- option top-level minimal:
geoserver
/user
- option top-level redundant:
geoserver
/geoserver-user
- option nested minimal:
project
/geoserver
/user
- option nested redundant:
osgeo project
/geoserver
/geoserver-user
I made an export of the mbox yesterday ...resulting in a 1.3G text file. What is the best way to provide that to you?
Notes:
- I tend to prefer singular (
project
,user
) rather than plural (projects
,users
) - We may of already lost any hope of consistency, can categories be renamed?
comment:17 by , 3 months ago
Replying to jive:
Thanks for experimenting with category names, advise what to do and I will update the geoserver and geonetwork planning accordingly.
- option top-level minimal:
geoserver
/user
- option top-level redundant:
geoserver
/geoserver-user
- option nested minimal:
project
/geoserver
/user
- option nested redundant:
osgeo project
/geoserver
/geoserver-user
I'd start with the short version as you stated. I'm still feeling iffy about burying projects under project as that affects more than geoserver and QGIS main listings are planning to be coming on board in the next month or 2.
I feel like for now we should go with:
- option top-level minimal:
geoserver
/user
and move one level down after further discussions with all project stakeholders, which right now is just QGIS, geoserver, and pgrouting
One thing I did notice as another consequence of going 3 levels, is the description on the home page only shows for the uppest parent level even if subparents have descriptions. I haven't investigated if there is a fix for that.
Like if you look at SAC sub category, it has no description showing on homepage though it does have a description. So we might loose that for pgRouting and QGIS, their descriptions if they are buried one level down in a project folder.
I also haven't investigated other ways for organizing the home page like custom plugins etc. I think there might be many more we haven't considered. Right now we are using the categories page like a lazy homepage. It would be great if someone could take on that work to figure out our options, cause I'd rather focus on system administration than UI look and feel which I know I suck at.
I made an export of the mbox yesterday ...resulting in a 1.3G text file. What is the best way to provide that to you?
If it's the same format as you gave @cvvergara last time, it should work. I think she was able to make sense of the part obfuscated email addresses. I'll leave it up to her to respond.
Notes:
- I tend to prefer singular (
project
,user
) rather than plural (projects
,users
)- We may of already lost any hope of consistency, can categories be renamed?
Yes as mentioned categories can be renamed easily cause internally discourse just works on category id numbers. In fact I think Vicky has to in order to load the mailing lists, put them on the root and then rename them and move them down the category hierarchy where they belong cause the mbox importer uses the mbox to arrive at the name.
However if you start with one name and decide to change later, I think we might need to put redirects in (which I recall there being a page for that within discourse) so that if people were navigating via the UI, they will be redirected to the new location.
comment:18 by , 3 months ago
As I write I forwarded the mail that contains the way I got the information for the test on staging.discourse
The load happens at the root, then the category needs to be moved to the final destination.
Steps: Create the geoserver category
- logo (send me a link for the logo to use, I can use the logo on staging)
- description of the category
- security: everyone can: see (no Reply no create)
Create geoserver-user category
- Jody and Andrea as owners
The "user" category will be created on the root when importing
- Once its created to avoid mishaps
- security:
everyone
can: See (no Reply no create)
- security:
Move "user" under geoserver category
- logo (send me a link for the logo to use if different from the above)
- description of the category
- security:
everyone
can: See (no Reply no create) or can See, Reply (no Create)geoserver-user
can See, Reply, Create
About the 3 levels, its new! I could not do it the last time I tried. From other tickets, grouping categories like "Local Chapters", "Initiatives", "GeoInquietos" have been created.
We are still the "few" in discourse.
So, lets focus on this ticket about GeoServer here, and open a new ticket about reorganizing the categories. That one can take care of the consequences of the changes (like redirection).
PS. Personally I like to see pgRouting on the root
comment:19 by , 3 months ago
branding is here: https://github.com/geoserver/geoserver/wiki/GeoServer-Branding
by , 3 months ago
Attachment: | 3-levels.png added |
---|
comment:20 by , 3 months ago
Replying to cvvergara:
The "user" category will be created on the root when importing
- Once its created to avoid mishaps
- security:
everyone
can: See (no Reply no create)
For clarification, assume you mean:
The "geoserver-user" category will be created on the root when importing
Move "user" under geoserver category
Move "geoserver-user" category under geoserver category and rename to "user"
About the 3 levels, its new! I could not do it the last time I tried.
We have some already You'll see that SAC and OSGeoLive categories are now 3 levels down:
PS. Personally I like to see pgRouting on the root
Yah I do too, and if I can ever convince PostGIS and GEOS PSC to move, I'd want to see them on the root too.
I also think that if QGIS moves down one level to projects it's going to crowd out the other projects, cause QGIS has developer, users, various language specific categories, various extension specific categories. Heck they might even want 3 levels from QGIS (which would make it 4 levels, too hard to manage).
Which means to be fair to the other projects, if QGIS is at the top other projects need to be allowed to be at the top too.
But I think having 3 levels down for our internal committees like SAC, Board, Marketing makes great sense.
comment:21 by , 3 months ago
Can we move discussion of top-level category out of this ticket https://discourse.osgeo.org/t/how-would-you-like-to-add-new-categories-for-projects-or-committees/28330/18 (or just make a decision as SAC admins please).
For this activity:
geoserver
andgeoserver/user
(and we can move it later if needed)
- mbox export is Herę
- peterafrigis if you can share your sed clean up recipe it would be useful for this and other migrations to discourse
- branding is here: https://github.com/geoserver/geoserver/wiki/GeoServer-Branding
Is anything else needed from me? I would like to make this transition in the next two weeks (before I go on vacation). This would give the community most of August to transition before shutting down email alongside the GeoServer 2.24.0 release (when more users will notice something is happening).
comment:22 by , 3 months ago
cat geoserver-users.mbox | sed 's/@\.\.\.[0-9]\{1,6\}\.\.\./@anonymised.com/g' > geoserver-users-anon.mbox
This simply changes the anonymised email addresses @...\d+...
( e.g. andrea.aime@...84...
) into a valid address, and it solved the import problem.
comment:23 by , 3 months ago
Tasks done
- Created GeoServer Users group
- Owner: jive
- slug: geoserver-users
- Created GeoServer category
- slug: geoserver
- everyone: Can See (topics)
- Imported mailbox to: User category
- Category name: User
- Category parent: GeoServer
- Slug: geoserver-user
- Mail: geoserver-user@…
- everyone: Can see (topics)
- geoserver-users: Can see, reply, create (topics)
comment:24 by , 3 months ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
comment:25 by , 3 months ago
Resolution: | fixed |
---|---|
Status: | closed → reopened |
We are running into some questions but I see that this ticket has been closed:
- The instructions you provided indicate geoserver-user@… (not geoserver-user@…). I could not remember what was decided on here?
- The name, email notification, and URL slug seem inconsistent. Sometimes "geoserver-user" and sometimes "geoserver-users"?
- Have reports of Peter being unable to reply to a topic in geoserver-user category? Is this not a public forum...
- For the category we have the logo, and then the text “Communication line with GeoServer.” Where does this come from? And can we please edit it ...
The goal is to make nice instructions here https://discourse.osgeo.org/t/welcome-to-geoserver-user-forum/31009
comment:26 by , 3 months ago
The thread https://discourse.osgeo.org/t/testing-new-topic-by-mail/49596/22 goes over some of the testing of email interaction.
- mailing list mode is not appropriate as you then get all interactions
The use of "GeoServer User" is preferred resulting in the following changes:
Group title: GeoServer Users" → GeoServer User"
Group slug: geoserver-users → geoserver-user
Change description to:
Java web application to share and edit geospatial data. Publish data from any major spatial data source using open standards.
comment:27 by , 6 weeks ago
I have closed the geoserver-user list:
prevents anyone from subscribing, but the archives will remain public, and list usable by current members.
And am exporting the mbox for final update
@jive
Please send me the mailbox, I will set it up in staging for you to evaluate.