gitea disable tarball download

[robe]

Been discussion this on matrix, that we should disable tarball download from gitea.

When gitea generates tarballs for repos, it generally does this real time unless there is an archive. This takes resources and I think sometimes causes a DDOS when some crawler like those recent AI crawlers attack.

It also takes up disk space and snapshot space if these aren't purged quick enough.

We discussed there is not much need for people to download tarballs from gitea, since all the relevant ones can be found on download.osgeo.org, gitlab or github.

I'll leave this open for a bit in case anyone wants to comment otherwise.

It's an easy change to undo.

[lnicola]

Not just space, but also a lot of CPU usage IIRC. I just tried to download one for PostGIS and it took about 30 seconds, and I didn't even get the ZIP for some reason (GET request finished with a 15 MB payload, but no download).

I filed ​https://git.osgeo.org/gitea/sac/ansible-deployment/pulls/72 for when we're ready to do this.

[strk]

Isn't that a job for robots.txt ?

[lnicola]

No, very few robots are nice enough to respect that. Looks like the download is a POST, but bots were still calling it, which is also not very nice.

[lnicola]

I mentioned this on IRC, but to download an archive you need to get a CSRF token from a <script> block in the page, do a POST request with it, then come back later (not sure when) and do a GET to actually download the file.

I don't think anyone is using this on CI.

[robe]

I applied @lnicola pull request and ran

on both dev and production

make check-gitea
make deploy-gitea

On production it made changes to osgeo7-nginx as well it seems

I put in a diff clause in the check and deploy so I could see what changes were being made I assume the change was just to move stuff into a snippet file

@@ -20,6 +20,7 @@
 # Global maximum creation limit of repository per user
 MAX_CREATION_LIMIT = 40
 PREFERRED_LICENSES = AGPL-3.0,GPL-3.0,LGPL-3.0
+DISABLE_DOWNLOAD_SOURCE_ARCHIVES = true
 
 [server]
 DOMAIN       = git.osgeo.org

changed: [osgeo7_tracsvn]

TASK [gitea : unset secrets] ****************************************************************************************************************************************************************************************************
ok: [osgeo7_tracsvn]

TASK [gitea : Install snippets/gitea.conf file on osgeo7_nginx] *****************************************************************************************************************************************************************
--- before
+++ after: /home/robe/.ansible/tmp/ansible-local-139987zz3fe5fx/tmpn09hgd8r/nginx-snippet-gitea.j2
@@ -0,0 +1,15 @@
+# WARNING: This file is managed by ansible,
+# manual changes may be reverted, see
+# https://git.osgeo.org/gitea/sac/ansible-deployment
+
+# First attempt to serve request as file, then
+# as directory, then fall back to displaying a 404.
+#try_files $uri $uri/ =404;
+if (-f $document_root/under_maintenance_gitea.html) {
+  return 503;
+}
+client_max_body_size 0;
+include /etc/nginx/proxy_protocol_params;
+proxy_pass http://tracsvn.lxd:3001/;
+proxy_redirect off;
+

changed: [osgeo7_tracsvn -> osgeo7_nginx(osgeo7.osgeo.osuosl.org)]

TASK [gitea : Install _git.osgeo.org.ansible file on osgeo7_nginx] **************************************************************************************************************************************************************
--- before: /etc/nginx/sites-available/_git.osgeo.org.ansible
+++ after: /home/robe/.ansible/tmp/ansible-local-139987zz3fe5fx/tmprymlc1zo/nginx-site-git.j2
@@ -11,16 +11,7 @@
 error_log /var/log/nginx/git.osgeo.org.error_log info;
 
 location /gitea/ {
-  # First attempt to serve request as file, then
-  # as directory, then fall back to displaying a 404.
-  #try_files $uri $uri/ =404;
-  if (-f $document_root/under_maintenance_gitea.html) {
-    return 503;
-  }
-  client_max_body_size 0;
-  include /etc/nginx/proxy_protocol_params;
-  proxy_pass http://tracsvn.lxd:3001/;
-  proxy_redirect off;
+  include snippets/gitea.conf;
 }
 
 location = /robots.txt {

changed: [osgeo7_tracsvn -> osgeo7_nginx(osgeo7.osgeo.osuosl.org)]

TASK [gitea : Install _gitea.osgeo.org.ansible file on osgeo7_nginx] ************************************************************************************************************************************************************
--- before: /etc/nginx/sites-available/_gitea.osgeo.org.ansible
+++ after: /home/robe/.ansible/tmp/ansible-local-139987zz3fe5fx/tmpgvxaeh_v/nginx-site-gitea.j2
@@ -8,21 +8,10 @@
 
 access_log /var/log/nginx/gitea.osgeo.org.access_log pcombined;
 error_log /var/log/nginx/gitea.osgeo.org.error_log info;
-rewrite_log on;
-#rewrite ^(.*)$ /gitea/$1? break;
 
 
 location /gitea/ {
-  # First attempt to serve request as file, then
-  # as directory, then fall back to displaying a 404.
-  #try_files $uri $uri/ =404;
-  if (-f $document_root/under_maintenance_gitea.html) {
-    return 503;
-  }
-  client_max_body_size 0;
-  include /etc/nginx/proxy_protocol_params;
-  proxy_pass http://tracsvn.lxd:3001/;
-  proxy_redirect off;
+  include snippets/gitea.conf;
 }