Login required for svn checkout of GRASS GIS

[wenzeslaus]

What happens:

Anonymous svn checkout of GRASS GIS requires credentials.

Expected behavior:

No credentials required for reading.

How to reproduce:

Run Ubuntu in Docker:

docker run -it --rm ubuntu

Then install Subversion and update certificates:

apt update -y
apt install -y subversion ca-certificates
svn checkout https://svn.osgeo.org/grass/grass/trunk

Alternatively, run Fedora in Docker:

docker run -it --rm fedora bash

yum update -y
yum install -y subversion
svn checkout https://svn.osgeo.org/grass/grass/trunk

In both cases I get:

Authentication realm: <https://svn.osgeo.org:443> OSGeo Login
Password for 'root':

I observed the same behavior on two computers (Ubuntu, no Docker).

In all cases authentication request comes after a random number of downloaded files, not at the beginning or after a specific file.

[strk]

Milestone renamed

[martin]

To a certain degree this error occurs randomly (!)
After transferring a varying number of files, Apache kicks in claiming "client denied by server configuration".

Actually, Apache-DAV-SVN is authenticating access to every single file in the SVN-repo individually, causing dozends of auth requests per second just for a single checkout. Now, since checkout over my slow 2 Mbit/s line at home works flawlessly, I'd suspect an overflow of authentication requests being the root cause.

The Apache "mod_ldap" connector allows caching LDAP requests - in fact it does by default, but may require some tuning. I'll add some custom setting and will re-check later today or tomorrow if that makes a difference. If it does, then I'd also suspect the overall SVN performance to improve.

[martin]

My very first test on a well-connected machine worked quite nicely after the change:

tracsvn:~# cat /etc/apache2/mods-available/ldap.conf
LDAPSharedCacheSize 1000000
LDAPCacheEntries 4096
LDAPOpCacheEntries 4096
[...]

Please test and respond.

[wenzeslaus]

Sorry, still the same here (99.4 Mbps download as measured by Google) using the tests above.

[robe]

FWIW I was having the same issue about the time this ticket was reported with GDAL.

For GDAL SVN I don't have commit access to I pull annonymously. It was prompting me for password to do an svn update or svn checkout. It seems fine now.

[strk]

Can mod_evasive have a role here ? Too many consecutive requests ? I remember SVN checkouts made lots of PROPFIND to the very same URL. But any such block should not prompt for a password: is a password prompt what the original report complains about ?

[wenzeslaus]

Replying to strk:

is a password prompt what the original report complains about ?

Yes, this is what I see (using the Docker image above):

$ svn checkout https://svn.osgeo.org/grass/grass/trunk
A    trunk/general
...
A    trunk/general/g.rename/testsuite/test_overwrite.py
A    trunk/general/g.rename/main.c
A    trunk/general/g.pnmcomp/Makefile
A    trunk/general/g.dirseps/g.dirseps.html
A    trunk/general/g.proj/g.proj.html
A    trunk/general/g.proj/create.c
Authentication realm: <https://svn.osgeo.org:443> OSGeo Login
Password for 'root': 

[robe]

okay I just experienced the same issue doing a checkout of gdal trunk.

I think I thought it was fixed, because I can do an svn update fine after even though I never successfully authenticated doing svn checkout.

When I try to type in my user name and password (which doesn't have write rights to gdal repo if keeps on prompting for password, and finally ends with too many tries).

So guess it's still an issue.

[EliL]

I also just experienced this doing a checkout of gdal. Checkout might make more requests than update.

[martin]

I ran a couple of consecutive tests and, indeed, mod-evasive makes a difference.

Its config says: The client will be added to the blocking list for one minute if it exceeds 50 requests per second.
Now that we're running a significantly updated version of SVN - upgraded from 1.5.1 to 1.8.10 - the server probably responds a lot faster than it did before and mod-evasive kicks in.

Surprisingly a couple of hosts have been whitelisted in mod-evasive - looks like the issue might be a known buddy.

I'm now leaving mod-evasive disabled so everybody can test and - maybe - later start re-enabling and tuning the parameters. On the other hand: Increasing thresholds will almost make mod-evasive useless anyway.

[wenzeslaus]

I can report a successful checkout using the Docker test above (at 46.5 Mbps).

[martin]

Thanks for testing.
I've now re-enabled mod-evasive but I've also increased the "hits per host and second" threshold by four times (value of 200 now). Would you please test and report again ?

[martin]

Further analysis revealed, that 200 hits per host and second won't suffice, threshold now increased to 450.
Hey, this means that our SVN service is capable of serving almost 450 hits per second ! ;-)

[wenzeslaus]

Works for me (Docker test, at 46.5 Mbps).

[robe]

I was able to do a full checkout of GDAL trunk without issues now.

I'm going to consider this fixed and closed.

Great job Martin!