Opened 4 years ago

Closed 4 years ago

#2128 closed defect (fixed)

Login required for svn checkout of GRASS GIS

Reported by: wenzeslaus Owned by: martin
Priority: blocker Milestone: Sysadmin Contract 2018-I
Component: Systems Admin Keywords: login, subversion, svn, credentials, grass
Cc:

Description

What happens:

Anonymous svn checkout of GRASS GIS requires credentials.

Expected behavior:

No credentials required for reading.

How to reproduce:

Run Ubuntu in Docker:

docker run -it --rm ubuntu

Then install Subversion and update certificates:

apt update -y
apt install -y subversion ca-certificates
svn checkout https://svn.osgeo.org/grass/grass/trunk

Alternatively, run Fedora in Docker:

docker run -it --rm fedora bash
yum update -y
yum install -y subversion
svn checkout https://svn.osgeo.org/grass/grass/trunk

In both cases I get:

Authentication realm: <https://svn.osgeo.org:443> OSGeo Login
Password for 'root':

I observed the same behavior on two computers (Ubuntu, no Docker).

In all cases authentication request comes after a random number of downloaded files, not at the beginning or after a specific file.

Change History (18)

comment:1 by strk, 4 years ago

Priority: normalblocker

comment:2 by strk, 4 years ago

Milestone: Sysadmin Contract 2018

comment:3 by martin, 4 years ago

Owner: changed from sac@… to martin
Status: newassigned

comment:4 by strk, 4 years ago

Milestone: Sysadmin Contract 2018Sysadmin Contract 2018-I

Milestone renamed

comment:5 by martin, 4 years ago

To a certain degree this error occurs randomly (!)
After transferring a varying number of files, Apache kicks in claiming "client denied by server configuration".

Actually, Apache-DAV-SVN is authenticating access to every single file in the SVN-repo individually, causing dozends of auth requests per second just for a single checkout. Now, since checkout over my slow 2 Mbit/s line at home works flawlessly, I'd suspect an overflow of authentication requests being the root cause.

The Apache "mod_ldap" connector allows caching LDAP requests - in fact it does by default, but may require some tuning. I'll add some custom setting and will re-check later today or tomorrow if that makes a difference. If it does, then I'd also suspect the overall SVN performance to improve.

comment:6 by martin, 4 years ago

My very first test on a well-connected machine worked quite nicely after the change:

tracsvn:~# cat /etc/apache2/mods-available/ldap.conf
LDAPSharedCacheSize 1000000
LDAPCacheEntries 4096
LDAPOpCacheEntries 4096
[...]

Please test and respond.

comment:7 by wenzeslaus, 4 years ago

Sorry, still the same here (99.4 Mbps download as measured by Google) using the tests above.

comment:8 by robe, 4 years ago

FWIW I was having the same issue about the time this ticket was reported with GDAL.

For GDAL SVN I don't have commit access to I pull annonymously. It was prompting me for password to do an svn update or svn checkout. It seems fine now.

comment:9 by strk, 4 years ago

Can mod_evasive have a role here ? Too many consecutive requests ? I remember SVN checkouts made lots of PROPFIND to the very same URL. But any such block should not prompt for a password: is a password prompt what the original report complains about ?

in reply to:  9 comment:10 by wenzeslaus, 4 years ago

Replying to strk:

is a password prompt what the original report complains about ?

Yes, this is what I see (using the Docker image above):

$ svn checkout https://svn.osgeo.org/grass/grass/trunk
A    trunk/general
...
A    trunk/general/g.rename/testsuite/test_overwrite.py
A    trunk/general/g.rename/main.c
A    trunk/general/g.pnmcomp/Makefile
A    trunk/general/g.dirseps/g.dirseps.html
A    trunk/general/g.proj/g.proj.html
A    trunk/general/g.proj/create.c
Authentication realm: <https://svn.osgeo.org:443> OSGeo Login
Password for 'root': 

comment:11 by robe, 4 years ago

okay I just experienced the same issue doing a checkout of gdal trunk.

I think I thought it was fixed, because I can do an svn update fine after even though I never successfully authenticated doing svn checkout.

When I try to type in my user name and password (which doesn't have write rights to gdal repo if keeps on prompting for password, and finally ends with too many tries).

So guess it's still an issue.

comment:12 by EliL, 4 years ago

I also just experienced this doing a checkout of gdal. Checkout might make more requests than update.

comment:13 by martin, 4 years ago

I ran a couple of consecutive tests and, indeed, mod-evasive makes a difference.

Its config says: The client will be added to the blocking list for one minute if it exceeds 50 requests per second.
Now that we're running a significantly updated version of SVN - upgraded from 1.5.1 to 1.8.10 - the server probably responds a lot faster than it did before and mod-evasive kicks in.

Surprisingly a couple of hosts have been whitelisted in mod-evasive - looks like the issue might be a known buddy.

I'm now leaving mod-evasive disabled so everybody can test and - maybe - later start re-enabling and tuning the parameters. On the other hand: Increasing thresholds will almost make mod-evasive useless anyway.

comment:14 by wenzeslaus, 4 years ago

I can report a successful checkout using the Docker test above (at 46.5 Mbps).

comment:15 by martin, 4 years ago

Thanks for testing.
I've now re-enabled mod-evasive but I've also increased the "hits per host and second" threshold by four times (value of 200 now). Would you please test and report again ?

comment:16 by martin, 4 years ago

Further analysis revealed, that 200 hits per host and second won't suffice, threshold now increased to 450.
Hey, this means that our SVN service is capable of serving almost 450 hits per second ! ;-)

comment:17 by wenzeslaus, 4 years ago

Works for me (Docker test, at 46.5 Mbps).

comment:18 by robe, 4 years ago

Resolution: fixed
Status: assignedclosed

I was able to do a full checkout of GDAL trunk without issues now.

I'm going to consider this fixed and closed.

Great job Martin!

Note: See TracTickets for help on using tickets.