gitea git commit hooks failing to trac upgrade of trac.db.api.DatabaseManager

[robe]

We've been having this issue on trac since recent upgrades that gitea commit hooks to trac are failing with

Error: Unable to check for upgrade of trac.db.api.DatabaseManager: TimeoutError: Unable to get database connection within 0 seconds.

making the calls directly via trac-admin works fine.

[lnicola]

What if you run trac-admin as git?

[strk]

I've just tried and succeeded (sudo -u git trac-admin /var/www/trac/env/postgis/ changeset added git bdecd88fe3f506b2b1f64de3f6cf25dc8f102f66 resulting in ​https://trac.osgeo.org/postgis/ticket/5782#comment:41

[robe]

Yah bizarred direct works. I'm really tempted to rebuild tracsvn from scratch -- maybe we can do that next year and just call it tracgit.

tracsvn has been around for 20 years and gone thru continuous upgrades - e.g. my first real experience with it was when it was a ganetti vm and I migrated it to lxd and then upgraded it from stretch. It probably was upgraded even early before me.

[strk]

For the record it took me 3 reloads to see this ticket, first attempts all gave me the "cannot get db connection within 0 seconds" error.

I wanted to say that manually running trac-admin usually works for me, I've continued using this workaround and it seemed to always work, while the git log seems to always fail. It would probably be wise to file a ticket upstream to trac itself to see if they have any hint about it.

[robe]

reported upstream - ​https://trac.edgewall.org/ticket/13793

[robe]

Upstream walked me thru adding some logging in the python code around debugging db and also to turn on db logging.

The logs now show this as the culprit

psycopg2.OperationalError: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: fe_sendauth: no password supplied

Why it doesn't trigger that when tracadmin is called directly using the same accounts gitea is using is a bit puzzling.

[robe]

I confirmed that is the issue cause when I set to trust for local connection in /etc/postgresql/16/main/pg_hba.conf it worked fine. I checked and the git account does have a .pgpass entry for trac account for all databases and it can connect to all databases. But I guess for some reason maybe psyqcopg2 can't access the file or it's running under another account when it does tracadmin call.

[robe]

For now I just put in the trac password in the /var/www/trac/env/postgis/trac.ini file and that seems to work okay. I also turned postgres logging back to what it was since the postgresql.log had ballooned to over 90GB.

[robe]

Okay issue resolved. Troubleshooted with lnicola and strk by adding env logging to the postgis git hook to see what environment variables it was using.

added to the top of /home/git/gitea-repositories/postgis/postgis.git/hooks/post-receive.d/osgeo

env > /home/git/env_from_git

Turns out that gitea overrides the HOME variable to:

HOME=/home/git/gitea/data/home

So it was expecting the .pgpass file to be there instead of /home/git/.pgpass. Copying the .pgpass there fixed the issue.

[strk]

Do you have any idea when did Gitea start changing HOME ? I ask because before the system upgrade things were working and I don't think we upgraded Gitea ?

[robe]

Replying to strk:

Do you have any idea when did Gitea start changing HOME ? I ask because before the system upgrade things were working and I don't think we upgraded Gitea ?

Yah totally baffling. We did not upgrade gitea and gitea service script /etc/init.d/gitea I don't think we changed either. So I still think somehow the upgrade changed how it determines home for service scripts.

I would expect home to be what this returns

`

sudo -u git bash -c 'echo $HOME'

`

But that returns the expected /home/git

not the /home/git/gitea/data/home that gitea seems to be using.

Perhaps it's some conditional code that gitea is hitting as a result of OS upgrade that wasn't being exercised before.