Downloading videos from PeerTube fails

[lnicola]

$ yt-dlp https://video.osgeo.org/w/qC7HNRU783f6aqpysigF87
[generic] Extracting URL: https://video.osgeo.org/w/qC7HNRU783f6aqpysigF87
[generic] qC7HNRU783f6aqpysigF87: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] qC7HNRU783f6aqpysigF87: Extracting information
[PeerTube] Extracting URL: peertube:video.osgeo.org:qC7HNRU783f6aqpysigF87
[PeerTube] qC7HNRU783f6aqpysigF87: Downloading video JSON
[info] qC7HNRU783f6aqpysigF87: Downloading 1 format(s): 720p
[download] Destination: pr-2521-simplify-dijkstra-code [qC7HNRU783f6aqpysigF87].mp4
[download]  14.7% of  604.68MiB at  472.32KiB/s ETA 18:37[download] Got error: Downloaded 93453056 bytes, expected 634057781 bytes. Retrying (1/10)...
[download] Destination: pr-2521-simplify-dijkstra-code [qC7HNRU783f6aqpysigF87].mp4
[download]  18.8% of  604.68MiB at  471.13KiB/s ETA 17:47[download] Got error: Downloaded 119154544 bytes, expected 634057781 bytes. Retrying (2/10)...
[download] Destination: pr-2521-simplify-dijkstra-code [qC7HNRU783f6aqpysigF87].mp4
[download]  51.1% of  604.68MiB at  581.76KiB/s ETA 08:40^C
ERROR: Interrupted by user

(it does say 581 KB/s, but it's practically 0)

[robe]

When I'm watching online the speed seems fine, no abruptness.

When I use yt-dlp, I'm getting 5MiB/s to 7MiB/s and it took 01:59 total to dnwoload the 604.68Mib file.

I'm wondering if maybe it's an issue with our network provider. I'm testing from Comcast Boston, so connectivity might be better there.

Do you have similar issue when downloading files from download.osgeo.org? Like try ​https://download.osgeo.org/proj/proj-data-1.16.zip

This file downloads at 14.19MiB/s and completes for me in takes 00:51 s to donwload the 694.72MiB file.

Download I would expect to be faster cause it has bonded nics so should have twice the thruput of osgeo9

[lnicola]

Do you have similar issue when downloading files from download.osgeo.org?

In the first 2-3 minutes it goes slowly to about 2.3 MB/s, then hovers around that. Afterwards, it gets less stable and drops to < 200 KB/s, then hovers around 1.5 MB/s. Average is 1431 KB/s for the entire file.

With yt-dlp it's a little different, I get a minute or two at 2-3 MB/s, then it drops sharply to ~0.

Is there any throttling involved?

[lnicola]

From a Nuremberg Hetzner VM: proj-data-1.16.zip averages out at 16.7 MB/s.

yt-dlp starts at a bit over 7 MB/s and doesn't fail, but still slows down after a while. The good part is that the download completes (100% of 668.73MiB in 02:14, roughly 5 MB/s).

So part of this must be a peering issue.

[robe]

Okay so still sounds like our network backbone, probably worth reporting to OSUOSL as we've had similar issues in the past. So when you are getting horrible speed which network provider are you on?

[robe]

Replying to lnicola:

Do you have similar issue when downloading files from download.osgeo.org?

In the first 2-3 minutes it goes slowly to about 2.3 MB/s, then hovers around that. Afterwards, it gets less stable and drops to < 200 KB/s, then hovers around 1.5 MB/s. Average is 1431 KB/s for the entire file.

With yt-dlp it's a little different, I get a minute or two at 2-3 MB/s, then it drops sharply to ~0.

I think nginx has some default throttling, not sure about our backbone and some nginx configs on domains I might have tweaked, but the peertube one I jsut took whatever they suggested is the best nginx config as I recal. But I think the nginx one is IP based, so we should be experiencing the same behavior unless you've been doing nothing but downloading stuff or you are on some sort of proxy that's been hogging resources.

Is there any throttling involved?

[lnicola]

I'm on my home network. There's no proxy involved on my side and I don't think nginx throttles by default (it does buffer the responses, but that can't be the case here).

[robe]

Okay so it's a network issue between network providers most likely. We'll have to report to osuosl. I forget what network checks they have us do to confirm where the issue is.