Opened 15 years ago
Closed 15 years ago
#435 closed task (fixed)
Benchmarking data download server
| Reported by: | Jeff McKenna | Owned by: | |
|---|---|---|---|
| Priority: | normal | Milestone: | |
| Component: | SysAdmin | Keywords: | |
| Cc: | aaime, dmorissette, warmerdam, aboudreault |
Description
As discussed in the last FOSS4G benchmarking meeting, we need a place to host the benchmarking data download. It's currently on a USACE server but connection speed is very slow...and in the long run we think OSGeo should host the data (and benchmarking test package) for the public to download. The data is ~32 GB in size.
Change History (9)
comment:1 by , 15 years ago
comment:2 by , 15 years ago
| Cc: | added |
|---|
Daniel, I wonder if maptools.org could host the benchmarking data? (the data is about 32GB) Can we handle this extra bandwidth demand?
(on IRC FrankW stated that this could not be hosted on telascience servers)
comment:3 by , 15 years ago
Mind the biggest part of the data is uncompressed. The shapefile and the big geotiffs (which are uncompressed) should go down a lot if they are tar.gz-ed (the ecw will not go down one bit instead).
One tgz per package would be a nice to redistribute the lot.
comment:4 by , 15 years ago
| Cc: | added |
|---|
For the raster data I would suggest we only distribute the ecw, and provide scripts to translate that to bigtiff and tiles and add overviews. I can prepare those scripts at some point.
comment:5 by , 15 years ago
Frank, you read my mind. I was actually looking for those scripts to replicate the tiffs on my box hoping to find them somewhere on goliath, but I had no such luck
comment:6 by , 15 years ago
| Cc: | added |
|---|
Jeff, yes, we could host it on maptools.org as long as we don't get hundreds of downloads of it per month. However I would not put in the usual "dl" directory to avoid also wasting space on the master server which is replicated to the live server. I'd place the files directly in a directory on the live server bypassing the normal replication mechanism. Let's look into that with Alan on Monday.
comment:7 by , 15 years ago
Alan, let me know when you can discuss using maptools.org to host the benchmarking package.
All: I generated a TIGER 2008 dataset for GeoServer and MapServer testing, that I will upload to this new server (it contains both as-is county files as well as processed merged files). I'm looking at TIGER 2008 styles now.
comment:8 by , 15 years ago
Alan, the files have transferred to your machine. Please upload the 4 zip files to maptools.org
raster.zip (.4 GB) vector-data-original.zip (.5 GB) vector-data-tiger08-tx-merged.zip (1.1 GB) vector-date-tiger08-tx-counties.zip (1.1 GB)
comment:9 by , 15 years ago
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
Alan has set this up on maptools.org (notes at http://wiki.osgeo.org/wiki/Benchmarking_2009#Download).
marking as fixed.
Can we set this up soon? (I'd like to point ESRI and ERDAS to this) Eventually this will be how the public can access our Web Server benchmarking files.
-jeff