Opened 4 months ago

Last modified 3 days ago

#690 new enhancement

GRASS GIS 8 in OSGeo4W NG

Reported by: sbl Owned by: osgeo4w-dev@…
Priority: major Component: Package
Version: Keywords: grass
Cc: landam, hellik

Description

The GRASS GIS devs are preparing the release of GRASS GIS 8.

With the upcoming new version in mind, I would like to discuss how we can get it into OSGeo4W.

Personally, I could imagine packages similar to QGIS (but not as many):

grass (refering to the current stable (point) release ) grass-rel (nightly build of the release branch) grass-ltr (refering to the legacy release, we do not have a concept of LTR/LTS right now (would become 7.8 once 8.0 is released)) grass-dev (nightly build of the develpment branch)

But here we are open for discussions, and solutions like with GRASS GIS 6 and GRASS GIS 7 are of course thinkable as well!

I am willing to contribute (in fact I tried to create a grass-dev package here: https://github.com/ninsbl/OSGeo4W/tree/grass_dev_workflows), where I tried to build the package with GH workflows. So, I would try to prepare PRs. Unfortunately, I cannot figure out what the entrypoint for packaging is in https://github.com/jef-n/OSGeo4W (I am more or less bound to GH actions), so I would need some help to get started.

Hope we can solve that together somehow...

Change History (8)

comment:1 Changed 4 months ago by jef

Unfortunately there is no entry point.

The original idea was to have some trigger on commit that figures out which packages need rebuilds (changed packages + reverse dependencies) and trigger their package.sh in order.

But I never got to that, as I cannot get QGIS to build (let alone Qt) using github actions because of disk space constraints. So I still have to build packages "outside".

Otherwise entry points would be the individual package.sh, which:

  • declare build dependencies, that are fetched in startlog,
  • fetch the sources
  • build
  • package in a local repository, which are finally uploaded in endlog

e.g. GRASS package.sh

comment:2 Changed 4 months ago by sbl

Thanks for your reply. For starters I tried (in a Github action), to:

  1. Download dependencies from OSGeo4W (so they live in /c/OSGeo4W/,
  2. Setting the current working directory to where the package.sh lives (src/grass/osgeo4w) and run it with bash.

However, although the relative path should be correct, build_helpers was not found, and after hardcoding the path to build_helpers, build_helpers was sourced, but then genini fails:

/d/a/OSGeo4W/OSGeo4W/scripts/genini: can't open x86_64/setup.ini.tmp - No such file or directory
mv: cannot stat 'x86_64/setup.ini.tmp': No such file or directory
/d/a/OSGeo4W/OSGeo4W/scripts/build-helpers: line 206: x86_64/setup.ini.bz2: No such file or directory

The attempt to setup the GH action is here: https://github.com/ninsbl/OSGeo4W/tree/grass_dev_workflows and the results (error messages) here: https://github.com/ninsbl/OSGeo4W/runs/3620990391?check_suite_focus=true

My feeling is that something needs to be run before package.sh (maybe some environment variables need to be set) or that the script needs to be called from a different place than where it lives. But I have no idea where I could find information on that.

If I could get that to work, I could contribute recipies for GRASS packages (PRs that are tested on GH), and probably even more important with regards to OSGeo4W integration, I could try to integrate the OSGeo4W build procedure as the basis for the CI / unittests for GRASS GIS, so new commits are tested with regards to effects on OSGeo4W builds...

So, e.g. any hint on how to call package.sh and pre-requisits to do so successfully, would be a great help and very much appreciated. Thanks.

comment:3 in reply to:  2 Changed 4 months ago by jef

Replying to sbl:

Thanks for your reply. For starters I tried (in a Github action), to:

  1. Download dependencies from OSGeo4W (so they live in /c/OSGeo4W/,

That's not necessary. Each package installs it's own copy of OSGeo4W dependencies in src/$P/osgeo4w/osgeo4W.

  1. Setting the current working directory to where the package.sh lives (src/grass/osgeo4w) and run it with bash.

But src/grass/osgeo4w/package.sh already exists!?

However, although the relative path should be correct, build_helpers was not found, and after hardcoding the path to build_helpers, build_helpers was sourced, but then genini fails:

/d/a/OSGeo4W/OSGeo4W/scripts/genini: can't open x86_64/setup.ini.tmp - No such file or directory
mv: cannot stat 'x86_64/setup.ini.tmp': No such file or directory
/d/a/OSGeo4W/OSGeo4W/scripts/build-helpers: line 206: x86_64/setup.ini.bz2: No such file or directory

Ok, x86_64/release was apparently not created automatically. IIRC this wasn't ever run on github, because QGIS isn't buildable anyway - after hours it would run out of space…

comment:4 Changed 4 months ago by sbl

Thank, @jef for following up. So, I will skip the OSGeo4W download.

In the github action the OSGeo4W repo is pulled to the build server, so the entire repo content is present at /d/a/OSGeo4W/OSGeo4W

Will try with creating x86_64/release in advance. Is it correct to create it in: src/$P/osgeo4w/ and call package.sh from there or where should the working directory be?

comment:5 in reply to:  4 Changed 4 months ago by jef

Replying to sbl:

Will try with creating x86_64/release in advance. Is it correct to create it in: src/$P/osgeo4w/ and call package.sh from there or where should the working directory be?

https://github.com/jef-n/OSGeo4W/commit/bdd21f516094f96fc8f3bcaa0b04bc85b7a62cb0

comment:6 Changed 4 months ago by sbl

Thanks, jef. This seems to have been a missing link for me. I enccountered some other minor issues with running this in a GH action, but most of them I managed to iron out.

Currently, I encounter issues with dumpbin.exe but that is probably just a misconfigured PATH and versions beeing different between GH actions and (dumpbin -exports does not work while dumpbin /EXPORTS does). Testing right now if my changes make it run through...

Will try to cleanup once this works properly and then create a PR.

Thanks again with your support on this...

comment:7 Changed 3 weeks ago by sbl

Note that RC1 for GRASS GIS 8 was released end of 2021:

https://github.com/OSGeo/grass/releases/tag/8.0.0RC1

comment:8 Changed 3 days ago by sbl

A new, RC2 for GRASS GIS 8 has just been released: https://github.com/OSGeo/grass/releases/tag/8.0.0RC2

Note: See TracTickets for help on using tickets.