wiki:HowToGit

Context Navigation

Version 99 (modified by wenzeslaus, 3 years ago) ( diff )
Update mutliple commits section to current workflow, master to main, smaller updates

Git How-to

The instructions for the common contributing cases are in CONTRIBUTING file.

The repo is here: https://github.com/OSGeo/grass

Below we discuss more advanced and specific workflows.

For returning contributors

If you are long-term GRASS GIS contributor returning back after some time off, please note that the development workflow changed significantly after migration from Subversion to Git (GitHub).

Important changes:

Direct committing to "main" (former "trunk" and "master") is not allowed.
Hence, you will create a feature branch and open a pull request for a change.
- Rationale: Pull requests are the perfect platform to discuss, improve, and check changes before merging.
- This applies to both contributors and core developers.

Workflow:

Fork the GRASS GIS repository, create feature branch(es) with the changes, and suggest your changes using pull requests.

Workflow for GRASS GIS Git repositories

The same procedure is recommended also for GRASS Addons repository.

Preparation: cloning the repo

First fork the GRASS GIS repo in the GitHub UI to your_GH_account. This is the same as what GitHub documentation suggests. See: Fork a repo and Syncing a fork in GitHub help.

Note: Add SSH key, see GitHub documentation.

Note: If you get the error message: 'ssh: connect to host github.com port 22: Connection refused' (e.g. because you are behind a firewall that blocks traffic on port 22), you can try to replace git@github.com: with ssh://git@ssh.github.com:443/, when you clone or remote add repos.

# "origin" points to your fork repo - IMPORTANT
git clone git@github.com:your_GH_account/grass.git

# add "upstream" remote
cd grass/
git remote add upstream git@github.com:OSGeo/grass.git

git remote -v
# you should see something like
origin	  git@github.com:your_GH_account/grass.git (fetch)
origin	  git@github.com:your_GH_account/grass.git (push)
upstream  git@github.com:OSGeo/grass.git (fetch)
upstream  git@github.com:OSGeo/grass.git (push)

Similarly for GRASS Addons repository:

git clone git@github.com:your_GH_account/grass-addons.git
cd grass-addons/
git remote add upstream git@github.com:OSGeo/grass-addons.git

Working with Git

Note: this requires the "remotes" to be set as shown above.

# updating from git server: fetch all branches from all remotes
git fetch --all

### a) updating the main branch
# merge updates into local main branch
git merge upstream/main

# at this point we have reached:
# (HEAD -> main, upstream/main)

### b) updating releasebranch_7_8
# switch to branch
# only once
git checkout -b releasebranch_7_8 origin/releasebranch_7_8
# next time, git checkout releasebranch_7_8

# merge updates into local branch
git merge upstream/releasebranch_7_8

# at this point we have reached:
# (HEAD -> releasebranch_7_8, upstream/releasebranch_7_8)

# update own remote
# - clears "Your branch is ahead of 'origin/main' by XX commits.", and
# - avoids "This branch is XX commits behind OSGeo:main." on GitHub web interface
git push origin main

# list existing branches
git branch -a

##### Implementation of own changes
# create new local branch (pick a new name for feature_branch_name)
git checkout -b feature_branch_name

# <make local source code changes>
vim ...

# list local changes
git status
git add file1.c file2.py ...
git commit -m 'my change with reasonable explanation...'

# push feature branch to origin, i.e. your fork of the OSGeo/grass repo
git push origin feature_branch_name
# create pull request in GitHub Web interface (the link is then shown in the terminal)

# during PR review phase, make more local changes if needed
git add .
git commit -m 'my second change'
git push origin feature_branch_name
# ..... will be added to existing pull request

Note: for different pull requests, simply create different feature branches.

Keep your local source code up to date

[from https://github.com/OSGeo/gdal/blob/master/CONTRIBUTING.md#working-with-a-feature-branch]

You may need to re-synchronize against the main branch if you need some bugfix or new capability that has been added since you created your branch

# assuming that "upstream" points to OSGeo/grass
git fetch upstream
git rebase upstream/main

# if rebase fails with "error: cannot rebase: You have unstaged changes...", then move your uncommitted local changes to "stash"
git stash
# now you can rebase
git rebase upstream/main
# apply your local changes on top
git stash pop

Continue do your changes and commit/push them (ideally to a feature branch, see above).

Updating your PR from main

Updating your PR from the main branch is often referred to as rebasing a PR. It comes into play in case there are changes in the main branch you need to incorporate into your (feature or fix) branch before the PR can be merged, you need to rebase your branch to upstream:

git fetch upstream
git rebase upstream/main
git status

Now the git status will tell you that your local branch and the branch in origin (which is your fork) have diverged:

Your branch and 'origin/fix-window-size' have diverged,
and have 7 and 5 different commits each, respectively.

If you try to push to your fork (origin), you will get rejected:

 ! [rejected]  fix-window-size -> fix-window-size (non-fast-forward)
error: failed to push some refs to 'https://...'
hint: Updates were rejected because the tip of your current branch is behind

Besides the provided information, both of the messages above will also suggest you to use git pull to update. That's not what you want to do (at least not with the default settings of git pull which does merge).

As git status says, there are different commits in each branch (the local one and the one in your fork). That's because git rebase changed hashes and dates of the existing commits in your local branch (because it reapplies the changes as new commits on top of the up-to-date upstream). The commits on the branch in your fork don't actually contain anything you don't have, so you can safely discard them. You can confirm this by looking at git log locally and looking at the branch in your fork on GitHub.

To push into the branch in the fork, you will need to overwrite what is already there, i.e., replace the branch in the fork by content in your local branch. This is done using force push:

git push --force origin fix-window-size

If you have access to one of the OSGeo repositories (namely OSGeo/grass in this case), before force pushing, you need be sure that origin points to your fork and not the OSGeo repo. You can check that using git remote -v.

Switching between branches

For an elegant way of multi-branches in separate directories with only a single repo clone, see

https://lists.osgeo.org/pipermail/grass-dev/2019-May/092653.html

Testing pull requests from other contributors in main

GitHub provides command line instructions under each pull request ("Pulls" tab). Please check there.

Applying a diff file locally

Downloading a pull request as a diff file (example):

cd path/to/grass_git/

# use the PR number and simply add .diff to the URL
wget https://github.com/OSGeo/grass/pull/174.diff

Patching local repo with git (continuing with example diff):

# first the stats about the patch (see what would be changed)
git apply --stat 174.diff

# dry run to detect errors (should show no output, i.e. no errors):
git apply --check 174.diff

# apply patch locally, continue committing as usual
git apply 174.diff
git status
# ... now comment on PR in GitHub, etc.

# FYI - here how to undo a local patch:
git apply --reverse grass_code_changes.diff

Checking out pull requests locally

A convenient way to checkout a pull request for testing purposes is using the following command pattern:

# assuming that "upstream" points to OSGeo/grass

git fetch upstream pull/<id>/head:<branch>

The <id> is the ID number and <branch> is the branch name of the pull request.

For example, using this information on a PR as displayed on GitHub:

Fix this error #123
Someone wants to merge 3 commits into OSGeo:main from Someone:fix-this-error

the command would look like:

git fetch upstream pull/123/head:fix-this-error

# then proceed to checkout for testing
git checkout fix-this-error

# after testing, you may delete locally
git checkout main
git branch -D fix-this-error

As a variation of the above, git alias can be set in git configuration file to make this even easier. Add following lines to either $HOME/.gitconfig (all git repositories) or .git/config (in source repository):

[alias]
	pr = "!f() { git fetch -fu ${2:-upstream} refs/pull/$1/head:pr/$1 && git checkout pr/$1; }; f"
	pr-clean = "!git checkout main ; git for-each-ref refs/heads/pr/* --format=\"%(refname)\" | while read ref ; do branch=${ref#refs/heads/} ; git branch -D $branch ; done"

How to use the aliases:

# Fetch and checkout a PR:
git pr <id> # (e.g. git pr 832)

git branch -a
  main
* pr/832
  pr/837
  releasebranch_7_6
  releasebranch_7_8

# Delete all PRs:
git pr-clean

Note: To be on the safe side, only use this for testing pull request!

Fixing bugs in a release branch

To directly fix bugs (ideally via feature branch), do

(example: https://github.com/OSGeo/grass/tree/releasebranch_7_8)

# push to release_branch, we assume it to be checked out

cd releasebranch_78/
# be sure to locally have all updates from server
git fetch --all
git branch --merged 

# create feature branch
git checkout -b r78_fix_xxx

# ... do changes...

git status
git add ...
git commit -m 'useful commit msg...'

# push to feature branch
git push upstream r78_fix_xxx

Now create the PR in GitHub UI using the link shown by git push ... in the terminal.

IMPORTANT: switch there to release_branch_X_Y**

After PR positive review, merge PR in GitHub.

Local cleanup after successful PR merge:

# switch to release branch
git checkout releasebranch_7_8

# be sure to locally have all updates
git fetch --all
git branch --merged 

# delete local feature branch as no longer needed
git branch -D r78_fix_xxx 
git fetch --all --prune
git branch -a

Backporting to release branches

Preparation

If you checked out the release branch into a separate directory, be sure to have "upstream" enabled as a remote:

git remote -v
# if upstream is missing, execute
git remote add upstream git@github.com:OSGeo/grass.git

Backporting of a single commit from main to release branch

Assumptions:

Your own fork is defined as "origin".
The OSGeo repo is defined as "upstream".
Using releasebranch_7_8 branch in the examples as the branch to backport to.

First, before you do the git cherry-pick, update the local repo and fork to state of upstream with the following four steps.

Get latest changes to your machine:

git fetch upstream --prune

Go to the branch:

git checkout releasebranch_7_8

Update the local branch with latest changes from upstream:

git rebase upstream/releasebranch_7_8

Update your the branch in your fork (optional, just to keep things organized):

git push origin releasebranch_7_8

Second, get commit hash from GitHub or git log in the main branch.

Third, cherry-pick the change from main into the branch:

git cherry-pick <hash>

Forth, verify:

git status
git log --max-count=5
git show

Fifth, push backport to the branch in upstream:

git push upstream releasebranch_7_8

This last steps will fail if somebody else did backport after you did the rebase step above. If that's the case, just update your local branch again with the same git fetch and git rebase commands as before.

Made a mess?

If you made a mess in the upstream repo (the OSGeo repo) or you think you did, consult on grass-dev mailing list.

If you have something you don't like in your local branch or in the branch in your fork, you can trash any changes you made and replace your branch with whatever is in the upstream repo.

Make sure you don't have any local changes or changes on the branch in your fork because this will wipe them out.

Make sure your remotes are set right (see above, origin to your fork and upstream to OSGeo repo):

git remote -v

Make sure you are on the right branch (here using releasebranch_7_8):

git checkout releasebranch_7_8

Bring the local branch to the state of the upstream branch trashing all local changes:

git reset --hard upstream/releasebranch_7_8

Update the local repo from the upstream:

git fetch --all --prune
git rebase upstream/releasebranch_7_8

If you want to keep the branch in your fork up to date, you may need to also overwrite anything which is on that branch there. Be sure you are force pushing only to origin and that origin is your fork. Force push the state of the local branch into your fork:

git push --force origin releasebranch_7_8

To confirm that all went well, see whether the commits on your local branch, the corresponding branch in your fork, and the branch in the upstream (OSGeo) repo are the same.

Code review: generate a single diff for a PR with multiple commits

To speed up reviewing of a PR with multiple commits by reading a single diff file, you may diff the branch against the branch origin point. For example, if you PR 28 is on local branch pr_28, branched from the main branch, you can do:

git checkout pr_28

# note: with the three dots, it will only show diffs from changes on your side:
git diff main...

For more hints, see e.g. this Stackoverflow discussion.

Merging of Pull Requests

Rationale: We should try to have clean history and good commit messages. This helps really a lot when you try to understand why something is implemented the way it is.

When a Pull Requests (PR) has multiple commits, the merge commit is more or less mandatory because if you don't have it, you can't use git revert.

PR with single commit

Proposed: when a PR only has a single commit, the "merge commit" doesn't offer anything and it can be avoided by rebasing the feature branch:

Workflow: GitHub > button "Merge pull request" > "Rebase and merge"

Next, you may locally delete the feature branch.

PR with multiple commits

All commits for a PR are squashed into one commit when merging to the base branch (that is usually the main branch). If the individual commit messages are not usable as is for the commit message of this new commit, modify the PR description so that it is usable as a commit message when maintainers merge the PR.

You can have multiple commits in a PR or squash them depending on what you see as most appropriate, but usually multiple commits help certain aspects of the review.

Citing co-authors in a Git commit message

For co-authors, it is suggested using Co-authored-by at the end of the commit message with this syntax:

...

Co-authored-by: name <name@example.com>

An empty line (i.e., two new lines) is necessary and sufficient (GitHub documentation is little confusing there, but the generally that's what's accepted).