Using Git to maintain GDAL workflow
This article describes how to maintain GDAL development workflow with Git against GDAL Subversion repository.
Git repository setup using svn2git
svn2git is a tool written in Ruby which makes it easy to mirror SVN repository in Git, especially if one wants to mirror complete SVN repository based on the standard layout with: turnk, tags and branches. svn2git also makes it easy to update the Git mirror pulling latest changes from SVN.
Here is example of initial import of GDAL repository from SVN to Git:
$ svn2git http://svn.osgeo.org/gdal/ --exclude sandbox --exclude spike
It will import content of trunk, tags and branches.
Next, one may want to frequently update the Git repository pulling latest changes from SVN:
$ svn2git --rebase
The local repository can be published on remote server like GitHub this way:
$ git remote add origin firstname.lastname@example.org:GITHUB_USERNAME/REPO_NAME.git $ git push origin master
Then, it is ready to keep the Git mirror in remote location (e.g. GitHub) updated:
$ svn2git --rebase $ git push origin master
Git repository setup using pygit-svn-mirror
Creating mirror of Subversion repository of PROJ.4 project at GitHub? involves the following commands:
$ mkdir /path/to/proj4/mirror $ cd /path/to/proj4/mirror $ git-svn-mirror.py init --from=https://svn.osgeo.org/metacrs/proj/ --email@example.com:<USRNAME>/proj.4.git
Then, update (synchronise) the mirror from its workbench directory:
cd /path/to/proj4/mirror git-svn-mirror.py update
or from any folder but with workbench location pointed explicitly:
git-svn-mirror.py update -w /path/to/proj4/mirror
Git repository setup using git-svn
First, initialize local Git repository for copy of trunk form Subversion:
$ git svn init --prefix=svn --trunk https://svn.osgeo.org/gdal/trunk/
The option --trunk explicitly specifies intention of this operation. We use
--prefix so remote svn branches are named eg. 'svn/trunk' rather than 'trunk', since otherwise you get unhelpful name clashes when doing cross-branch operations.
It is also possible to copy complete repository, including trunk, branches and tags modules. Learn about --stdlayout option dedicated for this purpose. However, note that this initial fetch is a time consuming process. (fetching of SVN trunk takes nearly one hour).
Next, fetch complete copy of GDAL trunk:
$ git svn --authors-file=/path/to/gdal-git-authors.txt fetch
The file gdal-git-authors.txt (attached) is used by Git to translate names of SVN committers to Git names which uses format "Full name <e-mail>". Specifying the --authors-file option is optional, but recommended. You can also do this once via
git config --add svn.authorsfile /path/to/gdal-git-authors.txt, then you don't have to specify it when you're running other
git svn commands as described below.
It may be a good idea to compress Git repository in order to save some space. For example, SVN trunk occupies nearly 140 MB of disk space. Git can compress it to about 100 MB. Git provides a dedicated command for this purpose git-gc:
$ git gc
You can bring your local Git repository up to date against GDAL trunk any time.
$ git svn --authors-file=/path/to/gdal-git-authors.txt rebase
The rebase command is equivalent to svn update. Learn more about rebase in git-svn manual.
==> Caution ! : I (E. Rouault) have had problems with the workflow detailed below, especially when there's a conflict between one of your local change in GIT and a change in SVN. The solution seems to create a GIT branch where you do your local changes, and keep your GIT master in sync with SVN. Then you rebase your master on to your local branch, resolve the conflict in it, commit it, and then (forward) merge the branch into master. At that point, you can git svn dcommit. Workflow inspired by http://stackoverflow.com/questions/629048/git-svn-dcommit-error-restart-the-commit
After this step, you maintain your own local repository of GDAL trunk. You can proceed with your own development, make changes and commit them to your local repository.
$ git commit -m "my big feature"
If you are GDAL committer, you can push your changes to GDAL Subversion repository directly from your local Git repository of GDAL trunk:
$ git svn dcommit --dry-run $ git svn dcommit
You can also push the local copy to your own Git remote repository to share your development experiments, then others will be able to clone it. For example, you can push it to GitHub:
$ git push --all firstname.lastname@example.org:mloskot/gdal.git
Once local clone of a git-svn enabled repository is ready for the cycle of svn update and git push, the routine of updating mirror includes the following two commands only:
$ git svn --authors-file=/path/to/gdal-git-authors.txt rebase $ git push email@example.com:mloskot/gdal.git
=> Note from user etiennesky: I had to use the following command after the initial fetch:
git reset --hard remotes/trunk
There is an up-to-date Git repository with mirror of GDAL Subversion trunk, branches and tags available at https://github.com/OSGeo/gdal maintained by Even Rouault using the pygit-svn-mirror tool (see section above).
Note, if you are GDAL committer, clone of this repository will not allow you to issue git svn dcommit command to push your changes back to GDAL trunk. If you expect this functionality, you need to maintain Git repository of GDAL trunk on your own according to the steps described above. Quoting one of best practices listed here:
For the sake of simplicity and interoperating with SVN, it is recommended that all git-svn users clone, fetch and dcommit directly from the SVN server (the remote SVN repository that is), and avoid all git-clone/pull/merge/push operations between git repositories and branches which are either retrieved via git svn clone and which are also used to push back changesets into the remote SVN repository.
This process may be time consuming, so an alternative is to use rsync to copy existing GDAL tree enabled with git-svn from another committer.
- Importing from Subversion on GitHub?
- git-svn manual
- Can different git-svn clones of the same svn repository expect to be able to share changes then git svn dcommit? on StackOverflow?
- Using Git for SVN Repositories Workflow
- Howto use Git and svn together
- Using git to maintain Subversion branches
- git-svn in 30 minutes
gdal-git-authors.txt (4.5 KB) - added by 14 months ago.
Git authors file to use with git-svn option --authors-file. Note, if this option is specified and git svn encounters an SVN committer name that does not exist in the authors-file, git svn will abort operation. The user will then have to add the appropriate entry. Re-running the previous git svn command after the authors-file is modified should continue operation.
Download all attachments as: .zip