Opened 6 years ago

Closed 5 years ago

#7080 closed enhancement (wontfix)

Migrate build system to CMake

Reported by: miurahr Owned by: warmerdam
Priority: normal Milestone: closed_because_of_github_migration
Component: ConfigBuild Version: svn-trunk
Severity: major Keywords: cmake
Cc:

Description (last modified by miurahr)

There were initiatives to migrate build system to CMake in 2012-2013, CMAKE4GDAL project. I have now been developing a build configuration on CMake that continues CMake4Gdal project https://github.com/miurahr/gdal/commits/compile_with_cmake

I do it because it seems to be a good chance to challenge again. There are a couple of reasons.

  • CMake support gcc, clang, mingw and VC++
  • CMake support GNU Make(Linux), Ninja(VS/Windows), nmake(VC++), XCode(mac)

Other merits we will get after successfully migrating to CMake on all platforms

  • Incremental compile and test that improve productivity
  • Out-of-source compile that help Appvoyer / Travis CI tests speedy, developers can test it in multiple options.
  • Single build configuration for all platforms.

Demerit

  • There are not enough documentation for CMake.
  • We are not fluent for CMake now.

So migrating to CMake should be considered carefully. I'd like to propose a step-by-step migration strategy.

  1. Develop CMake configuration on topic branch or github project.
  1. Merge CMake build configuration co-existence with GNU Make and VC configurations. and add CI test on Travis / Appvoyer for cmake. Now it is a state of a technical preview. Still team will release binaries building with legacy build system. At this time, all contributor should aware that he/she should update both cmake and legacy one.
  1. Drop legacy build system when every core developers satisfy CMake/IDE experiences.

Change History (11)

comment:1 by miurahr, 6 years ago

Description: modified (diff)

comment:2 by bishop, 6 years ago

I would like to note that the CMake build system for GDAL and many other dependency libraries already exists and work fine on Windows/Linux/Mac OS.

  1. The main repository: https://github.com/nextgis-borsch/lib_gdal (look on status table for drivers in the Readme)
  2. The NextGIS borsch repository: https://github.com/nextgis-borsch/borsch
  3. My speech on foss4g 2016 about this build system: http://ftp5.gwdg.de/pub/misc/openstreetmap/FOSS4G-2016/foss4g-2016-1231-borsch_modern_build_system_for_c_c_gis_projects-hd.webm

I think it will be nice to enhance this system rather than create new one.

comment:3 by miurahr, 6 years ago

Bishop, Thank you for sharing the borsch project. IMHO, I did't make new project but I continues your approach and project in CMake4GDAL project in 2011-2012.(actually import your repository as a start point)

A target of my effort is an integration of cmake scripts and current GDAL source tree.

As my motivation, I'd like to contribute to current GDAL project using modern IDEs (CLion/Visual Studio 2017) which support CMake as a target system component. These modern IDEs use cmake script not only as a build system, also as integrated dependency configuration for IDEs.

In this integration scope , a separate project to prepare build system is not enough, and integration on current GDAL source is necessary.

Last edited 6 years ago by miurahr (previous) (diff)

comment:4 by bishop, 6 years ago

I don't understand your motivation. There is the modern CMake for GDAL which solve the dependency headache and works on all main platforms include mobile. I support this repository up to date. An this is really current GDAL project. It is not ideal but working solution. You took the ancient and outdated CMake sources and plan to duplicate already done work instead enhance this one.

By the way all integration with current GDAL sources available. There is a special python script to rearrange sources for CMake project (see https://github.com/nextgis-borsch/borsch/blob/master/opt/tools.py organize).

As for IDE - Visual Studio 2017 and Windows in not the one target/IDE platform for GDAL.

comment:5 by miurahr, 6 years ago

I don't want to rearrange source tree. Borsch seems a project to rearrange the tree with cmake.

My motivation is to use CLion IDE for refactoring a complex driver (GML), so I look for cmake migration project on current source tree then I take your old project as a start point.

As a side effect, migrating to CMake makes VS17 also usable as IDE for GDAL.

It is better to take a step-by-step way for migration in order to minimize efforts by ecosystem players such as debian package maintainer, install package builder, document maintainers, etc. ie. (1) add cmake build on current tree and build system, (2) take cmake as a primary build system (3) drop old build system, (4) rearrange source tree (if want).

I'd like to contribute step-1 of above.

comment:6 by bishop, 6 years ago

You are not quite right (see https://github.com/nextgis-borsch/borsch/blob/master/README.md). The Borsch is not the project about rearrange GDAL sources tree. The changes added to GDAL sources tree make it more suitable to CMake ideology, modern GDAL 2.x and it complex (raster/vector/network) drivers, dependencies needs and provide more flexible approach for configuring the build.

Also I don't see how you plan solve dependencies problem, which already solved in Borsch.

How do you plan to get iconv, json-c, geos, proj4, expat and other libraries especially on MacOS, Android, iOS builds (Windows too for specific compiler - see discussion around http://osgeo-org.1560.x6.nabble.com/gdal-dev-Linking-against-gdal201-dll-with-different-MSVC-versions-td5310281.html)?

In Borsch dependency can be added with the single line of code in CMake file. Everything is build with the one compiler. There is a solution for prebuild vendor libraries - see https://github.com/nextgis-borsch/lib_ecw and https://github.com/nextgis-borsch/lib_mrsid.

Do you tested your solution with another modern IDSs like Qt Creator, Android studio, Xcode? What about static builds for all platforms?

As I noted, Borsch already works with this modern IDEs and on different operation systems. The noted Linux (Debian) OS - it builds too (there are targets to create deb and rpm) and exists ppa which shows Ubuntu builds too.

All steps you described in your plan were already done and I think the Borsch is the next logical step. I think your proposal is a step aside (back?) and more suitable for your needs (refactoring a complex driver (GML)) but it will be regression for tasks I noted.

comment:7 by Even Rouault, 6 years ago

Dmitry, I think Borsch and 'plain' cmake approaches solve different needs, and have different constraints, advantages and drawbacks. I'd see Borsch as a kind of super-build system, whereas Hiroshi is looking for something that is directly integrated in the GDAL tree, which enables to directly create patches, commit, etc. I'm pretty sure that we could re-use most of the great work you did to find dependencies with a more traditional cmake approach of putting those rules in a cmake/ directory.

comment:8 by bishop, 6 years ago

First of all need the answers for my questions about dependencies, build on different platforms, support different compiles, IDEs and you may come to some kind of Borsch. You cannot integrate CMake into GDAL tree not solving this problems.

Just rewrite make/automake files to CMake it seems to me no sense and will not provide new quality (maybe vice versa). It will only work for the sake of work.

Not all GDAL drivers are already supported by Borsch (https://github.com/nextgis-borsch/lib_gdal/blob/master/README.md). Let's help me finish complete support of all GDAL functionality and we can start process moving GDAL trunk to this build system. This is the only reason I cannot boost this process now.

If there is strong disagreement with rearranging source tree - this is not the Borsch requirement. It make things with drivers simpler and more logical, but not mandatory. The embedded to source tree third-party sources - also no problem to leave them (it may be added several presets for most popular build scenarious).

What is the complete list of Borsch problems which make it unsuitable for GDAL?

comment:9 by Even Rouault, 6 years ago

Correct me if I'm wrong, but my memories of your Borsch presentation at FOSS4G 2016 is that it requires all GDAL dependencies to be already built with cmake themselves, which would be quite a constraint as it prevents directly using packaged version on Linux if the upstream project hasn't adopted cmake, but I am a bit confused when looking at the repository since I see some scripts like https://github.com/nextgis-borsch/borsch/blob/master/cmake/FindPROJ4.cmake that seem to work with more conventional installations. I also see in https://github.com/nextgis-borsch a lot of repositories with forked versions of many libraries. This might indeed make things easier overall if you build the whole stack from scratch, which can makes sense in some cases, but this is typically something that prevents easy integration with the packages provided by a Linux distribution, or if you want to integrate with OSGeo4W or gisinternals provided dependencies.

Regarding dependencies, probably that Borsch makes thing easier and more relieable by uniformizing how a library is installed / detected. But the end result is that currently you have to maintain forks of all those libraries, which is not sustainable/scalable in the long term and doesn't make cooperation with upstream easy. For example https://github.com/nextgis-borsch/lib_gdal is a fork of GDAL 2.2.1. You have to resync regularly with releases. And the fact that there's no match with the dev version makes it impossible for GDAL developers working on trunk to use cmake as their main build system. I strongly believe that any out-of-tree solution is a huge barrier for adoption and cooperation (this is a general statement, not specific to Borsch)

I don't think a cmake based build system needs to be feature complete to be integrated in GDAL source tree. It can have an experimental status at the beginning and presumably be incrementally improved until it supports everything.

I think this discussion should probably be moved to the mailing list. There are probably more people there that can give their feedback.

comment:10 by Even Rouault, 6 years ago

comment:11 by Even Rouault, 5 years ago

Milestone: closed_because_of_github_migration
Resolution: wontfix
Status: newclosed

This ticket has been automatically closed because Trac is no longer used for GDAL bug tracking, since the project has migrated to GitHub. If you believe this ticket is still valid, you may file it to https://github.com/OSGeo/gdal/issues if it is not already reported there.

Note: See TracTickets for help on using tickets.