Release distribution contains generated files with "dev" in version

[Mike Taves]

A few surprises with the latest geos-3.10 release from ​http://download.osgeo.org/geos/geos-3.10.0.tar.bz2

• It contains generated files that shouldn't be distributed, e.g. cmake_install.cmake, include/geos/version.h, tools/geos-config, tools/geos.pc, and empty bin, and a fun script bisect_script.sh (full diff later)
• The generated files in this release still have "dev" as part of the version

• From the release build, the first line from the command geosop shows "GEOS 3.10.0dev" and not "GEOS 3.10.0"

I suspect the reason is that make dist was done in a git repository with these generated files.

The first "fix" is to expand CPACK_SOURCE_IGNORE_FILES (also with set(CPACK_VERBATIM_VARIABLES TRUE), e.g. ​done with PROJ. However, that doesn't guarantee that some other random file won't show up in the source distribution (e.g. bisect_script.sh).

The safest way to run make dist is to do something like the following:

sdist_dir=$(mktemp -d)
cd $sdist_dir
git clone --depth 1 --branch 3.10 https://git.osgeo.org/gitea/geos/geos.git
cd geos
# maybe edit Version.txt
mkdir build
cd geos
cmake -G"Unix Makefiles" ..
make dist

Another idea is to not use CMake for make dist. Use something like ​git-archive.

---

Here is the full diff of what I see in the release vs the source generated from the above steps in the 3.10 branch:

Only in geos-3.10.0: bin
Only in geos-3.10.0: bisect_script.sh
Only in geos-3.10.0: cmake_install.cmake
Files geos-3.10.0/include/geos/io/WKBConstants.h and geos-3.10.1dev/include/geos/io/WKBConstants.h differ
Files geos-3.10.0/include/geos/triangulate/quadedge/Vertex.h and geos-3.10.1dev/include/geos/triangulate/quadedge/Vertex.h differ
Files geos-3.10.0/include/geos/util.h and geos-3.10.1dev/include/geos/util.h differ
Only in geos-3.10.0/include/geos: version.h
Only in geos-3.10.0: ltmain.sh
Files geos-3.10.0/NEWS and geos-3.10.1dev/NEWS differ
Files geos-3.10.0/README.md and geos-3.10.1dev/README.md differ
Files geos-3.10.0/src/algorithm/Distance.cpp and geos-3.10.1dev/src/algorithm/Distance.cpp differ
Files geos-3.10.0/src/algorithm/LineIntersector.cpp and geos-3.10.1dev/src/algorithm/LineIntersector.cpp differ
Files geos-3.10.0/src/algorithm/MinimumBoundingCircle.cpp and geos-3.10.1dev/src/algorithm/MinimumBoundingCircle.cpp differ
Files geos-3.10.0/src/geom/LineSegment.cpp and geos-3.10.1dev/src/geom/LineSegment.cpp differ
Files geos-3.10.0/src/geom/util/Densifier.cpp and geos-3.10.1dev/src/geom/util/Densifier.cpp differ
Files geos-3.10.0/src/geom/util/GeometryTransformer.cpp and geos-3.10.1dev/src/geom/util/GeometryTransformer.cpp differ
Files geos-3.10.0/src/io/GeoJSONReader.cpp and geos-3.10.1dev/src/io/GeoJSONReader.cpp differ
Files geos-3.10.0/src/triangulate/VoronoiDiagramBuilder.cpp and geos-3.10.1dev/src/triangulate/VoronoiDiagramBuilder.cpp differ
Files geos-3.10.0/tests/unit/capi/GEOSDistanceTest.cpp and geos-3.10.1dev/tests/unit/capi/GEOSDistanceTest.cpp differ
Files geos-3.10.0/tests/unit/capi/GEOSFrechetDistanceTest.cpp and geos-3.10.1dev/tests/unit/capi/GEOSFrechetDistanceTest.cpp differ
Files geos-3.10.0/tests/unit/capi/GEOSGeom_setPrecisionTest.cpp and geos-3.10.1dev/tests/unit/capi/GEOSGeom_setPrecisionTest.cpp differ
Files geos-3.10.0/tests/unit/capi/GEOSHausdorffDistanceTest.cpp and geos-3.10.1dev/tests/unit/capi/GEOSHausdorffDistanceTest.cpp differ
Files geos-3.10.0/tests/unit/capi/GEOSSTRtreeTest.cpp and geos-3.10.1dev/tests/unit/capi/GEOSSTRtreeTest.cpp differ
Files geos-3.10.0/tests/unit/io/GeoJSONReaderTest.cpp and geos-3.10.1dev/tests/unit/io/GeoJSONReaderTest.cpp differ
Only in geos-3.10.0/tools: geos-config
Only in geos-3.10.0/tools: geos.pc
Files geos-3.10.0/Version.txt and geos-3.10.1dev/Version.txt differ

[pramsey]

I guess 3.10.1 will be out shortly.

[pramsey]

Could you stuff this through your process quickly? ​http://download.osgeo.org/geos/geos-3.10.1.tar.bz2

[Bas Couwenberg]

The release tarball now includes the web subdirectory, that really only belongs in the VCS.

The examples & tools subdirectories are missing from the tarball now.

[Mike Taves]

Sure, the "from_trac" subdir is from the download.osgeo link above, and "from_3.10_branch" is what I get from the 3.10 branch using the steps above, with an edit to Version.txt

$ diff -qr /tmp/from_trac/ /tmp/from_3.10_branch/
Files /tmp/from_trac/geos-3.10.1/CMakeLists.txt and /tmp/from_3.10_branch/geos-3.10.1/CMakeLists.txt differ
Only in /tmp/from_3.10_branch/geos-3.10.1: doc
Only in /tmp/from_trac/geos-3.10.1: doxygen
Only in /tmp/from_3.10_branch/geos-3.10.1/examples: capi_prepared.cpp
Only in /tmp/from_3.10_branch/geos-3.10.1/examples: capi_read.cpp
Only in /tmp/from_3.10_branch/geos-3.10.1/examples: capi_read_ts.cpp
Files /tmp/from_trac/geos-3.10.1/include/geos/noding/snap/SnappingNoder.h and /tmp/from_3.10_branch/geos-3.10.1/include/geos/noding/snap/SnappingNoder.h differ
Files /tmp/from_trac/geos-3.10.1/NEWS and /tmp/from_3.10_branch/geos-3.10.1/NEWS differ
Files /tmp/from_trac/geos-3.10.1/README.md and /tmp/from_3.10_branch/geos-3.10.1/README.md differ
Files /tmp/from_trac/geos-3.10.1/src/noding/snap/SnappingNoder.cpp and /tmp/from_3.10_branch/geos-3.10.1/src/noding/snap/SnappingNoder.cpp differ
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSCoveredByTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSCoversTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSCrossesTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSDisjointTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSGeomToHEX_bufTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSGeomToWKB_bufTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSGeomTypeIdTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSGetExteriorRingTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSGetGeometryNTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSGetInteriorRingNTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSOverlapsTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSRelatePatternTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSRelateTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSSymDifferencePrecTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSSymDifferenceTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSTouchesTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSUnaryUnionPrecTest.cpp
Only in /tmp/from_trac/geos-3.10.1/tests/unit/capi: GEOSUnionCascadedTest.cpp
Only in /tmp/from_trac/geos-3.10.1: web

[Mike Taves]

correction to current "safest way" is:

sdist_dir=$(mktemp -d)
cd $sdist_dir
git clone --depth 1 --branch 3.10 https://git.osgeo.org/gitea/geos/geos.git
cd geos
# edit Version.txt
mkdir build
cd build
cmake -G"Unix Makefiles" ..
make dist
# copy or upload .tar.bz2 file somewhere
cd /tmp
rm -rf $sdist_dir

I still think a better long-term solution would be to make a POSIX script that uses git-archive. This should be targeted for a future release, and I'll have a think about it as I would also want it for PROJ too.

[pramsey]

Ugh, this is much improved, sorry.

​http://download.osgeo.org/geos/test/geos-3.10.1.tar.bz2

[Mike Taves]

Now identical to the updated 3.10 branch, but the new CPACK_SOURCE_IGNORE_FILES setting is too aggressive and appears to exclude template files, and the source does not get past the configuration:

$ cmake ..
...
CMake Error: File /tmp/from_trac/geos-3.10.1/include/geos/version.h.in does not exist.
CMake Error at include/CMakeLists.txt:30 (configure_file):
  configure_file Problem configuring file
...
CMake Error: File /tmp/from_trac/geos-3.10.1/tools/geos-config.in does not exist.
CMake Error at tools/CMakeLists.txt:25 (configure_file):
  configure_file Problem configuring file
Call Stack (most recent call first):
  tools/CMakeLists.txt:65 (configure_install_geos_config)

do the new CPACK_SOURCE_IGNORE_FILES patterns need $ to prevent partial matches or something?

[Mike Taves]

try this:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index e5994bd..2b9161b 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -417,8 +417,9 @@ if(NOT _is_multi_config_generator)
     "/php/"
     "/.*build-.*/"
     "cmake_install\\\\.cmake"
-    "/include/geos/version\\\\.h"
-    "/tools/geos-config"
+    "/include/geos/version\\\\.h\$"
+    "/tools/geos-config\$"
+    "/tools/geos\\\\.pc\$"
     "/bin/"
     ${PROJECT_BINARY_DIR}
     )

[pramsey]

Again, with your suggestion. ​​http://download.osgeo.org/geos/test/geos-3.10.1.tar.bz2

It's building here ATM. (update: built and passed check)

[Mike Taves]

That looks like it should do!

Also, be aware that download links should use https://download.osgeo.org, as some modern browsers will refuse to download from unsecure "http" URLs.

[pramsey]

Wonderful. Thanks so much for your patience and help Mike.