Opened 3 weeks ago

Closed 2 weeks ago

#5779 closed defect (fixed)

create_upgrade.pl fails for 3.5.0-beta1

Reported by: sebastic Owned by: strk
Priority: medium Milestone: PostGIS 3.5.0
Component: build Version: master
Keywords: Cc:

Description

The Debian package for 3.5.0-beta1 failed to build on some architectures due to parallel build issue: 

---- Making all in utils
make[3]: Entering directory '/build/postgis/utils'
sed -e '1,/^__END__/d' postgis_restore.pl.in \
        | sort -u \
        > postgis_restore_data.hardcoded
/usr/bin/make -C ../raster/rt_pg rtpostgis_upgrade.sql
make[4]: Entering directory '/build/postgis/raster/rt_pg'
/usr/bin/make -C ../raster/rt_pg uninstall_rtpostgis.sql
make[4]: Entering directory '/build/postgis/raster/rt_pg'
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I../../postgis/ rtpostgis.sql.in > rtpostgis.sql.tmp
grep -v '^#' rtpostgis.sql.tmp | \
/usr/bin/perl -lpe "s'MODULE_PATHNAME'\$libdir/postgis_raster-3'g;s'@extschema@\.''g" > rtpostgis.sql
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I../../postgis/ rtpostgis_upgrade_cleanup.sql.in > rtpostgis_upgrade_cleanup.sql.tmp
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I../../postgis/ rtpostgis.sql.in > rtpostgis.sql.tmp
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I../../postgis/ rtpostgis_drop.sql.in > rtpostgis_drop.sql.tmp
rm -f rtpostgis.sql.tmp
/usr/bin/perl ../../utils/create_uninstall.pl rtpostgis.sql 170 > uninstall_rtpostgis.sql
grep -v '^#' rtpostgis_upgrade_cleanup.sql.tmp | \
/usr/bin/perl -lpe "s'MODULE_PATHNAME'\$libdir/postgis_raster-3'g;s'@extschema@\.''g" > rtpostgis_upgrade_cleanup.sql
grep -v '^#' rtpostgis.sql.tmp | \
/usr/bin/perl -lpe "s'MODULE_PATHNAME'\$libdir/postgis_raster-3'g;s'@extschema@\.''g" > rtpostgis.sql
make[4]: Leaving directory '/build/postgis/raster/rt_pg'
grep -v '^#' rtpostgis_drop.sql.tmp | \
/usr/bin/perl -lpe "s'MODULE_PATHNAME'\$libdir/postgis_raster-3'g;s'@extschema@\.''g" > rtpostgis_drop.sql
rm -f rtpostgis_upgrade_cleanup.sql.tmp
grep: rtpostgis.sql.tmp: No such file or directory
rm -f rtpostgis.sql.tmp
rm -f rtpostgis_drop.sql.tmp
/usr/bin/perl ../../utils/create_upgrade.pl rtpostgis.sql > rtpostgis_upgrade.sql.in
Unable to locate target new version number in rtpostgis.sql
make[4]: *** [Makefile:156: rtpostgis_upgrade.sql.in] Error 255
make[4]: *** Deleting file 'rtpostgis_upgrade.sql.in'
make[4]: Leaving directory '/build/postgis/raster/rt_pg'
make[3]: *** [Makefile:71: ../raster/rt_pg/rtpostgis_upgrade.sql] Error 2
make[3]: Leaving directory '/build/postgis/utils'
make[2]: *** [GNUmakefile:28: all] Error 1
make[2]: Leaving directory '/build/postgis'
make[1]: *** [debian/rules:129: override_dh_auto_build] Error 2
make[1]: Leaving directory '/build/postgis'

The build failed on ​amd64 CI (parallel=2), ​s390x buildd (parallel=2). It did not locally at first with parallel=11 but did the second time, but again not with parallel=1. This suggests a race condition between the Makefile dependencies.

Chaining the commands to create the SQL file seems to help, but that might also just be the race condition: 

--- postgis.orig/raster/rt_pg/Makefile.in
+++ postgis/raster/rt_pg/Makefile.in
@@ -129,9 +129,9 @@ endif
 #       in the extension/ folder
 #
 %.sql: %.sql.in
-       $(SQLPP) -I@top_builddir@/postgis/ $< > $@.tmp
+       $(SQLPP) -I@top_builddir@/postgis/ $< > $@.tmp && \
        grep -v '^#' $@.tmp | \
-       $(PERL) -lpe "s'MODULE_PATHNAME'\$(MODULEPATH)'g;s'@extschema@\.''g" > $@
+       $(PERL) -lpe "s'MODULE_PATHNAME'\$(MODULEPATH)'g;s'@extschema@\.''g" > $@ && \
        rm -f $@.tmp
 
 ../../liblwgeom/.libs/liblwgeom.a:

Change History (11)

comment:1 by strk, 3 weeks ago

Can you still reproduce reverting [5aba84713102a23d8755d1df3f56c277e1dff6b1/git] ?

I saw utils/Makefile has targets to build thing under raster/ which I think was meant to be used when raster is disabled. Those utils/ rules should do nothing if everything under raster/ is already built, and the commit I'm referencing above is building raster/ *after* utils/

comment:2 by robe, 3 weeks ago

As noted by Bas on postgis-dev mailing list

Reverting 5aba84713102a23d8755d1df3f56c277e1dff6b1 seems to resolve the issue, the Salsa CI and buildd builds that failed (more than once) before succeeded now.

Kind Regards,

Bas

comment:3 by sebastic, 3 weeks ago

strk wrote on postgis-dev:

We've upgraded it, Regina updated the ticket with this information and I've further pushed a temptative fix so if you can test that'd be useful ( commit [4b786d9992479041f4bc0cf559cc10e3feca4e2c/git] )

That commit cannot be found on ​https://git.osgeo.org/gitea/postgis/postgis.git (yet).

Last edited 3 weeks ago by strk (previous) (diff)

comment:4 by strk, 3 weeks ago

Now it is, please check (Gitea failed to contact trac due to database connection limit being reached so the ticket wasn't updated)

comment:5 by sebastic, 3 weeks ago

With the changes from 4b786d9992479041f4bc0cf559cc10e3feca4e2c the build fails again: 

---- Making all in postgis
make[4]: Entering directory '/build/postgis-3.5.0~beta1+dfsg/extensions/postgis'
mkdir -p sql
/usr/bin/make -C ../../raster/rt_pg sql_objs
make[5]: Entering directory '/build/postgis-3.5.0~beta1+dfsg/raster/rt_pg'
/usr/bin/make -C ../../raster/rt_pg sql_objs
make[5]: Entering directory '/build/postgis-3.5.0~beta1+dfsg/raster/rt_pg'
/usr/bin/make -C ../../raster/rt_pg sql_objs
make[5]: Entering directory '/build/postgis-3.5.0~beta1+dfsg/raster/rt_pg'
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I./../../postgis ../../postgis/postgis.sql.in > sql/postgis_for_extension.sql.tmp
/usr/bin/perl -pe 's/BEGIN\;//g ; s/COMMIT\;//g' ../../spatial_ref_sys.sql > sql/spatial_ref_sys.sql
/usr/bin/perl ../../utils/create_spatial_ref_sys_config_dump.pl ../../spatial_ref_sys.sql > sql/spatial_ref_sys_config_dump.sql
grep -v '^#' sql/postgis_for_extension.sql.tmp | \
/usr/bin/perl -lpe \
        "s'MODULE_PATHNAME'\$libdir/postgis-3'g" \
        | /usr/bin/perl -pe 's/BEGIN\;//g ; s/COMMIT\;//g' > sql/postgis_for_extension.sql
echo '-- Just tag extension postgis version as "ANY"' > sql/postgis--TEMPLATED--TO--ANY.sql
echo '-- Installed by postgis 3.5.0beta1' >> sql/postgis--TEMPLATED--TO--ANY.sql
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I../../postgis/ rtpostgis.sql.in > rtpostgis.sql.tmp
cat postgis.control.in \
        | sed -e 's|@EXTVERSION@|3.5.0beta1|g' \
        | sed -e 's|@EXTENSION@|postgis|g' \
        | sed -e 's|@MODULEPATH@|$libdir/postgis-3|g' \
        > postgis.control
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I../../postgis/ rtpostgis_upgrade_cleanup.sql.in > rtpostgis_upgrade_cleanup.sql.tmp
echo '-- Built on 2024-09-18 14:44:20' >> sql/postgis--TEMPLATED--TO--ANY.sql
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I../../postgis/ rtpostgis_drop.sql.in > rtpostgis_drop.sql.tmp
rm -f sql/postgis_for_extension.sql.tmp
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I../../postgis/ rtpostgis.sql.in > rtpostgis.sql.tmp
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I../../postgis/ rtpostgis_legacy.sql.in > rtpostgis_legacy.sql.tmp
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I../../postgis/ rtpostgis.sql.in > rtpostgis.sql.tmp
echo "-- Nothing to do here" > sql/postgis--unpackaged.sql
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I../../postgis/ rtpostgis_upgrade_cleanup.sql.in > rtpostgis_upgrade_cleanup.sql.tmp
printf '\\echo Use "CREATE EXTENSION postgis" to load this file. \\quit\n' > sql/postgis--3.5.0beta1.sql
/usr/bin/perl ../../utils/create_upgrade.pl sql/postgis_for_extension.sql > sql/postgis_upgrade_for_extension.sql.in
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I../../postgis/ rtpostgis_drop.sql.in > rtpostgis_drop.sql.tmp
cat sql/postgis_for_extension.sql sql/spatial_ref_sys_config_dump.sql sql/spatial_ref_sys.sql >> sql/postgis--3.5.0beta1.sql
grep -v '^#' rtpostgis.sql.tmp | \
/usr/bin/perl -lpe "s'MODULE_PATHNAME'\$libdir/postgis_raster-3'g;s'@extschema@\.''g" > rtpostgis.sql
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I../../postgis/ rtpostgis_upgrade_cleanup.sql.in > rtpostgis_upgrade_cleanup.sql.tmp
grep -v '^#' rtpostgis_upgrade_cleanup.sql.tmp | \
/usr/bin/perl -lpe "s'MODULE_PATHNAME'\$libdir/postgis_raster-3'g;s'@extschema@\.''g" > rtpostgis_upgrade_cleanup.sql
grep -v '^#' rtpostgis_drop.sql.tmp | \
/usr/bin/perl -lpe "s'MODULE_PATHNAME'\$libdir/postgis_raster-3'g;s'@extschema@\.''g" > rtpostgis_drop.sql
grep -v '^#' rtpostgis_legacy.sql.tmp | \
/usr/bin/perl -lpe "s'MODULE_PATHNAME'\$libdir/postgis_raster-3'g;s'@extschema@\.''g" > rtpostgis_legacy.sql
grep -v '^#' rtpostgis.sql.tmp | \
/usr/bin/perl -lpe "s'MODULE_PATHNAME'\$libdir/postgis_raster-3'g;s'@extschema@\.''g" > rtpostgis.sql
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I../../postgis/ rtpostgis_legacy.sql.in > rtpostgis_legacy.sql.tmp
rm -f rtpostgis_legacy.sql.tmp
rm -f rtpostgis_upgrade_cleanup.sql.tmp
rm -f rtpostgis.sql.tmp
grep -v '^#' rtpostgis.sql.tmp | \
/usr/bin/perl -lpe "s'MODULE_PATHNAME'\$libdir/postgis_raster-3'g;s'@extschema@\.''g" > rtpostgis.sql
#strip BEGIN/COMMIT since these are not allowed in extensions
grep -v '^#' rtpostgis_upgrade_cleanup.sql.tmp | \
/usr/bin/perl -lpe "s'MODULE_PATHNAME'\$libdir/postgis_raster-3'g;s'@extschema@\.''g" > rtpostgis_upgrade_cleanup.sql
grep -v '^#' rtpostgis_upgrade_cleanup.sql.tmp | \
/usr/bin/perl -lpe "s'MODULE_PATHNAME'\$libdir/postgis_raster-3'g;s'@extschema@\.''g" > rtpostgis_upgrade_cleanup.sql
rm -f rtpostgis_drop.sql.tmp
grep: rtpostgis.sql.tmp: No such file or directory
rm -f rtpostgis.sql.tmp
grep -v '^#' rtpostgis_drop.sql.tmp | \
/usr/bin/perl -lpe "s'MODULE_PATHNAME'\$libdir/postgis_raster-3'g;s'@extschema@\.''g" > rtpostgis_drop.sql
grep: /usr/bin/perl ../../utils/create_upgrade.pl rtpostgis.sql > rtpostgis_upgrade.sql.in
# WARNING: we should make sure these are not internal to function bodies
rtpostgis_upgrade_cleanup.sql.tmp: No such file or directory
grep -v '^#' rtpostgis_legacy.sql.tmp | \
/usr/bin/perl -lpe "s'MODULE_PATHNAME'\$libdir/postgis_raster-3'g;s'@extschema@\.''g" > rtpostgis_legacy.sql
grep: rtpostgis_upgrade_cleanup.sql.tmp: No such file or directory
/usr/bin/perl ../../utils/create_uninstall.pl rtpostgis.sql 170 > uninstall_rtpostgis.sql
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I../../postgis/ rtpostgis_drop.sql.in > rtpostgis_drop.sql.tmp
rm -f rtpostgis.sql.tmp
rm -f rtpostgis_upgrade_cleanup.sql.tmp
grep: rtpostgis_drop.sql.tmp: No such file or directory
Unable to locate target new version number in rtpostgis.sql
cat ../../postgis/common_before_upgrade.sql ../../postgis/postgis_before_upgrade.sql sql/postgis_upgrade_for_extension.sql.in ../../postgis/postgis_after_upgrade.sql ../../postgis/common_after_upgrade.sql | /usr/bin/perl -pe 's/BEGIN\;//g ; s/COMMIT\;//g' > sql/postgis_upgrade_for_extension.sql
make[5]: *** [Makefile:156: rtpostgis_upgrade.sql.in] Error 255
make[5]: *** Deleting file 'rtpostgis_upgrade.sql.in'
make[5]: *** Waiting for unfinished jobs....
/usr/bin/perl ../../utils/create_upgrade.pl rtpostgis.sql > rtpostgis_upgrade.sql.in
rm -f rtpostgis_upgrade_cleanup.sql.tmp
rm -f rtpostgis_drop.sql.tmp
/usr/bin/perl ../../utils/create_uninstall.pl rtpostgis.sql 170 > uninstall_rtpostgis.sql
/usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool -I../../postgis/ rtpostgis_legacy.sql.in > rtpostgis_legacy.sql.tmp
rm -f rtpostgis_legacy.sql.tmp
/usr/bin/perl ../../utils/create_upgrade.pl rtpostgis.sql > rtpostgis_upgrade.sql.in
/usr/bin/perl ../../utils/create_uninstall.pl rtpostgis.sql 170 > uninstall_rtpostgis.sql
/usr/bin/perl -pe "s/BEGIN\;//g ; s/COMMIT\;//g; s/^(DROP .*)\;/SELECT postgis_extension_drop_if_exists('postgis', '\1');\n\1\;/" sql/postgis_upgrade_for_extension.sql > sql/postgis_upgrade.sql
Unable to locate target new version number in rtpostgis.sql
Unable to locate target new version number in rtpostgis.sql
make[5]: Leaving directory '/build/postgis-3.5.0~beta1+dfsg/raster/rt_pg'
make[5]: *** [Makefile:156: rtpostgis_upgrade.sql.in] Error 255
make[5]: *** Deleting file 'rtpostgis_upgrade.sql.in'
make[5]: *** Waiting for unfinished jobs....
make[4]: *** [Makefile:76: ../../raster/rt_pg/rtpostgis_upgrade_cleanup.sql] Error 2
make[4]: *** Deleting file '../../raster/rt_pg/rtpostgis_upgrade_cleanup.sql'
make[4]: *** Waiting for unfinished jobs....
make[5]: *** [Makefile:156: rtpostgis_upgrade.sql.in] Error 255
make[5]: *** Waiting for unfinished jobs....
grep -v '^#' rtpostgis_drop.sql.tmp | \
/usr/bin/perl -lpe "s'MODULE_PATHNAME'\$libdir/postgis_raster-3'g;s'@extschema@\.''g" > rtpostgis_drop.sql
grep: rtpostgis_drop.sql.tmp: No such file or directory
grep -v '^#' rtpostgis_legacy.sql.tmp | \
/usr/bin/perl -lpe "s'MODULE_PATHNAME'\$libdir/postgis_raster-3'g;s'@extschema@\.''g" > rtpostgis_legacy.sql
make[5]: Leaving directory '/build/postgis-3.5.0~beta1+dfsg/raster/rt_pg'
make[4]: *** [Makefile:76: ../../raster/rt_pg/uninstall_rtpostgis.sql] Error 2
make[4]: *** Deleting file '../../raster/rt_pg/uninstall_rtpostgis.sql'
rm -f rtpostgis_drop.sql.tmp
rm -f rtpostgis_legacy.sql.tmp
make[5]: Leaving directory '/build/postgis-3.5.0~beta1+dfsg/raster/rt_pg'
make[4]: *** [Makefile:76: ../../raster/rt_pg/rtpostgis_drop.sql] Error 2
make[4]: *** Deleting file '../../raster/rt_pg/rtpostgis_drop.sql'
make[4]: Leaving directory '/build/postgis-3.5.0~beta1+dfsg/extensions/postgis'
make[3]: *** [Makefile:36: all] Error 1
make[3]: Leaving directory '/build/postgis-3.5.0~beta1+dfsg/extensions'
make[2]: *** [GNUmakefile:34: all] Error 1
make[2]: Leaving directory '/build/postgis-3.5.0~beta1+dfsg'
make[1]: *** [debian/rules:129: override_dh_auto_build] Error 2
make[1]: Leaving directory '/build/postgis-3.5.0~beta1+dfsg'

comment:6 by ImreSamu, 3 weeks ago

I tested commit: 4b786d9 ( POSTGIS="3.5.0dev 4b786d9" ) with "make -j2" + CircleCI Arm64 (aarch64) + Bookworm + testing in Docker-PostGIS environment - and Looks OK, no problems detected 

#26 50.05 + make -j2
#26 50.06 /usr/bin/perl ./utils/repo_revision.pl
#26 50.07 Wrote rev file at 4b786d9

...

#30 0.512  PostgreSQL 17rc1 (Debian 17~rc1-1.pgdg120+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
#30 0.512  POSTGIS="3.5.0dev 4b786d9" [EXTENSION] PGSQL="170" GEOS="3.14.0dev-CAPI-1.20.0" SFCGAL="SFCGAL 2.0.0, CGAL 6.0, BOOST 1.74.0" PROJ="9.6.0 NETWORK_ENABLED=OFF URL_ENDPOINT=https://cdn.proj.org USER_WRITABLE_DIRECTORY=/var/lib/postgresql/.local/share/proj DATABASE_PATH=/usr/local/share/proj/proj.db" (compiled against PROJ 9.14.0) GDAL="GDAL 3.10.0dev-9d7f03ea36, released 2024/09/18" LIBXML="2.9.14" LIBJSON="0.16" LIBPROTOBUF="1.4.1" WAGYU="0.5.0 (Internal)" TOPOLOGY RASTER

I re-ran 3 Docker images in CircleCI-ARM64, and now all three completed successfully (they failed yesterday):

Of course, this could just be a coincidence, and need more test.

By the way, I also noticed the 3.5.0beta1 issue on Arm64 with (Alpine 3.19, Alpine 3.20, Bookworm).

my build Dockerfile

relevant part/command: 

    && git log -1 > /_pgis_last_commit.txt \
    && ldconfig \
    && ./autogen.sh \
# configure options taken from:
# https://anonscm.debian.org/cgit/pkg-grass/postgis.git/tree/debian/rules?h=jessie
    && ./configure \
        --enable-lto \
    && make -j$(nproc) \
    && make install \

comment:7 by sebastic, 3 weeks ago

You can probably make it fail with more parallel processes, on my system I use 11 of the 12 cores.

comment:8 by robe, 3 weeks ago

#5780 might be a duplicate but started seeing it recently so had flagged as a separate.

in reply to:  7 comment:9 by robe, 2 weeks ago

Replying to Bas Couwenberg:

You can probably make it fail with more parallel processes, on my system I use 11 of the 12 cores.

Can you retry with @strk patch he committed at Temptative fix in [be1c2273cdfe0490a43b0148bb8f18ce57a63916/git]

comment:10 by sebastic, 2 weeks ago

Replacing the patch which reverted ​5aba8471 with the changes from both ​4b786d99 and ​be1c2273 also seems to work.

Salsa CI builds succeeded (​1) as did the buildd jobs (​2) on the architectures where they failed before.

comment:11 by robe, 2 weeks ago

Resolution: → fixed
Status: new → closed

great thanks for confirming.

Note: See TracTickets for help on using tickets.