#5370 closed defect (fixed)
bessie regress failures on raster after upgrade to GDAL 3.6.1 and Proj 9.2.0
| Reported by: | robe | Owned by: | robe |
|---|---|---|---|
| Priority: | blocker | Milestone: | PostGIS 3.4.0 |
| Component: | QA/buildbots | Version: | 3.3.x |
| Keywords: | Cc: |
Description
To get around #5368, I took off the cirrus FreeBSD 12.3 testing and upgraded bessie so she's testing more or less same versions of things. Though cirrus was never testing raster. I think we had disabled raster testing on cirrus a while ago cause it was giving mysterious errors.
But anyway after the upgrade surgery, bessie is in tears https://debbie.postgis.net/view/PostGIS/job/PostGIS_Worker_Run/4437/label=bessie/console
with these errors which might just be a postgres account issue that somehow the context changed and the account running under doesn't have rights to ENABLE_ALL / DISABLE_ALL anymore
20:32:28 raster/test/regress/check_gdal .. failed (diff expected obtained: /home/jenkins/tmp/pgis_reg_5da0ec6e80a4a1e414433e79289482c35ef32474/test_230_diff) 20:32:28 ----------------------------------------------------------------------------- 20:32:28 --- ./raster/test/regress/check_gdal_expected 2023-04-14 20:25:42.515822000 -0400 20:32:28 +++ /home/jenkins/tmp/pgis_reg_5da0ec6e80a4a1e414433e79289482c35ef32474/test_230_out 2023-04-14 20:32:28.456935000 -0400 20:32:28 @@ -3,7 +3,9 @@ 20:32:28 DISABLE_ALL 20:32:28 ENABLE_ALL 20:32:28 t 20:32:28 +WARNING: Unknown GDAL driver: PNG 20:32:28 +WARNING: Unknown GDAL driver: JPEG 20:32:28 GTiff PNG JPEG 20:32:28 -t 20:32:28 +f 20:32:28 DISABLE_ALL 20:32:28 t 20:32:28 ----------------------------------------------------------------------------- 20:32:49 raster/test/regress/rt_fromgdalraster .. failed (diff expected obtained: /home/jenkins/tmp/pgis_reg_5da0ec6e80a4a1e414433e79289482c35ef32474/test_275_diff) 20:32:49 ----------------------------------------------------------------------------- 20:32:49 --- ./raster/test/regress/rt_fromgdalraster_expected 2023-04-14 20:25:42.529874000 -0400 20:32:49 +++ /home/jenkins/tmp/pgis_reg_5da0ec6e80a4a1e414433e79289482c35ef32474/test_275_out 2023-04-14 20:32:49.627946000 -0400 20:32:49 @@ -1,3 +1,4 @@ 20:32:49 -(0,0,90,90,1,-1,0,0,0,3)|(8100,1479000,182.592592592593,114.982851945091,0,255)|(8100,1479000,182.592592592593,114.982851945091,0,255)|(8100,1453500,179.444444444444,116.438931167192,0,255) 20:32:49 +WARNING: Unknown GDAL driver: PNG 20:32:49 +ERROR: RASTER_fromGDALRaster: Could not open bytea with GDAL. Check that the bytea is of a GDAL supported format 20:32:49 1|(-168,85,10,10,0.083,-0.083,0,0,4269,3)|(100,0,0,0,0,0)|(100,0,0,0,0,0)|(100,0,0,0,0,0) 20:32:49 2|(-168,85,10,10,0.083,-0.083,0,0,4326,3)|(100,0,0,0,0,0)|(100,0,0,0,0,0)|(100,0,0,0,0,0) 20:32:49 ----------------------------------------------------------------------------- 20:32:49 raster/test/regress/rt_asgdalraster .. failed (diff expected obtained: /home/jenkins/tmp/pgis_reg_5da0ec6e80a4a1e414433e79289482c35ef32474/test_276_diff) 20:32:50 ----------------------------------------------------------------------------- 20:32:50 --- ./raster/test/regress/rt_asgdalraster_expected 2023-04-14 20:25:42.525932000 -0400 20:32:50 +++ /home/jenkins/tmp/pgis_reg_5da0ec6e80a4a1e414433e79289482c35ef32474/test_276_out 2023-04-14 20:32:49.988208000 -0400 20:32:50 @@ -1,16 +1,18 @@ 20:32:50 +WARNING: Unknown GDAL driver: PNG at character 49 20:32:50 +WARNING: Unknown GDAL driver: JPEG at character 49 20:32:50 0 20:32:50 1 20:32:50 1 20:32:50 1 20:32:50 -1 20:32:50 -1 20:32:50 -1 20:32:50 -1 20:32:50 -1 20:32:50 -1 20:32:50 -1 20:32:50 -1 20:32:50 -1 20:32:50 -1 20:32:50 -1 20:32:50 -1 20:32:50 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:50 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:50 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:50 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:50 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:50 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:50 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:50 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:50 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:50 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:50 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:50 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:50 ----------------------------------------------------------------------------- 20:32:50 raster/test/regress/rt_astiff .. ok in 384 ms 20:32:50 raster/test/regress/rt_asjpeg .. failed (diff expected obtained: /home/jenkins/tmp/pgis_reg_5da0ec6e80a4a1e414433e79289482c35ef32474/test_278_diff) 20:32:50 ----------------------------------------------------------------------------- 20:32:50 --- ./raster/test/regress/rt_asjpeg_expected 2023-04-14 20:25:42.526029000 -0400 20:32:50 +++ /home/jenkins/tmp/pgis_reg_5da0ec6e80a4a1e414433e79289482c35ef32474/test_278_out 2023-04-14 20:32:50.753779000 -0400 20:32:50 @@ -1,10 +1,11 @@ 20:32:50 +WARNING: Unknown GDAL driver: JPEG at character 18 20:32:50 t 20:32:50 ERROR: The pixel type of band 1 in the raster is not 8BUI. The JPEG format can only be used with the 8BUI pixel type. 20:32:50 -1 20:32:50 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:50 ERROR: The pixel type of band 1 in the raster is not 8BUI. The JPEG format can only be used with the 8BUI pixel type. 20:32:50 -1 20:32:50 -1 20:32:50 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:50 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:50 ERROR: The pixel type of band 1 in the raster is not 8BUI. The JPEG format can only be used with the 8BUI pixel type. 20:32:50 NOTICE: The JPEG format only permits one or three bands. The first band will be used. 20:32:50 ERROR: The pixel type of band 1 in the raster is not 8BUI. The JPEG format can only be used with the 8BUI pixel type. 20:32:50 -1 20:32:50 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:50 ----------------------------------------------------------------------------- 20:32:50 raster/test/regress/rt_aspng .. failed (diff expected obtained: /home/jenkins/tmp/pgis_reg_5da0ec6e80a4a1e414433e79289482c35ef32474/test_279_diff) 20:32:51 ----------------------------------------------------------------------------- 20:32:51 --- ./raster/test/regress/rt_aspng_expected 2023-04-14 20:25:42.526326000 -0400 20:32:51 +++ /home/jenkins/tmp/pgis_reg_5da0ec6e80a4a1e414433e79289482c35ef32474/test_279_out 2023-04-14 20:32:51.109555000 -0400 20:32:51 @@ -1,11 +1,12 @@ 20:32:51 +WARNING: Unknown GDAL driver: PNG at character 17 20:32:51 t 20:32:51 ERROR: The pixel type of band 1 in the raster is not 8BUI or 16BUI. The PNG format can only be used with 8BUI and 16BUI pixel types. 20:32:51 -1 20:32:51 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:51 ERROR: The pixel type of band 1 in the raster is not 8BUI or 16BUI. The PNG format can only be used with 8BUI and 16BUI pixel types. 20:32:51 -1 20:32:51 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:51 ERROR: The pixel type of band 1 in the raster is not 8BUI or 16BUI. The PNG format can only be used with 8BUI and 16BUI pixel types. 20:32:51 ERROR: The pixel type of band 1 in the raster is not 8BUI or 16BUI. The PNG format can only be used with 8BUI and 16BUI pixel types. 20:32:51 ERROR: The pixel type of band 1 in the raster is not 8BUI or 16BUI. The PNG format can only be used with 8BUI and 16BUI pixel types. 20:32:51 -1 20:32:51 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:51 NOTICE: The PNG format only permits one, three or four bands. The first band will be used. 20:32:51 -1 20:32:51 +ERROR: rt_raster_to_gdal: Could not load the output GDAL driver 20:32:51 ----------------------------------------------------------------------------- 20:33:21 raster/test/regress/permitted_gdal_drivers .. failed (diff expected obtained: /home/jenkins/tmp/pgis_reg_5da0ec6e80a4a1e414433e79289482c35ef32474/test_311_diff) 20:33:22 ----------------------------------------------------------------------------- 20:33:22 --- ./raster/test/regress/permitted_gdal_drivers_expected 2023-04-14 20:25:42.525312000 -0400 20:33:22 +++ /home/jenkins/tmp/pgis_reg_5da0ec6e80a4a1e414433e79289482c35ef32474/test_311_out 2023-04-14 20:33:21.981975000 -0400 20:33:22 @@ -1,3 +1,5 @@ 20:33:22 +WARNING: Unknown GDAL driver: PNG 20:33:22 +WARNING: Unknown GDAL driver: JPEG 20:33:22 4326 20:33:22 ERROR: RASTER_fromGDALRaster: Could not open bytea with GDAL. Check that the bytea is of a GDAL supported format 20:33:22 ERROR: rt_band_load_offline_data: Cannot open offline raster: <VRTDataset rasterXSize="1000" rasterYSize="1"><VRTRasterBand band="1" subClass="VRTRawRasterBand"><SourceFilename>/etc/passwd</SourceFilename></VRTRasterBand></VRTDataset> 20:33:22 -----------------------------------------------------------------------------
Before she was running with:
-------------- Compiler Info ------------- 23:11:05 C compiler: cc -std=gnu99 -g -O2 -fno-math-errno -fno-signed-zeros -Wall 23:11:05 C++ compiler (Wagyu): cc -std=c++11 -x c++ 23:11:05 C++ compiler (FlatGeobuf): cc -std=c++11 -x c++ 23:11:05 CPPFLAGS: -I/usr/local/include -I/usr/local/include -I/usr/local/include -I/usr/local/include/libxml2 -I/usr/local/include -I/usr/local/include -I/usr/local/include/json-c -I/usr/local/include -DNDEBUG 23:11:05 LDFLAGS: -lm 23:11:05 SQL preprocessor: /usr/bin/cpp -traditional-cpp -w -P 23:11:05 Archiver: llvm-ar rs 23:11:05 23:11:05 -------------- Additional Info ------------- 23:11:05 POSTGIS lib file includes minor: ENABLED 23:11:05 Interrupt Tests: DISABLED use: --with-interrupt-tests to enable 23:11:05 23:11:05 -------------- Dependencies -------------- 23:11:05 GEOS config: /usr/local/bin/geos-config 23:11:05 GEOS version: 3.10.3 23:11:05 GDAL config: /usr/local/bin/gdal-config 23:11:05 GDAL version: 3.5.0 23:11:05 SFCGAL config: /usr/local/bin/sfcgal-config 23:11:05 SFCGAL version: 1.4.1 23:11:05 PostgreSQL config: /usr/local/bin/pg_config 23:11:05 PostgreSQL version: PostgreSQL 13.7 23:11:05 PROJ4 version: 82 23:11:05 Libxml2 config: /usr/local/bin/xml2-config 23:11:05 Libxml2 version: 2.9.13 23:11:05 JSON-C support: yes 23:11:05 protobuf support: yes 23:11:05 protobuf-c version: 1004000 23:11:05 PCRE support: Version 2 23:11:05 Perl: /usr/local/bin/perl 23:11:05 23:11:05 --------------- Extensions --------------- 23:11:05 PostgreSQL EXTENSION support: enabled 23:11:05 PostGIS Raster: enabled 23:11:05 PostGIS Topology: enabled 23:11:05 SFCGAL support: enabled 23:11:05 Address Standardizer support: enabled 23:11:05 23:11:05 -------- Documentation Generation -------- 23:11:05 xsltproc: /usr/local/bin/xsltproc 23:11:05 xsl style sheets: 23:11:05 dblatex: 23:11:05 convert: /usr/local/bin/convert 23:11:05 mathml2.dtd: http://www.w3.org/Math/DTD/mathml2/mathml2.dtd 23:11:05
and now she is running:
20:26:02 -------------- Dependencies -------------- 20:26:02 GEOS config: /usr/local/bin/geos-config 20:26:02 GEOS version: 3.11.1 20:26:02 GDAL config: /usr/local/bin/gdal-config 20:26:02 GDAL version: 3.6.2 20:26:02 SFCGAL config: /usr/local/bin/sfcgal-config 20:26:02 SFCGAL version: 1.4.1 20:26:02 PostgreSQL config: /usr/local/bin/pg_config 20:26:02 PostgreSQL version: PostgreSQL 13.10 20:26:02 PROJ4 version: 92 20:26:02 Libxml2 config: /usr/local/bin/xml2-config 20:26:02 Libxml2 version: 2.10.3 20:26:02 JSON-C support: yes 20:26:02 protobuf support: yes 20:26:02 protobuf-c version: 1004001 20:26:02 PCRE support: Version 2 20:26:02 Perl: /usr/local/bin/perl
Change History (7)
comment:1 by , 13 months ago
comment:2 by , 13 months ago
Thanks. At a glance it looks like that patch might also fix the issue I am seeing with most of the drivers missing.
The regression failures I traced back to missing drivers and not permission issue as I had originally though.
For example doing this:
SET postgis.gdal_enabled_drivers = 'ENABLE_ALL'; SELECT short_name, long_name FROM ST_GDALDrivers();
Just lists:
short_name | long_name ------------+-------------------------------------------- VRT | Virtual Raster DERIVED | Derived datasets using VRT pixel functions GTiff | GeoTIFF COG | Cloud optimized GeoTIFF generator HFA | Erdas Imagine Images (.img) MEM | In Memory Raster (6 rows)
I did a
pkg update && pkg upgrade
and only thing that got upgraded was sqlite. So I'm assuming that patch hasn't been pushed out yet.
comment:4 by , 11 months ago
Did you use quarterly or latest for pkg? Cf /etc/pkg/FreeBSD.conf
To enable latest:
mkdir -p /usr/local/etc/pkg/repos
echo "FreeBSD: { enabled: no }" > /usr/local/etc/pkg/repos/FreeBSD.conf
sed -e "s/quarterly/latest/" /etc/pkg/FreeBSD.conf > /usr/local/etc/pkg/repos/FreeBSD_latest.conf
comment:5 by , 11 months ago
@lbartoletti
Sorry didn't see your note until I did some more things. But I think it's all resolved now.
My /usr/local/etc/pkg/repos/FreeBSD.conf
looks like:
FreeBSD: {
url: "pkg+http://pkg.FreeBSD.org/${ABI}/quarterly",
mirror_type: "srv",
signature_type: "fingerprints",
fingerprints: "/usr/share/keys/pkg",
enabled: yes
}
I used ports instead
portsnap fetch extract #first time # there after portsnap fetch update export ALLOW_UNSUPPORTED_SYSTEM=yes cd /usr/ports/graphics/gdal/ && make install clean make config # to change options
And unchecked some of the things like MySQL, ODBC etc, just keeping the basics.
That seemed to work fine and gave me GDAL 3.7.1
Last run of 3.2 branch looked good and had the below. I'm going to renable raster testing on bessie now on the 3.3 and master that I had disabled it on.
llvmorg-13.0.0-0-gd7b669b3a303), 64-bit 12:53:15 Postgis 3.2.6dev - (59a8f9d) - 2023-06-20 16:49:51 12:53:15 scripts 3.2.6dev 59a8f9d 12:53:15 raster scripts 3.2.6dev 59a8f9d 12:53:15 GEOS: 3.11.1-CAPI-1.17.1 12:53:15 PROJ: 9.2.0 12:53:15 SFCGAL: 1.4.1 12:53:15 GDAL: GDAL 3.7.0, released 2023/05/02
comment:6 by , 11 months ago
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
Note:
See TracTickets
for help on using tickets.
"unknown driver" is caused by missing dependency for GDAL. It's fixed now, cf: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=269708