Opened 6 years ago

Closed 6 years ago

Last modified 6 years ago

#2396 closed defect (fixed)

regression tests fail on big-endian architectures

Reported by: mnencia Owned by: pramsey
Priority: medium Milestone: PostGIS 2.1.2
Component: postgis Version: 2.1.x
Keywords: Cc: mwanner

Description

Building postgis 2.0.3 for debian unstable (buildd logs here https://buildd.debian.org/status/package.php?p=postgis) we got a cunit error on all big-endian architectures that build correctly (spark, s390, powerpc, mips)

Cofigure on spark says:

PostGIS is now configured for sparc-unknown-linux-gnu

 -------------- Compiler Info ------------- 
  C compiler:           sparc-linux-gnu-gcc -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security
  C++ compiler:         sparc-linux-gnu-g++ -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security
  SQL preprocessor:     /usr/bin/cpp -traditional-cpp -P

 -------------- Dependencies -------------- 
  GEOS config:          /usr/bin/geos-config
  GEOS version:         3.3.3
  GDAL config:          /usr/bin/gdal-config
  GDAL version:         1.9.0
  PostgreSQL config:    /usr/lib/postgresql/9.1/bin/pg_config
  PostgreSQL version:   PostgreSQL 9.1.9
  PROJ4 version:        47
  Libxml2 config:       /usr/bin/xml2-config
  Libxml2 version:      2.9.1
  JSON-C support:       yes
  PostGIS debug level:  0
  Perl:                 /usr/bin/perl

 --------------- Extensions --------------- 
  PostGIS Raster:       enabled
  PostGIS Topology:     enabled

 -------- Documentation Generation -------- 
  xsltproc:             /usr/bin/xsltproc
  xsl style sheets:     /usr/share/xml/docbook/stylesheet/nwalsh
  dblatex:              /usr/bin/dblatex
  convert:              /usr/bin/convert
  mathml2.dtd:          http://www.w3.org/Math/DTD/mathml2/mathml2.dtd

The failure is:

Exp:   0103000000010000000500000000917E9BA468294100917E9B8AEA2841C976BE1FA4682941C976BE9F8AEA2841B39ABE1FA46829415ACCC29F8AEA284137894120A4682941C976BE9F8AEA284100917E9BA468294100917E9B8AEA2841
Obt:   00000000030000000100000005412968A49B7E91004128EA8A9B7E9100412968A41FBE76C94128EA8A9FBE76C9412968A41FBE9AB34128EA8A9FC2CC5A412968A4204189374128EA8A9FBE76C9412968A49B7E91004128EA8A9B7E9100
  Test: test_lwgeom_force_clockwise ... FAILED
    1. cu_libgeom.c:750  - CU_ASSERT_STRING_EQUAL(in_ewkt,out_ewkt)

Attachments (1)

fix-big-endian (6.3 KB) - added by mwanner 6 years ago.
comment out a few more tests that rely on little-endian serialization

Download all attachments as: .zip

Change History (43)

comment:1 Changed 6 years ago by mwanner

Cc: mwanner added

comment:2 Changed 6 years ago by robe

This seems a little bizarre to me because the two representations appear to be identical (I'm running on windows 64-bit postgis 2.2.0)

try running this query and let me know if you get the same results as I do.

WITH outs As (SELECT 'exp'::text As type, '0103000000010000000500000000917E9BA468294100917E9B8AEA2841C976BE1FA4682941C976BE9F8AEA2841B39ABE1FA46829415ACCC29F8AEA284137894120A4682941C976BE9F8AEA284100917E9BA468294100917E9B8AEA2841'::geometry As geom
UNION ALL
SELECT 'obt'::text As type,'00000000030000000100000005412968A49B7E91004128EA8A9B7E9100412968A41FBE76C94128EA8A9FBE76C9412968A41FBE9AB34128EA8A9FC2CC5A412968A4204189374128EA8A9FBE76C9412968A49B7E91004128EA8A9B7E9100'::geometry As geom)
SELECT a.type, ST_Equals(a.geom, b.geom) As is_equal
 , ST_OrderingEquals(a.geom, b.geom) As is_ord_equal , ST_Ndims(a.geom)
FROM outs As a INNER JOIN outs As b ON (a.type != b.type);

the output I get for above is

type | is_equal | is_ord_equal | st_ndims
-----+----------+--------------+----------
exp  | t        | t            |        2
obt  | t        | t            |        2

comment:3 Changed 6 years ago by mwanner

On my ARM box, this yields:

 type | is_equal | is_ord_equal | st_ndims 
------+----------+--------------+----------
 exp  | t        | t            |        2
 obt  | t        | t            |        2
(2 rows)

So, yes, that looks like the same result.

comment:4 Changed 6 years ago by robe

Okay I'm guess the problem might be the default output on endian is different and the cunit is taking the default behavior.

For example try this:

WITH outs As (SELECT 'exp'::text As type, '0103000000010000000500000000917E9BA468294100917E9B8AEA2841C976BE1FA4682941C976BE9F8AEA2841B39ABE1FA46829415ACCC29F8AEA284137894120A4682941C976BE9F8AEA284100917E9BA468294100917E9B8AEA2841'::geometry As geom
UNION ALL
SELECT 'obt'::text As type,'00000000030000000100000005412968A49B7E91004128EA8A9B7E9100412968A41FBE76C94128EA8A9FBE76C9412968A41FBE9AB34128EA8A9FC2CC5A412968A4204189374128EA8A9FBE76C9412968A49B7E91004128EA8A9B7E9100'::geometry As geom)
SELECT a.type, ST_Equals(a.geom, b.geom) As is_equal
 , ST_OrderingEquals(a.geom, b.geom) As is_ord_equal , ST_Ndims(a.geom)
, ST_AsEWKB(a.geom, 'NDR')  = ST_AsEWKB(a.geom, 'XDR') As xdr_geom
FROM outs As a INNER JOIN outs As b ON (a.type != b.type);

I think the only solution is to change the cunit test to be explicit about endianess and not leave it up to the OS.

pramsey (Paul) -- what is you take on this issue?

comment:5 Changed 6 years ago by mnencia

On Debian sparc porter box this is the last query results

 type | is_equal | is_ord_equal | st_ndims | xdr_geom
------+----------+--------------+----------+----------
 exp  | t        | t            |        2 | f
 obt  | t        | t            |        2 | f
(2 rows)

comment:6 Changed 6 years ago by mwanner

Same on ARM here:

 type | is_equal | is_ord_equal | st_ndims | xdr_geom 
------+----------+--------------+----------+----------
 exp  | t        | t            |        2 | f
 obt  | t        | t            |        2 | f
(2 rows)

comment:7 Changed 6 years ago by mwanner

Oh.. wait a second... my ARM is an 'armel' one. That's little endian and passes all the CUnit tests, including test_lwgeom_force_clockwise this ticket is about. Please excuse the noise. (It also fails, but later and differently; I'll file another ticket for that).

comment:8 Changed 6 years ago by robe

What exactly is an armel? Anyrate probably the best thing for me to do is explicitly specify the endian for this cunt test.

comment:9 Changed 6 years ago by mwanner

AFAIUI ARM processors can generally run in any of the two endianess modes. However, most other components require the processor to run in either big or little endian. Debian armel is a port for ARM running in little-endian mode. I cannot switch that box to big-endian.

comment:10 Changed 6 years ago by robe

Milestone: PostGIS 2.1.0PostGIS 2.2.0

pramsey you think we should do anything with this or just leave the test alone. Your call. I'm pushing to 2.2.0. Switch back if you can get to it before 2.1.0 goes out the door.

comment:11 Changed 6 years ago by mwanner

Uh.. this is postgis-2.0 failing. I'd expect a bug fix in 2.0.4. Or are you saying these are not supported architectures for 2.0 (or even 2.1)?

comment:12 Changed 6 years ago by robe

Milestone: PostGIS 2.2.0PostGIS 2.0.4

Switched to 2.0.4. They should in theory be supported, but just not many people running them so hard to know when it's broken for these architectures. I suspect this is just the unit test issue, and not something serious. I'm just nto sure how hard it is to change the unit test since I haven't scanned yet to see what it is doing.

comment:13 Changed 6 years ago by mnencia

I'm afraid it's not a matter of an unit test issue. I ran the regression tests on Sparc and I obtained a couple of failures.

I attach them here as I don't know if the issues are related.

--- regress_expected	2012-02-27 22:08:12.000000000 +0000
+++ /tmp/pgis_reg/test_24_out	2013-08-01 14:44:21.000000000 +0000
@@ -154,7 +154,7 @@
 148|LINESTRING(0 0,5 0,10 0)
 149|GEOMETRYCOLLECTION EMPTY
 150|SRID=6;GEOMETRYCOLLECTION(POLYGON((0 0,1 0,1 1,0 1,0 0)))
-151|0103000020E61000000100000005000000000000000000000000000000000000000000000000000000000000000000F03F000000000000F03F000000000000F03F000000000000F03F000000000000000000000000000000000000000000000000
+151|0020000003000010E600000001000000050000000000000000000000000000000000000000000000003FF00000000000003FF00000000000003FF00000000000003FF0000000000000000000000000000000000000000000000000000000000000
 152|4326
 152.1|t
 152.2|4326
@@ -167,7 +167,7 @@
 159|MULTIPOLYGON EMPTY
 160|MULTIPOINT(1 1)
 161|MULTILINESTRING((0 0,1 1),(2 2,3 3))
-162|010200000003000000F771D98DE33826C00000000000004440F771D98DE33826C000000000008051400000000000805140F771D98DE33826C0
+162|000000000200000003C02638E38DD971F74044000000000000C02638E38DD971F740518000000000004051800000000000C02638E38DD971F7
 163|POLYGON((0 0 0,1 0 0,1 1 0,0 1 0,0 0 0))
 164|POLYGON((0 0 0,1 0 0,1 1 0,0 1 0,0 0 1))
 ERROR:  geometry contains non-closed rings
--- empty_expected	2012-02-27 22:08:12.000000000 +0000
+++ /tmp/pgis_reg/test_35_out	2013-08-01 14:44:32.000000000 +0000
@@ -23,13 +23,13 @@
 T3.16|
 T3.17|
 T3.18|
-ST_Buffer(empty, tolerance) == empty|010300000000000000
-ST_Union(geometry, empty) == geometry|0103000000010000000400000000000000000000000000000000000000000000000000244000000000000000000000000000001440000000000000144000000000000000000000000000000000
-ST_Union(empty, empty) == empty|010300000000000000
-ST_Intersection(geometry, empty) == geometry|010300000000000000
-ST_Intersection(empty, empty) == empty|010300000000000000
-ST_Difference(geometry, empty) == geometry|0103000000010000000400000000000000000000000000000000000000000000000000244000000000000000000000000000001440000000000000144000000000000000000000000000000000
-ST_Difference(empty, geometry) == empty|010300000000000000
+ST_Buffer(empty, tolerance) == empty|000000000300000000
+ST_Union(geometry, empty) == geometry|0000000003000000010000000400000000000000000000000000000000402400000000000000000000000000004014000000000000401400000000000000000000000000000000000000000000
+ST_Union(empty, empty) == empty|000000000300000000
+ST_Intersection(geometry, empty) == geometry|000000000300000000
+ST_Intersection(empty, empty) == empty|000000000300000000
+ST_Difference(geometry, empty) == geometry|0000000003000000010000000400000000000000000000000000000000402400000000000000000000000000004014000000000000401400000000000000000000000000000000000000000000
+ST_Difference(empty, geometry) == empty|000000000300000000
 ST_Distance(geometry, empty) == NULL|
 ST_DWithin(geometry, empty, tolerance) == FALSE|f
 ST_Within(geometry, empty) == FALSE|f
@@ -47,8 +47,8 @@
 ST_NRings(empty) == 0|0
 ST_NumPoints(empty) == 0|0
 ST_NPoints(empty) == 0|0
-ST_GeometryN(empty, n) == empty|010300000000000000
-ST_ExteriorRing(empty) == empty|010200000000000000
+ST_GeometryN(empty, n) == empty|000000000300000000
+ST_ExteriorRing(empty) == empty|000000000200000000
 ST_InteriorRingN(empty, n) == NULL|
 ST_Area(empty) == 0|0
 ST_Length(empty) == 0|0
--- sql-mm-compoundcurve_expected	2012-02-27 22:08:12.000000000 +0000
+++ /tmp/pgis_reg/test_42_out	2013-08-01 14:44:37.000000000 +0000
@@ -66,11 +66,11 @@
 public.compoundcurve.the_geom_3dz effectively removed.
 public.compoundcurve.the_geom_3dm effectively removed.
 public.compoundcurve.the_geom_2d effectively removed.
-valid wkt compound curve 1|0109000000020000000102000000030000009FE5797057376340E09398B1B2373BC05AAE0A165F0963409F6760A2493D3DC0DB6286DFB057634082D8A1B32F843EC0010200000004000000DB6286DFB057634082D8A1B32F843EC075B4E4D0C60C634031FA5D1A371540C0D7197CED9B636340A3CB59A7630A41C050F4A72AC0FB6240974769FCE3CF41C0
-valid wkt compound curve 2|0109000000010000000102000000060000009FE5797057376340E09398B1B2373BC05AAE0A165F0963409F6760A2493D3DC0DB6286DFB057634082D8A1B32F843EC075B4E4D0C60C634031FA5D1A371540C0D7197CED9B636340A3CB59A7630A41C050F4A72AC0FB6240974769FCE3CF41C0
-valid wkt compound curve 3|0109000000010000000102000000030000000CE586D73CF36240BBC46888F0523BC0102E91C951E76240DF90A1BEC0F841C0F970C100FFD7624074ADE6CE86CD3BC0
-valid wkt compound curve 4|0109000000020000000102000000030000009FE5797057376340E09398B1B2373BC05AAE0A165F0963409F6760A2493D3DC0DB6286DFB057634082D8A1B32F843EC0010800000005000000DB6286DFB057634082D8A1B32F843EC0DB6286DFB057634082D8A1B32F843EC075B4E4D0C60C634031FA5D1A371540C0D7197CED9B636340A3CB59A7630A41C050F4A72AC0FB6240974769FCE3CF41C0
-valid wkt compound curve 5|010900000003000000010800000003000000468280E724BC6340BF4B46210B973BC0F890AEA18D8063402D9664151D483CC0EED64BB6EE726340903CA5BDA0863AC0010200000004000000EED64BB6EE726340903CA5BDA0863AC09FE5797057376340E09398B1B2373BC05AAE0A165F0963409F6760A2493D3DC0DB6286DFB057634082D8A1B32F843EC0010800000005000000DB6286DFB057634082D8A1B32F843EC0DB6286DFB057634082D8A1B32F843EC075B4E4D0C60C634031FA5D1A371540C0D7197CED9B636340A3CB59A7630A41C050F4A72AC0FB6240974769FCE3CF41C0
+valid wkt compound curve 1|000000000900000002000000000200000003406337577079E59FC03B37B2B19893E04063095F160AAE5AC03D3D49A260679F406357B0DF8662DBC03E842FB3A1D882000000000200000004406357B0DF8662DBC03E842FB3A1D88240630CC6D0E4B475C04015371A5DFA314063639BED7C19D7C0410A63A759CBA34062FBC02AA7F450C041CFE3FC694797
+valid wkt compound curve 2|000000000900000001000000000200000006406337577079E59FC03B37B2B19893E04063095F160AAE5AC03D3D49A260679F406357B0DF8662DBC03E842FB3A1D88240630CC6D0E4B475C04015371A5DFA314063639BED7C19D7C0410A63A759CBA34062FBC02AA7F450C041CFE3FC694797
+valid wkt compound curve 3|0000000009000000010000000002000000034062F33CD786E50CC03B52F08868C4BB4062E751C9912E10C041F8C0BEA190DF4062D7FF00C170F9C03BCD86CEE6AD74
+valid wkt compound curve 4|000000000900000002000000000200000003406337577079E59FC03B37B2B19893E04063095F160AAE5AC03D3D49A260679F406357B0DF8662DBC03E842FB3A1D882000000000800000005406357B0DF8662DBC03E842FB3A1D882406357B0DF8662DBC03E842FB3A1D88240630CC6D0E4B475C04015371A5DFA314063639BED7C19D7C0410A63A759CBA34062FBC02AA7F450C041CFE3FC694797
+valid wkt compound curve 5|0000000009000000030000000008000000034063BC24E7808246C03B970B21464BBF4063808DA1AE90F8C03C481D1564962D406372EEB64BD6EEC03A86A0BDA53C90000000000200000004406372EEB64BD6EEC03A86A0BDA53C90406337577079E59FC03B37B2B19893E04063095F160AAE5AC03D3D49A260679F406357B0DF8662DBC03E842FB3A1D882000000000800000005406357B0DF8662DBC03E842FB3A1D882406357B0DF8662DBC03E842FB3A1D88240630CC6D0E4B475C04015371A5DFA314063639BED7C19D7C0410A63A759CBA34062FBC02AA7F450C041CFE3FC694797
 ERROR:  incontinuous compound curve
 ERROR:  incontinuous compound curve
 ERROR:  geometry must have an odd number of points
@@ -80,8 +80,8 @@
 valid wkb compound curve 4|COMPOUNDCURVE((153.72942375 -27.2175704,152.29285719 -29.23940482,154.74034096 -30.51635287),CIRCULARSTRING(154.74034096 -30.51635287,154.74034096 -30.51635287,152.39926953 -32.16574411,155.11278414 -34.08116619,151.86720784 -35.62414508))
 valid wkb compound curve 5|COMPOUNDCURVE(CIRCULARSTRING(157.87950492 -27.59001358,156.01728901 -28.28169378,155.59163966 -26.52589021),(155.59163966 -26.52589021,153.72942375 -27.2175704,152.29285719 -29.23940482,154.74034096 -30.51635287),CIRCULARSTRING(154.74034096 -30.51635287,154.74034096 -30.51635287,152.39926953 -32.16574411,155.11278414 -34.08116619,151.86720784 -35.62414508))
 null response|
-minpoints issues - pass|01090000000100000001020000000200000000000000000000000000000000000000000000000000F03F000000000000F03F
-minpoints issues - pass|010900000001000000010800000003000000000000000000000000000000000000000000000000000000000000000000F03F000000000000F03F000000000000F03F
+minpoints issues - pass|000000000900000001000000000200000002000000000000000000000000000000003FF00000000000003FF0000000000000
+minpoints issues - pass|0000000009000000010000000008000000030000000000000000000000000000000000000000000000003FF00000000000003FF00000000000003FF0000000000000
 ERROR:  geometry requires more points
 ERROR:  geometry requires more points
 ERROR:  geometry requires more points
--- sql-mm-curvepoly_expected	2012-02-27 22:08:12.000000000 +0000
+++ /tmp/pgis_reg/test_43_out	2013-08-01 14:44:38.000000000 +0000
@@ -71,11 +71,11 @@
 public.curvepolygon.the_geom_3dm effectively removed.
 public.curvepolygon.the_geom_3dz effectively removed.
 public.curvepolygon.the_geom_4d effectively removed.
-valid wkt curve polygon 1|010A00000001000000010200000007000000CCDF061AD9F3614054093E6D99093EC0AB9085DBB6DD614081540229216040C0EBD7A828C33E62409BF026782A7E41C0000000C06BB2624000000020ADB440C08E632F616EAD6240C9F7B0BF1DD33DC09011EEC0DE4362407DD6672F76323EC0CCDF061AD9F3614054093E6D99093EC0
-valid wkt curve polygon 2|010A00000002000000010200000007000000CCDF061AD9F3614054093E6D99093EC0AB9085DBB6DD614081540229216040C0EBD7A828C33E62409BF026782A7E41C0000000C06BB2624000000020ADB440C08E632F616EAD6240C9F7B0BF1DD33DC09011EEC0DE4362407DD6672F76323EC0CCDF061AD9F3614054093E6D99093EC00102000000060000006844C4FE011B6240342E2993E0423FC0D45DAF9D93066240C4A0C305D62240C000000080AC31624000000020FBBE40C0000000E0107F6240000000C0A10440C04E1C0C14624C6240BF3FB6405C793FC06844C4FE011B6240342E2993E0423FC0
-valid wkt curve polygon 3|010A00000001000000010800000007000000CCDF061AD9F3614054093E6D99093EC0AB9085DBB6DD614081540229216040C0EBD7A828C33E62409BF026782A7E41C0000000C06BB2624000000020ADB440C08E632F616EAD6240C9F7B0BF1DD33DC09011EEC0DE4362407DD6672F76323EC0CCDF061AD9F3614054093E6D99093EC0
-valid wkt curve polygon 4|010A00000002000000010800000007000000CCDF061AD9F3614054093E6D99093EC0AB9085DBB6DD614081540229216040C0EBD7A828C33E62409BF026782A7E41C0000000C06BB2624000000020ADB440C08E632F616EAD6240C9F7B0BF1DD33DC09011EEC0DE4362407DD6672F76323EC0CCDF061AD9F3614054093E6D99093EC00102000000060000006844C4FE011B6240342E2993E0423FC0D45DAF9D93066240C4A0C305D62240C000000080AC31624000000020FBBE40C0000000E0107F6240000000C0A10440C04E1C0C14624C6240BF3FB6405C793FC06844C4FE011B6240342E2993E0423FC0
-valid wkt curve polygon 5|010A00000002000000010200000007000000CCDF061AD9F3614054093E6D99093EC0AB9085DBB6DD614081540229216040C0EBD7A828C33E62409BF026782A7E41C0000000C06BB2624000000020ADB440C08E632F616EAD6240C9F7B0BF1DD33DC09011EEC0DE4362407DD6672F76323EC0CCDF061AD9F3614054093E6D99093EC00109000000030000000108000000030000006844C4FE011B6240342E2993E0423FC0D45DAF9D93066240C4A0C305D62240C000000080AC31624000000020FBBE40C001020000000200000000000080AC31624000000020FBBE40C0000000E0107F6240000000C0A10440C0010800000003000000000000E0107F6240000000C0A10440C04E1C0C14624C6240BF3FB6405C793FC06844C4FE011B6240342E2993E0423FC0
+valid wkt curve polygon 1|000000000A000000010000000002000000074061F3D91A06DFCCC03E09996D3E09544061DDB6DB8590ABC04060212902548140623EC328A8D7EBC0417E2A7826F09B4062B26BC0000000C040B4AD200000004062AD6E612F638EC03DD31DBFB0F7C9406243DEC0EE1190C03E32762F67D67D4061F3D91A06DFCCC03E09996D3E0954
+valid wkt curve polygon 2|000000000A000000020000000002000000074061F3D91A06DFCCC03E09996D3E09544061DDB6DB8590ABC04060212902548140623EC328A8D7EBC0417E2A7826F09B4062B26BC0000000C040B4AD200000004062AD6E612F638EC03DD31DBFB0F7C9406243DEC0EE1190C03E32762F67D67D4061F3D91A06DFCCC03E09996D3E095400000000020000000640621B01FEC44468C03F42E093292E34406206939DAF5DD4C04022D605C3A0C4406231AC80000000C040BEFB2000000040627F10E0000000C04004A1C000000040624C62140C1C4EC03F795C40B63FBF40621B01FEC44468C03F42E093292E34
+valid wkt curve polygon 3|000000000A000000010000000008000000074061F3D91A06DFCCC03E09996D3E09544061DDB6DB8590ABC04060212902548140623EC328A8D7EBC0417E2A7826F09B4062B26BC0000000C040B4AD200000004062AD6E612F638EC03DD31DBFB0F7C9406243DEC0EE1190C03E32762F67D67D4061F3D91A06DFCCC03E09996D3E0954
+valid wkt curve polygon 4|000000000A000000020000000008000000074061F3D91A06DFCCC03E09996D3E09544061DDB6DB8590ABC04060212902548140623EC328A8D7EBC0417E2A7826F09B4062B26BC0000000C040B4AD200000004062AD6E612F638EC03DD31DBFB0F7C9406243DEC0EE1190C03E32762F67D67D4061F3D91A06DFCCC03E09996D3E095400000000020000000640621B01FEC44468C03F42E093292E34406206939DAF5DD4C04022D605C3A0C4406231AC80000000C040BEFB2000000040627F10E0000000C04004A1C000000040624C62140C1C4EC03F795C40B63FBF40621B01FEC44468C03F42E093292E34
+valid wkt curve polygon 5|000000000A000000020000000002000000074061F3D91A06DFCCC03E09996D3E09544061DDB6DB8590ABC04060212902548140623EC328A8D7EBC0417E2A7826F09B4062B26BC0000000C040B4AD200000004062AD6E612F638EC03DD31DBFB0F7C9406243DEC0EE1190C03E32762F67D67D4061F3D91A06DFCCC03E09996D3E095400000000090000000300000000080000000340621B01FEC44468C03F42E093292E34406206939DAF5DD4C04022D605C3A0C4406231AC80000000C040BEFB20000000000000000200000002406231AC80000000C040BEFB2000000040627F10E0000000C04004A1C000000000000000080000000340627F10E0000000C04004A1C000000040624C62140C1C4EC03F795C40B63FBF40621B01FEC44468C03F42E093292E34
 ERROR:  geometry contains non-closed rings
 ERROR:  geometry contains non-closed rings
 ERROR:  geometry contains non-closed rings
@@ -89,7 +89,7 @@
 valid ewkb curve polygon 3|CURVEPOLYGON(CIRCULARSTRING(143.620251668383 -30.0374973560768,142.928571472997 -32.751011968744,145.961323098919 -34.9856710615288,149.575653076172 -33.4115333557129,149.419724075848 -29.8246726805735,146.120941605547 -30.1971158627043,143.620251668383 -30.0374973560768))
 valid ewkb curve polygon 4|CURVEPOLYGON(CIRCULARSTRING(143.620251668383 -30.0374973560768,142.928571472997 -32.751011968744,145.961323098919 -34.9856710615288,149.575653076172 -33.4115333557129,149.419724075848 -29.8246726805735,146.120941605547 -30.1971158627043,143.620251668383 -30.0374973560768),(144.843993552527 -31.2612392402209,144.205519526017 -32.2721564488616,145.552307128906 -33.4920387268066,147.970809936523 -32.0361862182617,146.386972449926 -31.4740639157242,144.843993552527 -31.2612392402209))
 valid ewkb curve polygon 5|CURVEPOLYGON((143.620251668383 -30.0374973560768,142.928571472997 -32.751011968744,145.961323098919 -34.9856710615288,149.575653076172 -33.4115333557129,149.419724075848 -29.8246726805735,146.120941605547 -30.1971158627043,143.620251668383 -30.0374973560768),COMPOUNDCURVE(CIRCULARSTRING(144.843993552527 -31.2612392402209,144.205519526017 -32.2721564488616,145.552307128906 -33.4920387268066),(145.552307128906 -33.4920387268066,147.970809936523 -32.0361862182617),CIRCULARSTRING(147.970809936523 -32.0361862182617,146.386972449926 -31.4740639157242,144.843993552527 -31.2612392402209)))
-valid curve 6|010A0000000200000001090000000200000001080000000500000000000000000000000000000000000000000000000000004000000000000000000000000000000040000000000000F03F00000000000000400000000000000840000000000000104000000000000008400102000000040000000000000000001040000000000000084000000000000010400000000000001440000000000000F03F000000000000104000000000000000000000000000000000010800000003000000333333333333FB3F000000000000F03F666666666666F63F9A9999999999D93F333333333333FB3F000000000000F03F
+valid curve 6|000000000A00000002000000000900000002000000000800000005000000000000000000000000000000004000000000000000000000000000000040000000000000003FF0000000000000400000000000000040080000000000004010000000000000400800000000000000000000020000000440100000000000004008000000000000401000000000000040140000000000003FF00000000000004010000000000000000000000000000000000000000000000000000008000000033FFB3333333333333FF00000000000003FF66666666666663FD999999999999A3FFB3333333333333FF0000000000000
 ERROR:  geometry requires more points
-valid curve 8|010A0000000200000001090000000200000001080000000500000000000000000000000000000000000000000000000000004000000000000000000000000000000040000000000000F03F00000000000000400000000000000840000000000000104000000000000008400102000000020000000000000000001040000000000000084000000000000000000000000000000000010800000003000000333333333333FB3F000000000000F03F666666666666F63F9A9999999999D93F333333333333FB3F000000000000F03F
+valid curve 8|000000000A00000002000000000900000002000000000800000005000000000000000000000000000000004000000000000000000000000000000040000000000000003FF0000000000000400000000000000040080000000000004010000000000000400800000000000000000000020000000240100000000000004008000000000000000000000000000000000000000000000000000008000000033FFB3333333333333FF00000000000003FF66666666666663FD999999999999A3FFB3333333333333FF0000000000000
 null response|
--- tickets_expected	2012-12-19 18:28:02.000000000 +0000
+++ /tmp/pgis_reg/test_66_out	2013-08-01 14:44:50.000000000 +0000
@@ -23,7 +23,7 @@
 #122|CIRCULARSTRING(220268 150415,220227 150505,220227 150406)
 #124a|COMPOUNDCURVE(CIRCULARSTRING(0 0,1 1,1 0),(1 0,30 5),CIRCULARSTRING(30 5,34 56,67 89))
 ERROR:  incontinuous compound curve
-#145a|0103000020E610000000000000
+#145a|0020000003000010E600000000
 #145b|0
 #146|0|t|GEOMETRYCOLLECTION(LINESTRING(0 0,-1 -1),MULTIPOINT(1 2,2 3))
 ERROR:  Invalid hex string, length (267) has to be a multiple of two!
@@ -47,7 +47,7 @@
 #213|17
 #234|COMPOUNDCURVE((0 0,1 1))
 #241|0
-#254|010700000000000000
+#254|000000000700000000
 #259|
 #260|1667701
 #261|0
@@ -97,7 +97,7 @@
 #650|MULTIPOINT(0 0,1 1,2 2)
 #667|SRID=4326;CURVEPOLYGON(CIRCULARSTRING(30 40,-49.2314112161292 32.1963871193548,30 40))
 #677|1121395
-#680|01d107000000000000000024c000000000000049400000000000000040
+#680|00000007d1c02400000000000040490000000000004000000000000000
 #681a|
 #681b|
 #681c|
@@ -105,28 +105,28 @@
 #681e|
 #681f|
 #681g|
-#682|0103000020E610000000000000
-#683|0103000020E610000000000000
+#682|0020000003000010E600000000
+#683|0020000003000010E600000000
 #684,#2109|SRID=4326;POINT EMPTY
 #2109|SRID=3395;POINT EMPTY
-#685|0103000020E610000000000000
-#686|0107000020E610000000000000
+#685|0020000003000010E600000000
+#686|0020000007000010E600000000
 #687|f
 #689|f
 #690
-010200000003000000F771D98DE33826C00000000000004440F771D98DE33826C000000000008051400000000000805140F771D98DE33826C0
-#693a|0103000060E61000000100000005000000EA95B20C71C851C02B1895D409204540000000000000F03F9CC420B072C851C0C7BAB88D062045400000000000000840B1506B9A77C851C08E75711B0D20454000000000000000C0FF21FDF675C851C0F2D24D6210204540000000000000F03FEA95B20C71C851C02B1895D4092045400000000000000000
-#693b|0103000060E61000000100000007000000EA95B20C71C851C0AA605452272045400000000000000000386744696FC851C04703780B2420454000000000000034408638D6C56DC851C04703780B2420454000000000000034C08638D6C56DC851C0E3A59BC42020454000000000000014408638D6C56DC851C08048BF7D1D20454000000000000010409CC420B072C851C04703780B242045400000000000001840EA95B20C71C851C0AA605452272045400000000000003E40
+000000000200000003C02638E38DD971F74044000000000000C02638E38DD971F740518000000000004051800000000000C02638E38DD971F7
+#693a|0060000003000010E60000000100000005C051C8710CB295EA40452009D495182B3FF0000000000000C051C872B020C49C404520068DB8BAC74008000000000000C051C8779A6B50B14045200D1B71758EC000000000000000C051C875F6FD21FF40452010624DD2F23FF0000000000000C051C8710CB295EA40452009D495182B0000000000000000
+#693b|0060000003000010E60000000100000007C051C8710CB295EA40452027525460AA0000000000000000C051C86F69446738404520240B7803474034000000000000C051C86DC5D63886404520240B780347C034000000000000C051C86DC5D6388640452020C49BA5E34014000000000000C051C86DC5D638864045201D7DBF48804010000000000000C051C872B020C49C404520240B7803474018000000000000C051C8710CB295EA40452027525460AA403E000000000000
 #694
 ERROR:  Shell is not a line
 #695
 ERROR:  First argument must be a LINESTRING
-#696|010F000080060000000103000080010000000500000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000F03F0000000000000000000000000000F03F000000000000F03F0000000000000000000000000000F03F0000000000000000000000000000000000000000000000000000000000000000010300008001000000050000000000000000000000000000000000000000000000000000000000000000000000000000000000F03F0000000000000000000000000000F03F000000000000F03F0000000000000000000000000000F03F0000000000000000000000000000000000000000000000000000000000000000000000000000000001030000800100000005000000000000000000000000000000000000000000000000000000000000000000F03F00000000000000000000000000000000000000000000F03F0000000000000000000000000000F03F00000000000000000000000000000000000000000000F03F00000000000000000000000000000000000000000000000001030000800100000005000000000000000000F03F000000000000F03F0000000000000000000000000000F03F000000000000F03F000000000000F03F000000000000F03F0000000000000000000000000000F03F000000000000F03F00000000000000000000000000000000000000000000F03F000000000000F03F0000000000000000010300008001000000050000000000000000000000000000000000F03F00000000000000000000000000000000000000000000F03F000000000000F03F000000000000F03F000000000000F03F000000000000F03F000000000000F03F000000000000F03F00000000000000000000000000000000000000000000F03F00000000000000000103000080010000000500000000000000000000000000000000000000000000000000F03F000000000000F03F0000000000000000000000000000F03F000000000000F03F000000000000F03F000000000000F03F0000000000000000000000000000F03F000000000000F03F00000000000000000000000000000000000000000000F03F
+#696|008000000F0000000600800000030000000100000005000000000000000000000000000000000000000000000000000000000000000000000000000000003FF000000000000000000000000000003FF00000000000003FF000000000000000000000000000003FF000000000000000000000000000000000000000000000000000000000000000000000000000000080000003000000010000000500000000000000000000000000000000000000000000000000000000000000003FF000000000000000000000000000003FF00000000000003FF000000000000000000000000000003FF000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000300000001000000050000000000000000000000000000000000000000000000003FF0000000000000000000000000000000000000000000003FF000000000000000000000000000003FF0000000000000000000000000000000000000000000003FF0000000000000000000000000000000000000000000000000000000000000008000000300000001000000053FF00000000000003FF000000000000000000000000000003FF00000000000003FF00000000000003FF00000000000003FF000000000000000000000000000003FF00000000000003FF0000000000000000000000000000000000000000000003FF00000000000003FF000000000000000000000000000000080000003000000010000000500000000000000003FF0000000000000000000000000000000000000000000003FF00000000000003FF00000000000003FF00000000000003FF00000000000003FF00000000000003FF00000000000003FF0000000000000000000000000000000000000000000003FF0000000000000000000000000000000800000030000000100000005000000000000000000000000000000003FF00000000000003FF000000000000000000000000000003FF00000000000003FF00000000000003FF00000000000003FF000000000000000000000000000003FF00000000000003FF0000000000000000000000000000000000000000000003FF0000000000000
 #720|MULTIPOINT(-1113194.91 4838471.4,-1113194.91 7326837.72,-1113194.91 11028513.63,556597.45 4838471.4,556597.45 7326837.72,556597.45 11028513.63,2226389.82 4838471.4,2226389.82 7326837.72,2226389.82 11028513.63,3896182.18 4838471.4,3896182.18 7326837.72,3896182.18 11028513.63,5565974.54 4838471.4,5565974.54 7326837.72,5565974.54 11028513.63)
-#723|0101000020E61000006284F068E33826C00000000000004440
-#723|0107000020E610000000000000
-#723|0107000020E610000000000000
-#723|0101000020E61000006284F068E33826C00100000000804B40
+#723|0020000001000010E6C02638E368F084624044000000000000
+#723|0020000007000010E600000000
+#723|0020000007000010E600000000
+#723|0020000001000010E6C02638E368F08462404B800000000001
 #804|<gml:Point srsName="urn:ogc:def:crs:EPSG::4326"><gml:pos srsDimension="2">0 0</gml:pos></gml:Point>
 #845|t
 #834|GEOMETRYCOLLECTION(POINT(0 0 5),LINESTRING(10 10 5,10 0 5))
@@ -182,7 +182,7 @@
 #1453.2|f
 #1454|t
 #1414|CURVEPOLYGON Z EMPTY
-#1478|01040000200100000000000000
+#1478|00200000040000000100000000
 #745|GEOMETRYCOLLECTION(POLYGON((-72 42 1,-70 43 1,-71 41 1,-72 42 1)))
 #1450|POINT|POLYGON
 #1482|4326
@@ -217,7 +217,7 @@
 #1697.2|0
 #1697.3|1024
 #1734.1|1026
-#1755|01010000A0E6100000000000000040554000000000008041400000000000000000
+#1755|00A0000001000010E6405540000000000040418000000000000000000000000000
 #1776|POLYGON((0 0,10 0,10 10,0 0))|POLYGON((0 0,10 0,10 10,0 0))
 #1791|4.7
 ERROR:  ST_Segmentize: invalid max_distance 0 (must be >= 0)

comment:14 Changed 6 years ago by robe

Well 1478 seems to be the endian issue since:

SELECT ST_OrderingEquals(ST_GeomFromEWKT('00200000040000000100000000'), ST_GeomFromEWKT('01040000200100000000000000'))

returns true for me (can you verify it returns true for you?_ so they are the same geometry. Looks like 1755 is same endian issue. So painful solution I would guess is to make our tests endian agnostic.

pramsey -- are you around to venture an opinion on this issue?

comment:15 Changed 6 years ago by mnencia

The query is true also for me on sparc

comment:16 Changed 6 years ago by robe

revised at r11738 for 2.1.

Okay in looking at this. I just noticed this particular test:

 Test: test_lwgeom_force_clockwise ... FAILED
    1. cu_libgeom.c:750  - CU_ASSERT_STRING_EQUAL(in_ewkt,out_ewkt)

uses compare of wkbs which could be endian specific instead of the ewkt compare. I've run into situations like this and switched these tests to do a same check (which is more or less equivalent to ST_OrderingEquals).

The output fprintf is something we generally only do when we are debugging a test and not in the final iteration of a test. Though makes sense for us to include as part of test as a cross-check. That is neither here nor there. As a quick fix, I'm just getting rid of the fprintf and making it a standard same ASSERT check.

Paul can change back if he feels the need to be more verbose.

comment:17 Changed 6 years ago by mwanner

r11738 seems to fix the issue for all tested big-endian arches (i.e. spark, s390, powerpc, mips). Please back port to 2.0 so the fix will be available in 2.0.4. (It applies without conflict - as already done for Debian).

comment:18 Changed 6 years ago by robe

backported to 2.0 at r11748, up-ported to 2.2 at r11749.

I'll close this ticket out once I've figured out what to do with the PostgreSQL specific tests. I haven't had a chance to look at those yet.

comment:19 Changed 6 years ago by pramsey

There's still a lot of online regression test errors in this ticket. I think the answer to all of them is to wrap the bare geometry outut in ST_AsBinary(...,'ndr') to force an endianness on them. I can do that for the provided examples, but perhaps there are others hiding in the woodwork.

comment:20 Changed 6 years ago by mwanner

Note that I've uploaded postgis-2.1 to Debian. AFAICT it shows very similar failures. See: https://buildd.debian.org/status/package.php?p=postgis

comment:21 Changed 6 years ago by pramsey

Online tests fixed in 2.0 at r11934.

This changeset should be applied to 2.1 and 2.2, but probably can't just be passed to patch, going to just move this ticket ahead to 2.1 milestone.

comment:22 Changed 6 years ago by pramsey

Milestone: PostGIS 2.0.4PostGIS 2.1.1
Version: 2.0.x2.1.x

comment:23 Changed 6 years ago by robe

Are you planning to get to this in the next 2 weeks. Wanta release 2.1 in another 2 weeks or so. I think your patch above might work. I can help out with that. since 2.1 doesn't support 8.4, we don't have to worry about the bytea stuff so much though I guess some folks might have it set to escape so we'd want to force one way or the other.

comment:24 Changed 6 years ago by pramsey

It's terrible, detailed, boring work! I may get to it. If you beat me, I wouldn't complain.

comment:25 Changed 6 years ago by robe

okay I'll try to do it when I'm tired of thinking and in the mood for detail to get my mind off thinking!

comment:26 Changed 6 years ago by robe

Summary: liblwgeom CUnit tests fail on big-endian architectures.liblwgeom CUnit and PostgreSQL tests fail on big-endian architectures.

Done for 2.2 (trunk) at r11973

comment:27 Changed 6 years ago by robe

copied the wrong tickets file -- amended at r11974

comment:28 Changed 6 years ago by robe

Resolution: fixed
Status: newclosed

2.2 made some booboos so really r11973 - r11978

Backported to 2.1 at r11979 (for 2.1 assuming its enough the same as 2.2 that I could just copy over the files.

comment:29 Changed 6 years ago by mwanner

Resolution: fixed
Status: closedreopened

I'm sorry to report this, Debian's buildd still shows some issues that look related to endianness: https://buildd.debian.org/status/package.php?p=postgis

postgis-2.1.0-3 is the 2.1.0 release with r11979 plus a few other patches, see here: http://anonscm.debian.org/gitweb/?p=pkg-grass/postgis.git;a=tree;f=debian/patches;h=dac08de0246bc716c4405c24eedc61b09e5f9a1e;hb=677a81952a57a6cb893e24e2db908b3d019f69c4

comment:30 Changed 6 years ago by robe

I've patched the #690 mips/sparc tickets.sql failure for 2.1 at r11983

#2048 as far as I can tell is not an endian issue. Those are geos checks which I would expect to resolve the same regardless the endian input of the geometries. Which version of GEOS are you running on armel and armhf?

-#2048|1|f|f
+#2048|1|f|t
 #2048|2|t|t
-#2048|3|f|f
+#2048|3|f|t

The mipsel failure

make: *** [check] Error 1
-4|0|1
-Topology 'tt3d' dropped
+psql: FATAL:  the database system is in recovery mode

Looks like a GEOS bug too which crashed your PostGIS instance -- let us know what version of GEOS you are running. It may be patched in a newer GEOS or a bug in GEOS yet to be patched.

comment:31 Changed 6 years ago by robe

I did see a transposition incosistency but I reversed it the wrong way. This failing ticket should really refer to ticket #2084 so I'll have to fix that.

The sfcgal tickets.sql still needs to be fixed too but I'm not setup to test sfcgal so I am hesitant to change that. I'll put that one as a separate ticket to patch.

comment:32 Changed 6 years ago by robe

done for trunk 2.2. at r11991 (include ticket transpostion error fix). done ticket transposition fix for 2.1 at r11990.

comment:33 Changed 6 years ago by pramsey

Altering test for #2084 and reversing the within/coveredby functions would be interesting, since if they failed the same way (t/f) then it would indicate a problem in the cache logic, which is being uncovered on those chipsets (how?). Not sure how to debug without having access to an appropriate platform.

comment:34 Changed 6 years ago by mwanner

I applied r11983 and r11990 to the debian package (not uploaded to unstable, quite yet, though).

Debian has geos-3.3.3-1 in stable, testing, and unstable and on all architectures. The build log's configure output confirms this.

I'm unclear on #2084: can this now be an issue related to endianness?

comment:35 Changed 6 years ago by robe

my feeling (and I think pramsey agrees) is that #2084 is unlikely an endian issue, but might have something to do with architecture that you are catching and we are missing in caching logic. If you can run by reversing the tests as he says and provide the output would help diagnose if its a caching bug. You can also test with one row instead of two and see if the singleton answer is the same as the table answer.

I guess the fact all are running with the same GEOS might suggest that GEOS may not be the problem. Although there have been a few fixes in latest GEOS versions that are platform specific. Can't remember what they are offhand but I think more likely than an endian issue.

Any chance you can run with the latest 3.3 release. GEOS 3.3.3 is pretty old. GEOS 3.3.9 is latest of released GEOS 3.3 stable.

comment:36 Changed 6 years ago by robe

Resolution: fixed
Status: reopenedclosed

I'm going to close this one out since I feel we've dealt wth endian specific isues and whatever is left is a different issue.

Changed 6 years ago by mwanner

Attachment: fix-big-endian added

comment out a few more tests that rely on little-endian serialization

comment:37 Changed 6 years ago by mwanner

Resolution: fixed
Status: closedreopened
Summary: liblwgeom CUnit and PostgreSQL tests fail on big-endian architectures.regression tests fail on big-endian architectures

It turned out there are several other big-endian related issues remaining in 2.1.1. For that version to pass tests successfully on all Debian-supported architectures, I also had to disable a few more tests that checked against a little-endian serialization of data.

While I agree that's not a good way to make tests pass, it's the only option I saw ATM. Especially given that there seem to be a few other places doing the same, I guess a better solution isn't available, now.

comment:38 Changed 6 years ago by pramsey

Milestone: PostGIS 2.1.1PostGIS 2.1.2

If you could post the tests and diffs that failed, we can similarly wrap them in endian-independant outputs prior to comparison (or, rather, endian-explicit outputs).

comment:39 Changed 6 years ago by mwanner

The patch I added highlights the tests that fail. Please look up the logs from Debian for how exactly they failed - I cannot easily reproduce that, ATM.

comment:40 Changed 6 years ago by pramsey

Thanks, sorry I missed your patch upload, that was all the guidance I needed.

2.1 patch at r12226 and trunk at r12225

Please close if resolved.

comment:41 Changed 6 years ago by robe

Resolution: fixed
Status: reopenedclosed

It's been 6weeks. I'm going to assume mwanner is on holiday and this is a done deal. These are all regress tests anyway so don't affect build.

comment:42 Changed 6 years ago by mwanner

Sorry, I missed the "close if resolved" part. Seems okay to me.

Note: See TracTickets for help on using tickets.