Ticket #1152 (closed defect: invalid)

Opened 3 years ago

Last modified 2 years ago

SRID_MAXIMUM should be raised, gserialized_get/set_srid () should be patched and typemod int too

Reported by: vince Owned by: pramsey
Priority: medium Milestone: PostGIS 2.0.0
Component: postgis Version: trunk
Keywords: srid liblwgeom Cc:

Description

French IGN has defined its own projection codes (authority IGNF) for French needs (e.g. Lambert 93 with grid transformation support in Proj4 for extra precision) around 310000000. Using these projections to import a raster fails with error

gserialized_set_srid called with value (310024140) > SRID_MAXIMUM (999999)

Therefore, the 999999 value should be raised to at least 999 999 999.

Change History

Changed 3 years ago by vince

  • summary changed from SRID_MAXIMUM should be raised to at least 999999999 in liblwgeom.h to SRID_MAXIMUM should be raised to at least 999999999 in liblwgeom.h and gserialized_get_srid () should be patched

Besides, the function gserialized_get_srid() in g_serialized.c should be patched since it assumes wrongly that SRID are at maximum 21 bits long (310024140 = 0x127A97CC).

Changed 3 years ago by vince

  • summary changed from SRID_MAXIMUM should be raised to at least 999999999 in liblwgeom.h and gserialized_get_srid () should be patched to SRID_MAXIMUM should be raised, gserialized_get/set_srid () should be patched and typemod int too

And the 'typemod' int definition must be altered int32 -> int64 since it cannot accommodate a full 32_bit SRID.

Changed 3 years ago by robe

Paul,

Is this going to require a dump reload if we change this? I would assume so for those using typmod at anyrate.

Changed 3 years ago by pramsey

  • status changed from new to closed
  • resolution set to invalid

This is non-possible.

The SRID_MAXIMUM is not a personal whim on my part, it is a hard limit imposed by moving to typmod geometries. The typmod mechanism only allows 32 bits of typmod information (I did lobby for more at pgcon, but that's for the Future). Those 32 bits are allocated as follows

  • Plus/minus = Top bit.
  • Spare bits = Next 2 bits.
  • SRID = Next 21 bits.
  • TYPE = Next 6 bits.
  • ZM Flags = Bottom 2 bits.

So, it can be a *little bit* larger, but not enough to address this ticket.

Please note that external SRIDs of any size can be accommodated via the AUTH_SRID and AUTH_NAME facility, since the internal SRID is not necessarily supposed to map 1:1 to an external key (nor could it universally be expected to), so the entry for IGN would be

insert into spatial_ref_sys 
(srid, auth_name, auth_srid, srtext, proj4text) 
values 
(200000,'IGN', '310024140', 'something', 'something')

Changed 3 years ago by vince

Paul,

I know about typemod, this is well documented in the source.

The knot is not here. THe main hurdle is that it is impossible to import whatever raster with SRID 310024140, it seems. Anytime I try, the generated extent SRID passed with the SQL transaction gets truncated to 21 bit, and the raster line is not imported, because the process reject the extent geometry as bearing not the same SRID as the raster!

Changed 3 years ago by pramsey

Well, it sounds like the raster code needs a tighter limit on SRID number so the error is more explicit, but there's no way we're going to be able to store that SRID directly, it'll need to be a different internal number with a mapping to the external number.

Changed 3 years ago by vince

Okay Paul. Thanks for the info, but that means telling the IGN they should change somehow their definitions, or, at least, propose aliases in their IGNF_spatial_ref_sys.sql file.

Now, what about existing databases? I suppose there are a lot of French databases using 310024140 as SRID. Myself, in the magazine I write, have recommended to use exclusively the IGNF SRID rather than it's EPSG equivalent that does not support the extra grid-based precision. Will all these tables have to undergo an SRID modification?

Now, an additional question: why was that “typemod” composite object mapped to an int32 through bitfields? I mean, we ain’t living in the world of ZX81, Apple IIs or whatnots anymore: We’ve got plenty of memory, C supports structs, so what was the point?

Cheers, Vincent

Changed 3 years ago by robe

I have a somewhat unpleasant compromise to this which assumes there is nothing in the geometry itself being constrained by typmod and its just at the typmod column definition level.

I'm sure Paul and Mark and many others will oppose vehemently. The issue is mostly typmod? We still support the old way using maintenance functions and in fact raster doesn't even support typmod and I don't see how it can at least not for a very long while.

Move the constraint out and have it only apply for geometry columns created using typmod. Then when PostgreSQL finally changes typmod to support 64-bit we can relax again. This check is easy to put in for maintenance functions to check SRID and if above limit for typmod throw a warning and use the old way.

Changed 3 years ago by vince

That seems nice.

I didn't know that typmod was imposed by Pgsql. But (sorry if that question seems stupid), what exactly represents typmod at Pgsql level? V.

Changed 3 years ago by robe

It's stored as a number in the pg_attribute.atttypmod column for each table column.

so if a geometry column was created with the form ADD COLUMN geometry(POINT,4326) you'll see a number in that column corresponding to that. If it was created without typmod, that record would have -1 in the column.

The new geometry_columns view checks both typmod and constraints and uses the helper functions postgis_typmod_* and postgis_constraint_* to parse out the relevant srid, dim and type information from the typmod number or table constraints.

Changed 3 years ago by vince

I see. And the attypemod field is 32-bit… Thanks for the info. V.

Changed 3 years ago by robe

Refer to proposed workaround in #1505

Changed 3 years ago by vince

Great! Thanks for taking this problem into account.

If I’m not mistaken, the IGN has also anticipated the issue and adapted it's SRID to more reasonable values (around 10000). But, as you have pointed out, the problem remains for restoring out of a 1.5 table.

Changed 3 years ago by strk

As of r9145 a dump-reload (required when upgrading to 2.0 anyway) would try to change the SRID values and put them into a reserved zone. In absence of clashes this would let you restore the dump fully, but you'll need further interaction to update your SRIDS so to be moved _out_ of the reserved zone.

Changed 2 years ago by strk

As some references take here when talking about SRID ranges, here's the real (or new?) link to the discussion thread (a lonely one, unfortunately):  http://lists.osgeo.org/pipermail/postgis-devel/2012-February/018440.html

Note: See TracTickets for help on using tickets.