Opened 11 years ago

Closed 11 years ago

#3645 closed patch (fixed)

synchronize QGIS' SRS definitions with GDAL's 1.8.0 SRS database

Reported by: nirvn Owned by: jef
Priority: critical: causes crash or data corruption Milestone: Version 1.7.0
Component: Projection Support Version: Trunk
Keywords: Cc:
Must Fix for Release: No Platform: All
Platform Version: Awaiting user input: no

Description

Some QGIS´ SRS definitions are missing towgs84 values, including UTM Indian 1960 48N, resulting in misalignment of mixed-datum layers with projects with on the on the fly reprojection set.

Folks from the #gdal IRC channel said QGIS should update/synchronize it's SRS definition database with GDAL's 1.8.0 SRS database.

I can confirm that creating a custom SRS using GDAL's version of the UTM Indian 1960 48N definition (same as QGIS except QGIS misses the +towgs84 value) fixed all of my reprojection problems.

(see a bunch of screenshots with misaligned vectors from this now invalid ticket: http://trac.osgeo.org/qgis/ticket/3632)

Pasting some useful bits from a discussion in #gdal: <nIRV> EvenR, what would explain a difference in reprojection between qgis and gdal? <EvenR> it might depend on the GDAL version you have and the definition that QGIS uses for EPSG:3148. GDAL 1.8.0 has a TOWGS84[198,881,317,0,0,0,0] for EPSG:3148. <EvenR> you have to check what SRS definition qgis uses for EPSG:3148. I suspect it has not the TOWGS84 shift <nIRV> +proj=utm +zone=48 +a=6377276.345 +b=6356075.41314024 +units=m +no_defs <nIRV> indeed, there's no towgs84 in the qgis definition <EvenR> it could be appropriate that it synchronizes its SRS definitions (for that one, and also for others) with GDAL 1.8.0 SRS database <EvenR> in GDAL 1.8.0, a lot of TOWGS84 have been added by taking into account the preferred datum shift from the EPSG database <nIRV> omg, it actually works <nIRV> I created a custom crs using the indian 1960 48n definition and adding the +towgs84 bit, et voila <EvenR> you can point to http://trac.osgeo.org/gdal/changeset/18978 which is the changeset during GDAL 1.8.0 where the datum shift logic has been improved

Attachments (4)

srs_sync_gdal_part1.sql (214.0 KB) - added by nirvn 11 years ago.
srs_sync_gdal_part2.sql (171.2 KB) - added by nirvn 11 years ago.
update.php (883 bytes) - added by nirvn 11 years ago.
crs-syncdb.diff (12.6 KB) - added by jef 11 years ago.
patch to introduce crssync for synchronization on install

Download all attachments as: .zip

Change History (16)

Changed 11 years ago by nirvn

Attachment: srs_sync_gdal_part1.sql added

Changed 11 years ago by nirvn

Attachment: srs_sync_gdal_part2.sql added

comment:1 Changed 11 years ago by nirvn

Owner: changed from nobody to nirvn
Status: newassigned

I created a tiny script to a) flag the srs.db's tbl_srs 'parameters' column needing updates and b) ouput the sql update query.

The script compared qgis' srs.db against the proj4's EPSG file (http://trac.osgeo.org/proj/browser/trunk/proj/nad/epsg), the latter already synched with GDAL.

2157 SRS parameters needed update (mostly +towgs84 additions).

Changed 11 years ago by nirvn

Attachment: update.php added

comment:2 Changed 11 years ago by nirvn

I've attached the script used to come up with the sql update statements. It's unfortunately not in python but in php.

comment:3 Changed 11 years ago by jef

Type: bugpatch

the attached patch updates our db with the parameters of PROJ. This could be run on automatically at installation (using the python API).

Unfortunately there currently is a bug in PROJ that causes the routine to crash after fetching 16 projections (PROJ #100).

comment:4 Changed 11 years ago by nirvn

jef, as mentioned over irc, the updated srs.db you applied in changeset 15584 (http://trac.osgeo.org/qgis/changeset/15584) doesn't appear to have the latest definition for many datums (based in gdal 1.8).

To give one example, the epsg 3148 (indian 1960 48n) in your srs.db is: +proj=utm +zone=48 +a=6377276.345 +b=6356075.41314024 +units=m +no_defs

It's missing the +towgs84 declaration, found in latest gdal 1.8 and proj.4 epsg file: +proj=utm +zone=48 +a=6377276.345 +b=6356075.41314024 +towgs84=198,881,317,0,0,0,0 +units=m +no_defs

Without the +towgs84 declaration, on the fly reprojection of wgs84 & epsg 3148 datums have a +450m margin difference. It also affects coverting layers from one datum to the other using qgis' built-in "save as..." function.

You based your updated srs.db on proj 4.7.0 (http://trac.osgeo.org/proj/browser/tags/4.7.0/proj/nad/epsg) which the devs synched with latest gdal definitions afterwards (http://trac.osgeo.org/proj/browser/trunk/proj/nad/epsg).

Would it be possible to ship 1.7 with a srs.db synched with gdal latest datum definitions?

comment:5 in reply to:  4 Changed 11 years ago by msieczka

Replying to nirvn:

Would it be possible to ship 1.7 with a srs.db synched with gdal latest datum definitions?

Couple of months ago I have written a Shell script which creates a new srs.db based on the output of localy installed PROJ.4 and GDAL: http://trac.osgeo.org/qgis/browser/trunk/qgis/scripts/qgis_srs.sh.

It was used once to update the srs.db in SVN, about a year ago. After that, a few manual edits have been done on srs.db too.

The script would not retain those manual non-GDAL-derived SRS definitions. They would need to be taken care of. I guess that as of current trunk SVN these are srs_id>=3646.

Changed 11 years ago by jef

Attachment: crs-syncdb.diff added

patch to introduce crssync for synchronization on install

comment:7 Changed 11 years ago by pcav

Now EPSG:3003 is defined in QGIS as:

+proj=tmerc +lat_0=0 +lon_0=9 +k=0.9996 +x_0=1500000 +y_0=0 +ellps=intl
+towgs84=-104.1,-49.1,-9.9,0.971,-2.917,0.714,-11.68 +units=m +no_defs

instead of the old:

+proj=tmerc +lat_0=0 +lon_0=9 +k=0.9996 +x_0=1500000 +y_0=0 +ellps=intl
+no_defs

Similarly for EPSG:3004.

The corrections have been already inserted as fake EPSG codes:

40000
+proj=tmerc +lat_0=0 +lon_0=9 +k=0.9996 +x_0=1500000 +y_0=0 +ellps=intl
+units=m +towgs84=-104.1,-49.1,-9.9,0.971,-2.917,0.714,-11.68 +no_defs
40001
+proj=tmerc +lat_0=0 +lon_0=15 +k=0.9996 +x_0=2520000 +y_0=0 +ellps=intl
+units=m +towgs84=-104.1,-49.1,-9.9,0.971,-2.917,0.714,-11.68 +no_defs
40002
+proj=tmerc +lat_0=0 +lon_0=9 +k=0.9996 +x_0=1500000 +y_0=0 +ellps=intl
+units=m +towgs84=-168.6,-34.0,38.6,-0.374,-0.679,-1.379,-9.48 +no_defs
40003
+proj=tmerc +lat_0=0 +lon_0=9 +k=0.9996 +x_0=1500000 +y_0=0 +ellps=intl
+units=m +towgs84=-50.2,-50.4,84.8,-0.690,-2.012,0.459,-28.08 +no_defs

So the fake codes 40000 and 40001 are now duplicates, and should be removed. 40002 and 40002 are useful, as they define ad hoc datum for Sardinia and Sicily.

comment:8 in reply to:  7 Changed 11 years ago by jef

Replying to pcav:

So the fake codes 40000 and 40001 are now duplicates, and should be removed. 40002 and 40002 are useful, as they define ad hoc datum for Sardinia and Sicily.

The EPGS numbers are prefered. Removing the definitions might break layers that are assigned to one of those.

comment:9 Changed 11 years ago by jpalmer

The patch from http://trac.osgeo.org/qgis/ticket/3136 have also been reverted by this update from gdal. Can we re-apply them?

comment:10 Changed 11 years ago by jef

Must Fix for Release: YesNo

comment:11 Changed 11 years ago by jef

Owner: changed from nirvn to jef
Status: assignednew

comment:12 Changed 11 years ago by jef

Resolution: fixed
Status: newclosed

applied in commit:a9aafd07

Note: See TracTickets for help on using tickets.