QGIS fails to read undefined projection from user datum in shape.prj file

[neteler@…]

Hi,

when opening Italian maps, we usually have to define a "user" datum since PROJ4 doesn't contain the definitions (as there are 3 datums for the same EPSC code in Italy, depending on where you are). The QgsSpatialRefSys detection fails, however:

qgis ammprvBL.shp Warning: QgsSpatialRefSys::getRecord failed : select * from tbl_srs where parameters='+proj=tmerc +lat_0=0 +lon_0=9 +k=0.999600 +x_0=1500000 +y_0=0 +ellps=intl +towgs84=-104.1,-49.1,-9.9,0.971,-2.917,0.714,-11.68 +units=m +no_defs'

If I take out the towgs84 part from the .prj file, it works. It would be great to make QGIS tolerant for user defined towgs84 parameters which is rather the standard in Europe (many countries, many datums).

Proposal as general solution: Maybe the towgs84 part could be temporarily taken out for the SQL search and re-added after the getRecord request was done.

Best, Markus

PS: Attached the .prj file content

[neteler@…]

offending .proj file with towgs84 string

[wonder]

This is not that trivial as it looked like. Let's explain what happens when QGIS tries to match a projection with its spatial reference database:

• check whether projection's name is in user's or global database; if not
• check whether the whole proj4 string is in user's or global database; if not
• try to find heuristically by traversing database and using OGRSpatialReference::IsSame()

In your case first two options don't find equivalent in database, but third does and tells QGIS that it's the same projection as "+proj=tmerc +lat_0=0 +lon_0=9 +k=0.999600 +x_0=1500000 +y_0=0 +ellps=intl +units=m +no_defs". The problem is that OGR's IsSame() function is not robust enough to make a difference between this and the specified ones. (See ogrspatialreference.cpp in OGR if interested)

To get it working correctly you have to add it as a custom projection (menu Settings -> Custom projection). If IsSame() worked correctly, it wouldn't recognize any projection and you would be asked to select one from database or a default one would be used (depending on settings).

Thus I'm closing this ticket as it seems that we're unable to fix it. ... probably this should be reported to OGR project...?

[neteler@…]

You said that "The problem is that OGR's IsSame() function is not robust enough to make a difference between this and the specified ones." In fact, there is close to no difference besides the towgs84 part which would "just" be needed to be transferred to the projection found by QGIS in it's DB.

I don't know if failing on that completely and not recognize any projection is the best solution here. I think that the IsSame() heuristics work quite well. Substracting the input and the output (from IsSame()) projection string will show just towgs84 as difference which is perfect.

Best, Markus

[wonder]

Well, it seems that towgs84 parameter is quite important. Maybe in this case there's really small difference. But when looking to the QGIS's database of projections, there are (many) projections that specify towgs84 parameter, so I guess that it might be quite important - removing it temporarily would be incorrect.

A solution might be to use original proj string even if another projection has been declared as the same. But I don't know if this won't break other things or create some inconsistencies.

[neteler@…]

Hi,

I think we have a missunderstanding here. I meant to say that the towgs84 parameter should be taken out while *parsing* the string (to match the QGIS SRS DB), then reinserted, after having found the matching EPSG (which usually lacks the towgs84 parameter.

In case that a towgs84 parameter is already present in the QGIS SRS DB, it should use the user provided one (maybe pop up a notice that it was doing so).

Hope that's clarified now what I tried to say.

Markus

[wonder]

Replying to neteler@itc.it:

I think we have a missunderstanding here. I meant to say that the towgs84 parameter should be taken out while *parsing* the string (to match the QGIS SRS DB), then reinserted, after having found the matching EPSG (which usually lacks the towgs84 parameter.

I understand your idea. But the problem is in the way how QGIS copes with spatial references. Once proj4 string is read, QGIS searches for a SRS ID in SRS database. SRS ID is unique for every projection. Thus even if two projection are nearly same (e.g. with or without towgs84) they must have different SRS ID to enable QGIS understand the difference.

That's why taking out towgs84 temporiraly makes no sense - another SRS ID would be chosen instead.

Rescue from this situation is to add custom projection. Once added, your projection has its own SRS ID and will be detected correctly.

Martin

[neteler@…]

Hi Martin,

I am again falling into this trap:

Warning: QgsSpatialRefSys::getRecord failed : select * from tbl_srs where parameters='+proj=utm +zone=32 +ellps=intl +towgs84=-87.000,-98.000,-121.000 +units=m +no_defs'

Displaying these two maps with on-the-fly projection activated, they are shifted by around 100m:

PROJCS["UTM Zone 32, Northern Hemisphere",GEOGCS["international",DATUM["D_European_1950",SPHEROID["International_1924",6378388,297],TOWGS84[-87.000,-98.000,-121.000]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTIONTransverse_Mercator,PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",9],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["Meter",1]]

PROJCS["Transverse Mercator",GEOGCS["international",DATUM["D_Monte_Mario",SPHEROID["international",6378388,297]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTIONTransverse_Mercator,PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",9],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",1500000],PARAMETER["false_northing",0],UNIT["Meter",1]]

In my opinion at least a visible warning should pop up that reading the .proj file failed.

thanks, Markus

[tutey@…]

Hi

Please consider Markus request. QGIS *silent* failure to assign a datum shift is really bad. If, instead, a warning would pop up with info that datum was not recognized and the user is asked to define the CRS manually himself to workaround it, this would be a great help.

I just had this issue too, working with archival topo maps of Germany, for which the 584.8,67.0,400.3,0.105,0.013,-2.378,10.29 datum shift from Bessel 1841 to WGS84 was not recognized by QGIS and it *silently* fell to no datum shift at all. This has resulted in an over 250m error. For an unexperienced user this might mean real trouble.

Maciek

[homann]

This can't be done unless we re-do the entire projections system, earliest at 0.9.

[neteler]

In current SVN trunk i(pre 1.0.0) t continues to fail:

qgis viapri.shp
Python support ENABLED :-)
Loading python plugins
Loaded : Plugin installer (package: plugin_installer)
Critical: QgsSpatialRefSys::createFromWkt -- theWkt is uninitialised, operation failed
Warning: QgsSpatialRefSys::getRecord failed :  select * from tbl_srs where parameters='+proj=tmerc +lat_0=0 +lon_0=9 +k                                               =0.9996 +x_0=1500000 +y_0=0 +ellps=intl +towgs84=-104.1,-49.1,-9.9,0.971,-2.917,0.714,-11.68 +units=m +no_defs'
Warning: QgsSpatialRefSys::findMatchingProj will only work if prj acr ellipsoid acr and proj4string are set!...

It works when I manually remove the TOWGS84 string from the WKT (.prj) file.

Suggestion: Ignore the TOWGS84 string when matching the projection, and issue a *warning* (not a complete failure) that TOWGS84 differs.

Note that most (in theory all) Italian QGIS users have this problem. Likewise for other countries.

thanks, Markus

[homann]

Is this error still present? Another towgs bug has been fixed, I believe.

[homann]

Talked to timlinux, and the idea with have is if the specfied projection does not have an exact match (including +towgs84= )in the database of projections, it will ask the user if he/she wants to create a custom projection.

Do you think that will solve the issue? This wont happen until 1.1.0 I'm afraid, needs GUI changes and has potential for large changes of projection code needed.

[neteler]

Replying to homann:

Talked to timlinux, and the idea with have is if the specfied projection does not have an exact match (including +towgs84= )in the database of projections, it will ask the user if he/she wants to create a custom projection.

Do you think that will solve the issue?

I think so. But:

This wont happen until 1.1.0 I'm afraid, needs GUI changes and has potential for large changes of projection code needed.

Understood. But as suggested above: Please issue at least a *warning* that TOWGS84 differs (if not happening already).

Markus

[homann]

I'm sorry, but the warning is still a GUI change. That towgs differs is just a special case of the fact that the projections differs. As soons as GUI freeze is over i'll try to fix this.

[pcav]

There is a problem in PROJ4: there are several datum for EPSG 3003 (and others), but in PROJ4 the epsg file does not allow a relation 1:n with present format (thanks Markus for clarifying this). Currently, users need to define their special SRS, which is cumbersome and scary for newbies. GRASS solved it recently: ​http://grass.osgeo.org/screenshots/gui.php#wxpython -> Location wizard (geodetic datum selection for EPSG 3003 - Italy mainland) Would it be feasible to copy the approach and port it to QGIS?

[homann]

Automatic addition of a custom CRS for unknown to QGIS - but valid - CRS:es from e.g WKT in r11366. If there are any *different* problems from Italian qgissers, please open a new ticket.