#260 closed task (fixed)
Problem using locale with WKTReader
Reported by: | del | Owned by: | strk |
---|---|---|---|
Priority: | major | Milestone: | 3.1.1 |
Component: | Default | Version: | 3.1.0 |
Severity: | Annoyance | Keywords: | |
Cc: |
Description
Hello, I spotted the following problem (on at least ubuntu and gentoo):
When LANG=fr_FR.utf8 is set... WKTReader fails reading points.
Giving "POINT (2154.7500000000000000 321.5380000000000109)" to WKTReader leads to the following error: Unable to handle the packet: ParseException: Expected number but encountered word: '2154.7500000000000000'.
I checked in the source code... and WKTReader uses the strtod(...) function in io/StringTokenizer.cpp:76 which uses the locale to decide what looks like a number... or in French, we use "," instead of "." for the radix character.
This is kind of a problem for an application which cannot "force" the locale to be the standard C one...
Change History (14)
comment:1 by , 15 years ago
Resolution: | → duplicate |
---|---|
Status: | new → closed |
comment:2 by , 15 years ago
Resolution: | duplicate |
---|---|
Status: | closed → reopened |
Version: | 3.0.3 → 3.1.0 |
Hello,
I believe the problem isn't totally fixed yet. I'm using geos 3.1, here is a minimal test case that englithens the bug :
#include <locale.h> #include <iostream> #include <geos/geom/Point.h> #include <geos/io/WKTReader.h> #include <geos/io/WKTWriter.h> int main(int argc, char* argv[]) { setlocale(LC_ALL, argv[2]); geos::io::WKTReader r; geos::geom::Point* p = (geos::geom::Point*) r.read(argv[1]); geos::io::WKTWriter w; std::cout << w.write(p) << std::endl; }
yabo@xen ~/dev/geos_test_1 $ ./a.out 'POINT(42.42 0)' 'C' POINT (42.4200000000000017 0.0000000000000000) yabo@xen ~/dev/geos_test_1 $ ./a.out 'POINT(42.42 0)' 'en_EN.UTF8' POINT (42.4200000000000017 0.0000000000000000) yabo@xen ~/dev/geos_test_1 $ ./a.out 'POINT(42.42 0)' 'fr_FR.UTF8' terminate called after throwing an instance of 'geos::io::ParseException' what(): ParseException: Expected number but encountered word: '42.42' Aborted yabo@xen ~/dev/geos_test_1 $
I'm using Gentoo linux on a 64bit platform with geos 3.1.
yabo@xen ~/dev/geos_test_1 $ uname -ap Linux xen 2.6.28-gentoo-r5 #8 SMP PREEMPT Mon May 18 11:55:05 CEST 2009 x86_64 AMD Phenom(tm) II X4 810 Processor AuthenticAMD GNU/Linux yabo@xen ~/dev/geos_test_1 $ emerge -pv geos These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild R ] sci-libs/geos-3.1.0 USE="python -doc -ruby" 0 kB Total: 1 package (1 reinstall), Size of downloads: 0 kB yabo@xen ~/dev/geos_test_1 $
After inspecting the code, the point of failure is, as said in the first report, in io/StringTokenizer.cpp:76 where the strtod function is used, which is a locale-dependant function.
comment:3 by , 15 years ago
Owner: | changed from | to
---|---|
Status: | reopened → new |
Yeah, the fix for #201 was just when using the C-API. I'll take care of this, trunk is likely also affected.
comment:4 by , 15 years ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
Fixed in trunk as r2576. To be back-ported. Tests welcome.
comment:5 by , 15 years ago
I've also dropped explicit locale-handling from the C-API as it should be not required now (done at lower level)
comment:6 by , 15 years ago
Resolution: | fixed |
---|---|
Status: | closed → reopened |
Thanks for that quick response.
The same test code as above with the trunk-geos :
yabo@xen ~/dev/geos_test_1 $ LD_LIBRARY_PATH=../geos/trunk/_build/prefix/lib ./a.out 'POINT(42.42 0)' 'fr_FR.UTF8' POINT (42,4200000000000017 0,0000000000000000)
Obvisouly it worked too well :)
comment:7 by , 15 years ago
Resolution: | → fixed |
---|---|
Status: | reopened → closed |
Alright, Writer fixed too with r2578.
comment:8 by , 15 years ago
Thank you. I have a question though regarding the solution though. As I understand it, the CLocalizer object changes the locale of the program while the function is executed. What about threads ? If I have multiple threads and that a locale-sensitive operation happens in one thread while another one performs a read() or write() with Geos in may lead to very painful bugs to find, and would require some mutex-ing to solve (which is quite heavy).
I don't re-open the bug as this one is solved however I wanted to spot this potential quirk.
Agin, thanks alot for the super reactivity :).
comment:9 by , 15 years ago
GEOS is known to not be thread-safe. There are many other problems with threads (statics used)
comment:10 by , 15 years ago
GEOS does have a thread context object now. Perhaps locale state could be kept there.
comment:11 by , 15 years ago
That's just for the C-API and still doesn't prevent another thread, with another state to switch back to non-C locale on destruction.
I believe this is a duplicate of #201. Fixed in 3.1.