Problem using locale with WKTReader

[del]

Hello, I spotted the following problem (on at least ubuntu and gentoo):

When LANG=fr_FR.utf8 is set... WKTReader fails reading points.

Giving "POINT (2154.7500000000000000 321.5380000000000109)" to WKTReader leads to the following error: Unable to handle the packet: ParseException?: Expected number but encountered word: '2154.7500000000000000'.

I checked in the source code... and WKTReader uses the strtod(...) function in io/StringTokenizer.cpp:76 which uses the locale to decide what looks like a number... or in French, we use "," instead of "." for the radix character.

This is kind of a problem for an application which cannot "force" the locale to be the standard C one...

[sgillies]

I believe this is a duplicate of #201. Fixed in 3.1.

[yabo]

Hello,

I believe the problem isn't totally fixed yet. I'm using geos 3.1, here is a minimal test case that englithens the bug :

#include <locale.h>
#include <iostream>

#include <geos/geom/Point.h>
#include <geos/io/WKTReader.h>
#include <geos/io/WKTWriter.h>

int main(int argc, char* argv[])
{
  setlocale(LC_ALL, argv[2]);
  geos::io::WKTReader r;
  geos::geom::Point* p = (geos::geom::Point*) r.read(argv[1]);

  geos::io::WKTWriter w;
  std::cout << w.write(p) << std::endl;
}

yabo@xen ~/dev/geos_test_1 $ ./a.out  'POINT(42.42 0)' 'C'
POINT (42.4200000000000017 0.0000000000000000)
yabo@xen ~/dev/geos_test_1 $ ./a.out  'POINT(42.42 0)' 'en_EN.UTF8'
POINT (42.4200000000000017 0.0000000000000000)
yabo@xen ~/dev/geos_test_1 $ ./a.out  'POINT(42.42 0)' 'fr_FR.UTF8'
terminate called after throwing an instance of 'geos::io::ParseException'
  what():  ParseException: Expected number but encountered word: '42.42'
Aborted
yabo@xen ~/dev/geos_test_1 $ 

I'm using Gentoo linux on a 64bit platform with geos 3.1.

yabo@xen ~/dev/geos_test_1 $ uname -ap
Linux xen 2.6.28-gentoo-r5 #8 SMP PREEMPT Mon May 18 11:55:05 CEST 2009 x86_64 AMD Phenom(tm) II X4 810 Processor AuthenticAMD GNU/Linux
yabo@xen ~/dev/geos_test_1 $ emerge -pv geos  

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R   ] sci-libs/geos-3.1.0  USE="python -doc -ruby" 0 kB

Total: 1 package (1 reinstall), Size of downloads: 0 kB
yabo@xen ~/dev/geos_test_1 $ 

After inspecting the code, the point of failure is, as said in the first report, in io/StringTokenizer.cpp:76 where the strtod function is used, which is a locale-dependant function.

[strk]

Yeah, the fix for #201 was just when using the C-API. I'll take care of this, trunk is likely also affected.

[strk]

Fixed in trunk as r2576. To be back-ported. Tests welcome.

[strk]

I've also dropped explicit locale-handling from the C-API as it should be not required now (done at lower level)

[yabo]

Thanks for that quick response.

The same test code as above with the trunk-geos :

yabo@xen ~/dev/geos_test_1 $ LD_LIBRARY_PATH=../geos/trunk/_build/prefix/lib ./a.out 'POINT(42.42 0)' 'fr_FR.UTF8'
POINT (42,4200000000000017 0,0000000000000000)

Obvisouly it worked too well :)

[strk]

Alright, Writer fixed too with r2578.

[yabo]

Thank you. I have a question though regarding the solution though. As I understand it, the CLocalizer object changes the locale of the program while the function is executed. What about threads ? If I have multiple threads and that a locale-sensitive operation happens in one thread while another one performs a read() or write() with Geos in may lead to very painful bugs to find, and would require some mutex-ing to solve (which is quite heavy).

I don't re-open the bug as this one is solved however I wanted to spot this potential quirk.

Agin, thanks alot for the super reactivity :).

[strk]

GEOS is known to not be thread-safe. There are many other problems with threads (statics used)

[sgillies]

GEOS does have a thread context object now. Perhaps locale state could be kept there.

[strk]

That's just for the C-API and still doesn't prevent another thread, with another state to switch back to non-C locale on destruction.