When writing a WMC, bounds should not be rounded to a predefined precision

[bbinet]

In OpenLayers.Format.WMC, writing the BoundingBox element is using toPrecision(10) to round the bounds to a 10 digit precision in the WMC generated file.

When loading this generated WMC, it can leads to a bad calculation of the map extent because the difference of the rounded bounds with the originals are forcing the Map to change to the next larger scale...

Looking at the WMC specs, it looks like a precision does not need to be forced:  http://schemas.opengis.net/context/1.1.0/context.xsd : Bounds of BoundingBoxType are xs:decimal which does not expect a precision:  http://www.w3.org/TR/xmlschema-2/#decimal

As I was looking at these precision issues, I also found out that the same precision was forced to ol:maxExtent wereas it's unneeded since it's a vendor param.

Lastly, the format is using a precision of 10 digits to write sld:MinScaleDenominator and sld:MaxScaleDenominator, which type are double according to the specs :  http://schemas.opengis.net/sld/1.0.0/StyledLayerDescriptor.xsd

Because it's a double, I recommend to use a 16 digits precision :  http://en.wikipedia.org/wiki/Double_precision

The attached patch is addressing this issue.

[bartvde]

Hi bbinet, thanks for the patch and the thorough explanation. I think this makes sense and the patch looks good (except for a typo), so I'll probably commit this soon.

[bbinet]

Thanks for reviewing bartvde.

patch_2242-r9631-A1.diff fixes the typo.

[bartvde]

patch also fixing up tests

[bartvde]

Bbinet, your patch killed some of the tests, so I've fixed that up (Trac preview also not working on my patch btw, so plz download). Another change I made was to use precision 18 for the decimals (the page you reference states "All ·minimally conforming· processors ·must· support decimal numbers with a minimum of 18 decimal digits"), is that enough precision for you? Otherwise the testcase where the following check is made (output WMC == input WMC) is hard to maintain.

Tim, are you able to make a final review before we put this in? TIA.

[bartvde]

Forgot to mention, relevant tests pass in FF3.5

[bbinet]

I've tested your new patch with precision 18 for the decimals and it works fine for me. Thanks also for fixing tests.

[bartvde]

(In [9703]) When writing a WMC, bounds should not be rounded to a predefined precision, original patch by bbinet, r=me (closes #2242)