WMS ScaleHint precision problems

[bartvde@…]

Follow up to this e-mail, it seems the vprintf for %g has a default of 6 digits,
so I think we need to specify a better precision than 6:

       g      The double argument is converted in style  f  or  e
              (or  E for G conversions).  The precision specifies
              the number of significant digits.  If the precision
              is missing, 6 digits are given; if the precision is
              zero, it is treated as 1.  Style e is used  if  the
              exponent  from  its  conversion  is less than -4 or
              greater than or equal to the  precision.   Trailing
              zeros  are  removed from the fractional part of the
              result; a decimal point appears only if it is  fol-
              lowed by at least one digit.

Hi list,

I am running into ScaleHint precision issues, at least it is my guess this has
to do with precision. It seems to me the values outputted to the WMS
Capabilities don't have enough precision.

My MAP file has 3 layers:

layer1: MINSCALE=250,000 MAXSCALE=5,000,000
layer2: MINSCALE=75,000 MAXSCALE=250,000
layer3: MINSCALE=0 MAXSCALE=75,000

The ScaleHint values are:

layer1: <ScaleHint min="124.726" max="2494.51" />
layer2: <ScaleHint min="37.4177" max="124.726" />
layer3: <ScaleHint min="0" max="37.4177" />

If I do MAXSCALE for layer1 on my calculator I get:
2494,5142421481852288355319843683

My application translates this back to scale values using (taken from wmsparse.c
which is part of Chameleon):

#define INCHES_PER_METER 39.3701

#ifndef M_SQRT2
# define M_SQRT2        1.41421356237309504880  /* sqrt(2) */
#endif

double scaleHintToScaleDenominator( double dScaleHint, double dResolution)
{
    if (dScaleHint <= 0)
        return -1.0;

    if (dResolution <= 0)
        dResolution = 72.0;

    return (dScaleHint * INCHES_PER_METER * dResolution / M_SQRT2);
}

The result is:
layer1: MINSCALE=250000,57705, MAXSCALE=4999991,49705
layer2: MINSCALE=74999,97268,MAXSCALE=250000,57705
layer3: MINSCALE=-1,MAXSCALE=74999,97268

As you can see, especially for layer 1 the results are (way) off.

Where is the problem situated? Can it be fixed by having more precision in the
WMS GetCapabilities for scalehint values?

Thanks in advance.

Best regards,
Bart

[dmorissette]

I think we'll use %.15g like we use in many other places that output floating
point values. That should resolve your problem.

[dmorissette]

Fixed in 4.9 dev only. 

Do you need this backported to 4.8? That's a straightforward fix, but we want to
avoid backporting stuff unless really necessary.

[bartvde@…]

Daniel, backporting to 4.8.X would really help us a lot, thanks in advance. We
are not allowed to use development versions.

[dmorissette]

I have backported the fix, it will be in 4.8.2