Load balancing doesn't support failover

[brucedechant]

The load balancing algorithm used in the web tier doesn't remove server's that are not responding.

Example: In a 2 server site this shows up as a sequence of success/fail/success/fail/... operations because the load balancing algorithm is round robining between all of the servers in the list even "bad" ones.

If a "bad" server is encountered it needs to be removed from the load balancing list of servers and periodically checked and added back in if it becomes "good" again.

[brucedechant]

Fixed in sandbox/adsk/2.1 r4323

[brucedechant]

• Update failover retry time when a site contains only a single server
• Update MgUserInformation as the GetSiteStatus doesn't require authentication

Fixed in sandbox/adsk/2.1 r4329

[brucedechant]

Fixed in trunk r4403

[brucedechant]

Added NULL pointer check.Trunk r4440

[brucedechant]

Added NULL pointer check. sandbox/adsk/2.1 r4441

[christinebao]

After adding NULL pointer check, server unit test won't crash because of calling NULL pointer, but it still has 4 test cases fails.

This is because of code:

MgConnectionProperties* MgSiteManager::GetConnectionProperties(
    MgUserInformation* userInfo, MgSiteInfo::MgPortType portType, bool useSessionIp)
{
    …

        if (length > MgSiteInfo::HexStringLength)
        {
            STRING siteHexString = sessionId.substr(
                length - MgSiteInfo::HexStringLength, MgSiteInfo::HexStringLength);
            Ptr<MgSiteInfo> siteInfo = GetSiteInfo(siteHexString);

            if ((NULL != siteInfo.p) && (MgSiteInfo::Ok == siteInfo->GetStatus()))
            {
                connProps = GetConnectionProperties(userInfo, siteInfo, portType);
            }
            else
            {
                // This site is not currently working

                // We have a session, but it will not exist on any other machine so we force the session exception
                throw new MgSessionExpiredException(L"MgSiteManager.GetConnectionProperties",__LINE__,__WFILE__, NULL, L"", NULL);
            }
        }

siteInfo from GetSiteInfo(siteHexString) is NULL, and an exception throw out saying session expired.

Compared with the code before:

            Ptr<MgSiteInfo> siteInfo = new MgSiteInfo(siteHexString);

            if (MgSiteInfo::Ok == siteInfo->GetStatus())
            {
                connProps = GetConnectionProperties(userInfo, siteInfo, portType);
            }

Personally I think in the GetSiteInfo(…) function, if matchingSiteInfo is not found, we can new an instance of it. The code could be:

MgSiteInfo* MgSiteManager::GetSiteInfo(CREFSTRING hexString)
{
    MgSiteInfo* matchingSiteInfo = NULL;

    …

    if (matchingSiteInfo == NULL)
        matchingSiteInfo = new MgSiteInfo(hexString);

    return SAFE_ADDREF(matchingSiteInfo);
}

I tested it in unit test, and all test cases pass.

[christinebao]

Attach patch ​http://trac.osgeo.org/mapguide/attachment/ticket/1131/GetSiteInfo.patch for fixing server unit test.

Hi Bruce, would you please check this patch when you come back:

1. Is it a right fixing which will not break your submission?
   
2. There is another override MgSiteInfo* MgSiteManager::GetSiteInfo(CREFSTRING target, INT32 port). Shall we handle it in the same way?

I submit the patch to avoid server unit test fail in the following build. It's appreciated if you can review it and correct me if anything wrong.

Thanks & regards,
Christine

[waltweltonlair]

Fixed a ref-count bug in Christine's submission.

[christinebao]

These three KmlService test cases will throw out MgConnectionFailedException and make the site (typically 127.0.0.1) UnableToConnect. Then the GetNextSite() will fail.
As these three test cases actually do nothing, comment them temporarily to make server unit test pass. They will be considered later when Bruce reviews this defect.

[brucedechant]

Fixed failed KML unit tests. See submission r4488.

[brucedechant]

Apply updates to sandbox/adsk/2.1 r4489.

[brucedechant]

• Updated background check server(s) thread to shutdown faster
• Disabled starting the background check server(s) thread if we are running unit tests

sandbox/adsk/2.1 r4491

sandbox/adsk/2.2gp r4492

trunk r4493

[brucedechant]

Fixed failed KML unit tests.

See submission trunk r4495.

See submission sandbox/adsk/2.1 r4497.

[brucedechant]

• Turn on fail over retry background thread by default and disable it when running unit tests
• Adjust fail over retry time for web tier to correspond to a single MapGuide server instead of a multiple MapGuide server configuration

See submission sandbox/adsk/2.2gp r4574

See submission trunk r4575