Ticket #1423 (closed defect: fixed)

Opened 3 years ago

Last modified 2 years ago

Linux std::string not thread safe

Reported by: trevorwekel Owned by: trevorwekel
Priority: high Milestone: 2.2
Component: Server Version: 2.2.0
Severity: blocker Keywords:
Cc: brucedechant External ID:

Description

The std::string implementation on Linux uses a refcounted copy of the string data. Under certain circumstances, the implementation in not thread safe which causes "double free" errors and MapGuide server crashes under heavy load on Linux.

The MapGuide logs (access, error, etc) are especially susceptible to string thread safety because strings are queued to seperate thread before they are written to disk.

An effective way to test thread safety for std::string is to run a load test of 200+ concurrent users executing Http GETTILEIMAGE. This Http operation is very fast (<10 ms) and puts huge stress on the basic request processing and logging infrastructure. For maximum effectiveness, this test should be run on an 8 way box.

Attachments

stdstringCommon.patch Download (9.0 KB) - added by trevorwekel 3 years ago.
stdstringWeb.patch Download (1.2 KB) - added by trevorwekel 3 years ago.
stdstringServer.patch Download (30.8 KB) - added by trevorwekel 3 years ago.

Change History

Changed 3 years ago by trevorwekel

With MapGuide 2.2 RC0, the 200 user GETTILEIMAGE test will typically throw a "double free" error within the first 5 minutes of the test run on Linux. A fatal crash can be expected within the first hour.

Changed 3 years ago by trevorwekel

Changed 3 years ago by trevorwekel

Changed 3 years ago by trevorwekel

Changed 3 years ago by trevorwekel

Added patch files for std::string safety and other potential thread safety issues detected by Valgrind on Linux:

  • Return a STRING type created from a c_str() call forces the std::string template to allocate a new copy of the string.
  • Replace CREFSTRING with STRING return values for consistency. Additional c_str() calls may be required depending threading usage.
  • Add refcount to MgUserInformation::GetCurrentUserInfo() to guarantee object lifetime in a multi-threaded environment.
  • Null out various "Packet" variables to correct uninitialized references detected by Valgrind

Changed 3 years ago by trevorwekel

  • status changed from new to assigned

On a 200 user GETTILEIMAGE test, the patches above did not generate a single "double free" error or crash during a two hour run. The server access.log grew to over 3GB in size and contained over 24 million log entries.

Changed 3 years ago by brucedechant

  • cc brucedechant added

Changed 3 years ago by trevorwekel

Fixes for to branches/2.2 submitted in r5179, r5180, r5181

Changed 3 years ago by trevorwekel

  • status changed from assigned to closed
  • resolution set to fixed

Fixes for trunk submitted in r5183

Changed 2 years ago by hubu

Fixes for trunk submitted in r5780

Changed 2 years ago by brucedechant

Fixed for sandbox/adsk/2.2gp r5829

Changed 2 years ago by brucedechant

Fixed for sandbox/adsk/2.3r r5731, r5734

Note: See TracTickets for help on using tickets.