Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#6874 closed enhancement (fixed)

Improve performance of Oracle OGR driver

Reported by: avautour Owned by: warmerdam
Priority: normal Milestone: 2.3.0
Component: OGR_SF Version: 2.1.3
Severity: normal Keywords:
Cc:

Description

Opening an OGR data source without specifying the tables to open was extremely slow with our in house server.

I did some performance profiling and found that most of the time was spent in OGROCIDataSource::OpenTable(). More specifically, the time was spent waiting on the database queries in OGROCITableLayer::ReadTableDefinition() and OGROCILayer::LookupTableSRID().

I decided to trace an Oracle session to see why the queries were taking so long, I discovered that most of the time was spent in the parse stage of the query, not in the execution.

Oracle has a shared pool to cache to execution plans of queries, but it usually gets hit if the queries are exactly the same.

This improvement is to update the functions above to use bind variables so that the queries can be parsed for the first table, and just executed for the following tables, resulting in much better performance.

Attachments (3)

oracle_binds.patch (8.5 KB ) - added by avautour 7 years ago.
Proposed changes
initial_parse.txt (8.1 KB ) - added by avautour 7 years ago.
Oracle trace file of the initial parse when the shared pool was empty
subsequent_executions.txt (8.1 KB ) - added by avautour 7 years ago.
Oracle trace file when the parse stage hit the shared pool

Download all attachments as: .zip

Change History (7)

by avautour, 7 years ago

Attachment: oracle_binds.patch added

Proposed changes

by avautour, 7 years ago

Attachment: initial_parse.txt added

Oracle trace file of the initial parse when the shared pool was empty

by avautour, 7 years ago

Attachment: subsequent_executions.txt added

Oracle trace file when the parse stage hit the shared pool

comment:1 by Even Rouault, 7 years ago

Resolution: fixed
Status: newclosed

In 38150:

OCI: use prepared statements to speed up opening (fixes #6874, patch by Andre Vautour)

comment:2 by Even Rouault, 7 years ago

I've comitted the patch in trunk. Not there was a spurious ; in the below string that removed the where clause. I've fixed that

        352      const char* pszDimCmd =
 	353	            "SELECT COUNT(*)\n"
 	354	            "FROM ALL_SDO_GEOM_METADATA u, TABLE(u.diminfo) t\n";
 	355	            "WHERE u.table_name = :table_name\n"
 	356	            "  AND u.column_name = :geometry_name";

comment:3 by Even Rouault, 7 years ago

Milestone: 2.3.0

comment:4 by avautour, 7 years ago

Interesting, I'm not sure how that got in there. Regardless, thanks Even.

Note: See TracTickets for help on using tickets.