Opened 9 years ago

Closed 9 years ago

#3656 closed defect (fixed)

MULTI-LOAD is not available in -append -update mode - OCI

Reported by: msmitherdc Owned by: ilucena
Priority: normal Milestone:
Component: OGR_SF Version: svn-trunk
Severity: normal Keywords: oci append multi-load ogr
Cc:

Description

In the OCI driver when appending data in update more, single row inserts with a commit after each statement is done. Should be some options to use binds and to only commit on completion.

Change History (11)

comment:1 Changed 9 years ago by ilucena

From http://gdal.org/ogr/drv_oci.html:

  • MULTI_LOAD: If enabled new features will be created in groups of 100 per SQL INSERT command, instead of each feature being a separate INSERT command. Having this enabled is the fastest way to load data quickly. Multi-load mode is enabled by default for newly created layers, and may be forced on for existing layers by setting this option to YES, or disabled even for new layers by setting to NO

comment:2 Changed 9 years ago by ilucena

Resolution: fixed
Status: newclosed

comment:3 in reply to:  2 Changed 9 years ago by msmitherdc

Replying to ilucena:

Ivan, Layer Creation Options are ignored in update mode. So there is no way to enable MULTI_LOAD.

u4rt9mds@maps:/htdocs/dhs/marine$ ~/programming/MapServer/gdal_trunk/apps/ogr2ogr -append -update -skipfailures -f OCI OCI:user/pass@tns:MARINE_TEST2 MarineTraffic?.kmz -nln marine_test2 -lco MULTI_LOAD=YES WARNING: Layer creation options ignored since an existing layer is

being appended to.

comment:4 Changed 9 years ago by ilucena

Resolution: fixed
Status: closedreopened

Okay, I see what you mean.

comment:5 Changed 9 years ago by ilucena

We would need to find another way to make it work. As a create-option ogr2ogr will always block it uses with -append. That is ogr2ogr who is giving the message "Layer creation options ignored since an existing layer is being appended to.". The OCI driver or any other driver doesn't get the MULTI_LOAD or any other create-option when the layer already exist.

comment:6 Changed 9 years ago by msmitherdc

I see a couple of possible options here. The first and simplest is that MULTI-LOAD becomes the default for update mode. I'm trying to think of any reason you'd ever want single row inserts. Its certainly a much smaller number of instances. The second is a much larger change in changing the behavior of ogr2ogr to either allow lco's at update time or provide lmo's (Layer Modification Options).

I think the first option is the best for right now but longer term I wonder if something like #2 is needed.

comment:7 Changed 9 years ago by msmitherdc

Another option that I thought of is setting an OGR environment variable. Perhaps those are available regardless of creation or append?

comment:8 Changed 9 years ago by ilucena

Mike,

Your first idea sounds good to me. I mean, if it is an update then run in MULTI-LOAD mode.

Frank,

Should I proceed and change the code?

comment:9 Changed 9 years ago by warmerdam

Ivan,

I think the best approach is to use multi-load by default. I would note that GDALDataset has a Flush() method and after a flush any partial multi-load should be committed.

If you don't want to do it by default, you could use a CPLConfigOption() to control it (in addition/alternative to -lco). But, by no means will we alter ogr2ogr to use -lco in cases where it is not creating the layer.

ok?

comment:10 Changed 9 years ago by warmerdam

Component: defaultGDAL_Raster

comment:11 Changed 9 years ago by ilucena

Component: GDAL_RasterOGR_SF
Resolution: fixed
Status: reopenedclosed

Frank,

It is done. r20601

Html doc is also updated.

I am changing this ticket Component to OGR_SF, if you don't mind.

Note: See TracTickets for help on using tickets.