VSIFOpenL with access "w" does not overwrite file in zip archive

[Kyle Shannon]

If successive calls to VSIFOpenL are made with a "w" access flag, the file is not overwritten. I would expect more stdio like behavior, but I may be wrong:

#include "gdal_priv.h"

int main()
{
    VSILFILE *fin;
    fin = VSIFOpenL("vsi.cpp", "rb");
    vsi_l_offset offset;
    VSIFSeekL(fin, 0, SEEK_END);
    offset = VSIFTellL(fin);
    VSIRewindL(fin);
    char *data = (char*)CPLMalloc(offset * sizeof(char));
    VSIFReadL(data, offset, 1, fin);
    VSIFCloseL(fin);

    VSILFILE *fout;
    fout = VSIFOpenL("/vsizip/out.zip/vsi.cpp", "wb");
    VSIFWriteL(data, offset, 1, fout);
    VSIFCloseL(fout);

    fout = VSIFOpenL("/vsizip/out.zip/vsi.cpp", "wb");
    VSIFWriteL(data, offset, 1, fout);
    VSIFCloseL(fout);

    CPLFree(data);
    return 0;
}

and then list the contents of the archive:

kyle@kyle-workstation:~/Desktop$ g++ vsi.cpp -lgdal
kyle@kyle-workstation:~/Desktop$ ./a.out 
kyle@kyle-workstation:~/Desktop$ unzip -l out.zip 
Archive:  out.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
      609  1980-00-00 00:00   vsi.cpp
      609  1980-00-00 00:00   vsi.cpp
---------                     -------
     1218                     2 files

Also check plain "w" with the same results.

I may be missing something, I definitely have before.

[Even Rouault]

/vsizip/ in write mode has limited capabilities. It can append a new file to an existing zip, but cannot overwrite an existing file. There's no support for that in the minizip library, and I'm not sure this is technically feasable (and if it was possible, it could make holes in the zip if the new zipped file is bigger than the previous one)

Perhaps we should fail the VSIFOpenL() if we try to overwrite an existing file inside a ZIP ?

[Kyle Shannon]

Ah, I see. I am not sure what the best action would be, but I can see this happening with kmz files. I think an error might be appropriate with "w" and the file exists. Thanks.

[Even Rouault]

Could you elaborate more on the issue with KMZ files ?

[Kyle Shannon]

I shouldn't say that without looking at the kml driver(s). I haven't been successful in compiling libkml trunk for some time and not trying very hard. I will take a look. I don't know how writing datasets is handled.

[Kyle Shannon]

Even, I messed around with this a bit, and I found on unlikely issue, but an issue nonetheless:

kyle@kyle-workstation:4785$ ogr2ogr -f libkml vsi_test.kmz test.sqlite 
kyle@kyle-workstation:4785$ ogrinfo vsi_test.kmz 
INFO: Open of `vsi_test.kmz'
      using driver `LIBKML' successful.
1: test
kyle@kyle-workstation:4785$ unzip -l vsi_test.kmz
Archive:  vsi_test.kmz
  Length      Date    Time    Name
---------  ---------- -----   ----
      134  1980-00-00 00:00   doc.kml
     5486  1980-00-00 00:00   test.kml
       27  1980-00-00 00:00   style/style.kml
---------                     -------
     5647                     3 files
kyle@kyle-workstation:4785$ ogr2ogr -append vsi_test.kmz test.sqlite -nln doc
kyle@kyle-workstation:4785$ ogrinfo vsi_test.kmz 
INFO: Open of `vsi_test.kmz'
      using driver `LIBKML' successful.
1: test
2: doc
kyle@kyle-workstation:4785$ unzip -l vsi_test.kmz
Archive:  vsi_test.kmz
  Length      Date    Time    Name
---------  ---------- -----   ----
      227  1980-00-00 00:00   doc.kml
     5662  1980-00-00 00:00   test.kml
     5483  1980-00-00 00:00   doc.kml
       27  1980-00-00 00:00   style/style.kml
---------                     -------
    11399                     4 files
kyle@kyle-workstation:4785$ ogrinfo vsi_test.kmz doc
INFO: Open of `vsi_test.kmz'
      using driver `LIBKML' successful.

Layer name: doc
Geometry: Unknown (any)
Feature Count: 0
Layer SRS WKT:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        TOWGS84[0,0,0,0,0,0,0],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0,
        AUTHORITY["EPSG","8901"]],
    UNIT["degree",0.0174532925199433,
        AUTHORITY["EPSG","9108"]],
    AUTHORITY["EPSG","4326"]]
Name: String (0.0)
description: String (0.0)
timestamp: DateTime (0.0)
begin: DateTime (0.0)
end: DateTime (0.0)
altitudeMode: String (0.0)
tessellate: Integer (0.0)
extrude: Integer (0.0)
visibility: Integer (0.0)
drawOrder: Integer (0.0)
icon: String (0.0)

I also think that the libkml driver shouldn't allow a layer named 'doc' or handle that some how:

kyle@kyle-workstation:4785$ ogr2ogr -f libkml vsi_test.kmz test.sqlite -nln doc
kyle@kyle-workstation:4785$ unzip -l vsi_test.kmz
Archive:  vsi_test.kmz
  Length      Date    Time    Name
---------  ---------- -----   ----
      133  1980-00-00 00:00   doc.kml
     5483  1980-00-00 00:00   doc.kml
       27  1980-00-00 00:00   style/style.kml
---------                     -------
     5643                     3 files
kyle@kyle-workstation:4785$ ogrinfo vsi_test.kmz doc
INFO: Open of `vsi_test.kmz'
      using driver `LIBKML' successful.

Layer name: doc
Geometry: Unknown (any)
Feature Count: 0
Layer SRS WKT:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        TOWGS84[0,0,0,0,0,0,0],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0,
        AUTHORITY["EPSG","8901"]],
    UNIT["degree",0.0174532925199433,
        AUTHORITY["EPSG","9108"]],
    AUTHORITY["EPSG","4326"]]
Name: String (0.0)
description: String (0.0)
timestamp: DateTime (0.0)
begin: DateTime (0.0)
end: DateTime (0.0)
altitudeMode: String (0.0)
tessellate: Integer (0.0)
extrude: Integer (0.0)
visibility: Integer (0.0)
drawOrder: Integer (0.0)
icon: String (0.0)

It results in a bogus layer. Should I file a separate ticket for that and assign it to winkey?

[Even Rouault]

r25401 "Prevent CPLCreateFileInZip() from creating duplicated files in the same archive (#4785)"

r25402 "Prevent the LIBKML driver from creating a 'doc' layer (#4785)"