Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#758 closed defect (fixed)

Crash when reading SNAPPY compressed Parquet files through GDAL

Reported by: rouault Owned by: osgeo4w-dev@…
Priority: major Component: Package
Version: Keywords:
Cc:

Description

Cf ​https://github.com/OSGeo/gdal/issues/6115

Change History (10)

comment:1 by jef, 2 years ago

Not reproducable here. Are you seing invalid instructions in eventlog? conda is using ARROW_SSE42=OFF (which I can't find in source - ARROW_SIMD_LEVEL seems related, but unlike tiledb it apperantly checks for support at runtime and is even configurable at runtime via ARROW_USER_SIMD_LEVEL). 

C:\OSGeo4W>type run.bat 
type run.bat
set CPL_DEBUG=ON
ogrinfo -so test.parquet test
for %%c in (SNAPPY GZIP ZSTD BROTLI) do (
  if exist out_%%c.parquet del out_%%c.parquet
  ogr2ogr -lco COMPRESSION=%%c out_%%c.parquet test.parquet
  ogrinfo -so out_%%c.parquet out_%%c
)
dir *.parquet
C:\OSGeo4W>set CPL_DEBUG=ON 

C:\OSGeo4W>ogrinfo -so test.parquet test 
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_ECW_JP2ECW.dll using GDALRegister_ECW_JP2ECW.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_GEOR.dll using GDALRegister_GEOR.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_HDF5.dll using GDALRegister_HDF5.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_MrSID.dll using GDALRegister_MrSID.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_OCI.dll using RegisterOGROCI.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_SOSI.dll using RegisterOGRSOSI.
PARQUET: geo = {"version": "0.1.0", "primary_column": "geometry", "columns": {"geometry": {"crs": "GEOGCRS[\"WGS 84\",ENSEMBLE[\"World Geodetic System 1984 ensemble\",MEMBER[\"World Geodetic System 1984 (Transit)\"],MEMBER[\"World Geodetic System 1984 (G730)\"],MEMBER[\"World Geodetic System 1984 (G873)\"],MEMBER[\"World Geodetic System 1984 (G1150)\"],MEMBER[\"World Geodetic System 1984 (G1674)\"],MEMBER[\"World Geodetic System 1984 (G1762)\"],MEMBER[\"World Geodetic System 1984 (G2139)\"],ELLIPSOID[\"WGS 84\",6378137,298.257223563],ENSEMBLEACCURACY[2.0]],CS[ellipsoidal,2],AXIS[\"geodetic latitude (Lat)\",north],AXIS[\"geodetic longitude (Lon)\",east],UNIT[\"degree\",0.0174532925199433],USAGE[SCOPE[\"Horizontal component of 3D system.\"],AREA[\"World.\"],BBOX[-90,-180,90,180]],ID[\"EPSG\",4326]]", "bbox": [0, 2, 4, 2], "encoding": "WKB"}}}
GDAL: GDALOpen(test.parquet, this=000001C7F5CB5910) succeeds as Parquet.
ARROW: Memory pool: bytes_allocated = 0
ARROW: Memory pool: max_memory = 640
GDAL: GDALClose(test.parquet, this=000001C7F5CB5910)
GDAL: In GDALDestroy - unloading GDAL shared library.
INFO: Open of `test.parquet'
      using driver `Parquet' successful.

Layer name: test
Geometry: Point
Feature Count: 5
Extent: (0.000000, 2.000000) - (4.000000, 2.000000)
Layer SRS WKT:
GEOGCRS["WGS 84",
    ENSEMBLE["World Geodetic System 1984 ensemble",
        MEMBER["World Geodetic System 1984 (Transit)"],
        MEMBER["World Geodetic System 1984 (G730)"],
        MEMBER["World Geodetic System 1984 (G873)"],
        MEMBER["World Geodetic System 1984 (G1150)"],
        MEMBER["World Geodetic System 1984 (G1674)"],
        MEMBER["World Geodetic System 1984 (G1762)"],
        MEMBER["World Geodetic System 1984 (G2139)"],
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]],
        ENSEMBLEACCURACY[2.0]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    USAGE[
        SCOPE["Horizontal component of 3D system."],
        AREA["World."],
        BBOX[-90,-180,90,180]],
    ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
Geometry Column = geometry
boolean: Integer(Boolean) (0.0)
uint8: Integer (0.0)
int8: Integer (0.0)
uint16: Integer (0.0)
int16: Integer(Int16) (0.0)
uint32: Integer64 (0.0)
int32: Integer (0.0)
uint64: Real (0.0)
int64: Integer64 (0.0)
float32: Real(Float32) (0.0)
float64: Real (0.0)
string: String (0.0)
large_string: String (0.0)
timestamp_ms_gmt: DateTime (0.0)
timestamp_ms_gmt_plus_2: DateTime (0.0)
timestamp_ms_gmt_minus_0215: DateTime (0.0)
timestamp_s_no_tz: DateTime (0.0)
time32_s: Time (0.0)
time32_ms: Time (0.0)
time64_us: Integer64 (0.0)
time64_ns: Integer64 (0.0)
date32: Date (0.0)
date64: Date (0.0)
binary: Binary (0.0)
large_binary: Binary (0.0)
fixed_size_binary: Binary (2.0)
decimal128: Real (7.3)
decimal256: Real (7.3)
list_boolean: IntegerList(Boolean) (0.0)
list_uint8: IntegerList (0.0)
list_int8: IntegerList (0.0)
list_uint16: IntegerList (0.0)
list_int16: IntegerList (0.0)
list_uint32: Integer64List (0.0)
list_int32: IntegerList (0.0)
list_uint64: RealList (0.0)
list_int64: Integer64List (0.0)
list_float32: RealList(Float32) (0.0)
list_float64: RealList (0.0)
list_string: StringList (0.0)
fixed_size_list_boolean: IntegerList(Boolean) (0.0)
fixed_size_list_uint8: IntegerList (0.0)
fixed_size_list_int8: IntegerList (0.0)
fixed_size_list_uint16: IntegerList (0.0)
fixed_size_list_int16: IntegerList (0.0)
fixed_size_list_uint32: Integer64List (0.0)
fixed_size_list_int32: IntegerList (0.0)
fixed_size_list_uint64: RealList (0.0)
fixed_size_list_int64: Integer64List (0.0)
fixed_size_list_float32: RealList(Float32) (0.0)
fixed_size_list_float64: RealList (0.0)
fixed_size_list_string: StringList (0.0)
struct_field.a: Integer64 (0.0)
struct_field.b: Real (0.0)
struct_field.c.d: String (0.0)
struct_field.c.f: String (0.0)
struct_field.h: Integer64List (0.0)
struct_field.i: Integer64 (0.0)
map_boolean: String(JSON) (0.0)
map_uint8: String(JSON) (0.0)
map_int8: String(JSON) (0.0)
map_uint16: String(JSON) (0.0)
map_int16: String(JSON) (0.0)
map_uint32: String(JSON) (0.0)
map_int32: String(JSON) (0.0)
map_uint64: String(JSON) (0.0)
map_int64: String(JSON) (0.0)
map_float32: String(JSON) (0.0)
map_float64: String(JSON) (0.0)
map_string: String(JSON) (0.0)
dict: Integer (0.0), domain name=dictDomain

C:\OSGeo4W>for %c in (SNAPPY GZIP ZSTD BROTLI) do (
if exist out_%c.parquet del out_%c.parquet  
 ogr2ogr -lco COMPRESSION=%c out_%c.parquet test.parquet  
 ogrinfo -so out_%c.parquet out_%c 
) 

C:\OSGeo4W>(
if exist out_SNAPPY.parquet del out_SNAPPY.parquet  
 ogr2ogr -lco COMPRESSION=SNAPPY out_SNAPPY.parquet test.parquet  
 ogrinfo -so out_SNAPPY.parquet out_SNAPPY 
) 
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_ECW_JP2ECW.dll using GDALRegister_ECW_JP2ECW.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_GEOR.dll using GDALRegister_GEOR.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_HDF5.dll using GDALRegister_HDF5.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_MrSID.dll using GDALRegister_MrSID.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_OCI.dll using RegisterOGROCI.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_SOSI.dll using RegisterOGRSOSI.
PARQUET: geo = {"version": "0.1.0", "primary_column": "geometry", "columns": {"geometry": {"crs": "GEOGCRS[\"WGS 84\",ENSEMBLE[\"World Geodetic System 1984 ensemble\",MEMBER[\"World Geodetic System 1984 (Transit)\"],MEMBER[\"World Geodetic System 1984 (G730)\"],MEMBER[\"World Geodetic System 1984 (G873)\"],MEMBER[\"World Geodetic System 1984 (G1150)\"],MEMBER[\"World Geodetic System 1984 (G1674)\"],MEMBER[\"World Geodetic System 1984 (G1762)\"],MEMBER[\"World Geodetic System 1984 (G2139)\"],ELLIPSOID[\"WGS 84\",6378137,298.257223563],ENSEMBLEACCURACY[2.0]],CS[ellipsoidal,2],AXIS[\"geodetic latitude (Lat)\",north],AXIS[\"geodetic longitude (Lon)\",east],UNIT[\"degree\",0.0174532925199433],USAGE[SCOPE[\"Horizontal component of 3D system.\"],AREA[\"World.\"],BBOX[-90,-180,90,180]],ID[\"EPSG\",4326]]", "bbox": [0, 2, 4, 2], "encoding": "WKB"}}}
GDAL: GDALOpen(test.parquet, this=0000016F8C2664B0) succeeds as Parquet.
GDAL: Using Parquet driver
GDAL: GDALDriver::Create(Parquet,out_SNAPPY.parquet,0,0,0,Unknown,0000000000000000)
GDALVectorTranslate: 5 features written in layer 'test'
ARROW: Memory pool: bytes_allocated = 14144
ARROW: Memory pool: max_memory = 35584
GDAL: GDALClose(test.parquet, this=0000016F8C2664B0)
ARROW: Memory pool (writer layer): bytes_allocated = 128
ARROW: Memory pool (writer layer): max_memory = 36160
GDAL: GDALClose(out_SNAPPY.parquet, this=0000016F8D27CF90)
GDAL: In GDALDestroy - unloading GDAL shared library.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_ECW_JP2ECW.dll using GDALRegister_ECW_JP2ECW.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_GEOR.dll using GDALRegister_GEOR.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_HDF5.dll using GDALRegister_HDF5.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_MrSID.dll using GDALRegister_MrSID.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_OCI.dll using RegisterOGROCI.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_SOSI.dll using RegisterOGRSOSI.
PARQUET: geo = {"version":"0.4.0","primary_column":"geometry","columns":{"geometry":{"encoding":"WKB","bbox":[0.0,2.0,4.0,2.0],"orientation":"counterclockwise","geometry_type":"Point"}}}
PARQUET: gdal:schema = {"columns":{"boolean":{"type":"Integer","subtype":"Boolean"},"uint8":{"type":"Integer"},"int8":{"type":"Integer"},"uint16":{"type":"Integer"},"int16":{"type":"Integer","subtype":"Int16"},"uint32":{"type":"Integer64"},"int32":{"type":"Integer"},"uint64":{"type":"Real"},"int64":{"type":"Integer64"},"float32":{"type":"Real","subtype":"Float32"},"float64":{"type":"Real"},"string":{"type":"String"},"large_string":{"type":"String"},"timestamp_ms_gmt":{"type":"DateTime"},"timestamp_ms_gmt_plus_2":{"type":"DateTime"},"timestamp_ms_gmt_minus_0215":{"type":"DateTime"},"timestamp_s_no_tz":{"type":"DateTime"},"time32_s":{"type":"Time"},"time32_ms":{"type":"Time"},"time64_us":{"type":"Integer64"},"time64_ns":{"type":"Integer64"},"date32":{"type":"Date"},"date64":{"type":"Date"},"binary":{"type":"Binary"},"large_binary":{"type":"Binary"},"fixed_size_binary":{"type":"Binary","width":2},"decimal128":{"type":"Real","width":7,"precision":3},"decimal256":{"type":"Real","width":7,"precision":3},"list_boolean":{"type":"IntegerList","subtype":"Boolean"},"list_uint8":{"type":"IntegerList"},"list_int8":{"type":"IntegerList"},"list_uint16":{"type":"IntegerList"},"list_int16":{"type":"IntegerList"},"list_uint32":{"type":"Integer64List"},"list_int32":{"type":"IntegerList"},"list_uint64":{"type":"RealList"},"list_int64":{"type":"Integer64List"},"list_float32":{"type":"RealList","subtype":"Float32"},"list_float64":{"type":"RealList"},"list_string":{"type":"StringList"},"fixed_size_list_boolean":{"type":"IntegerList","subtype":"Boolean"},"fixed_size_list_uint8":{"type":"IntegerList"},"fixed_size_list_int8":{"type":"IntegerList"},"fixed_size_list_uint16":{"type":"IntegerList"},"fixed_size_list_int16":{"type":"IntegerList"},"fixed_size_list_uint32":{"type":"Integer64List"},"fixed_size_list_int32":{"type":"IntegerList"},"fixed_size_list_uint64":{"type":"RealList"},"fixed_size_list_int64":{"type":"Integer64List"},"fixed_size_list_float32":{"type":"RealList","subtype":"Float32"},"fixed_size_list_float64":{"type":"RealList"},"fixed_size_list_string":{"type":"StringList"},"struct_field.a":{"type":"Integer64"},"struct_field.b":{"type":"Real"},"struct_field.c.d":{"type":"String"},"struct_field.c.f":{"type":"String"},"struct_field.h":{"type":"Integer64List"},"struct_field.i":{"type":"Integer64"},"map_boolean":{"type":"String","subtype":"JSON"},"map_uint8":{"type":"String","subtype":"JSON"},"map_int8":{"type":"String","subtype":"JSON"},"map_uint16":{"type":"String","subtype":"JSON"},"map_int16":{"type":"String","subtype":"JSON"},"map_uint32":{"type":"String","subtype":"JSON"},"map_int32":{"type":"String","subtype":"JSON"},"map_uint64":{"type":"String","subtype":"JSON"},"map_int64":{"type":"String","subtype":"JSON"},"map_float32":{"type":"String","subtype":"JSON"},"map_float64":{"type":"String","subtype":"JSON"},"map_string":{"type":"String","subtype":"JSON"},"dict":{"type":"Integer"}}}
GDAL: GDALOpen(out_SNAPPY.parquet, this=000002052253DF90) succeeds as Parquet.
ARROW: Memory pool: bytes_allocated = 0
ARROW: Memory pool: max_memory = 0
GDAL: GDALClose(out_SNAPPY.parquet, this=000002052253DF90)
GDAL: In GDALDestroy - unloading GDAL shared library.
INFO: Open of `out_SNAPPY.parquet'
      using driver `Parquet' successful.

Layer name: out_SNAPPY
Geometry: Point
Feature Count: 5
Extent: (0.000000, 2.000000) - (4.000000, 2.000000)
Layer SRS WKT:
GEOGCRS["WGS 84",
    ENSEMBLE["World Geodetic System 1984 ensemble",
        MEMBER["World Geodetic System 1984 (Transit)"],
        MEMBER["World Geodetic System 1984 (G730)"],
        MEMBER["World Geodetic System 1984 (G873)"],
        MEMBER["World Geodetic System 1984 (G1150)"],
        MEMBER["World Geodetic System 1984 (G1674)"],
        MEMBER["World Geodetic System 1984 (G1762)"],
        MEMBER["World Geodetic System 1984 (G2139)"],
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]],
        ENSEMBLEACCURACY[2.0]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    USAGE[
        SCOPE["Horizontal component of 3D system."],
        AREA["World."],
        BBOX[-90,-180,90,180]],
    ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
Geometry Column = geometry
boolean: Integer(Boolean) (0.0)
uint8: Integer (0.0)
int8: Integer (0.0)
uint16: Integer (0.0)
int16: Integer(Int16) (0.0)
uint32: Integer64 (0.0)
int32: Integer (0.0)
uint64: Real (0.0)
int64: Integer64 (0.0)
float32: Real(Float32) (0.0)
float64: Real (0.0)
string: String (0.0)
large_string: String (0.0)
timestamp_ms_gmt: DateTime (0.0)
timestamp_ms_gmt_plus_2: DateTime (0.0)
timestamp_ms_gmt_minus_0215: DateTime (0.0)
timestamp_s_no_tz: DateTime (0.0)
time32_s: Time (0.0)
time32_ms: Time (0.0)
time64_us: Integer64 (0.0)
time64_ns: Integer64 (0.0)
date32: Date (0.0)
date64: Date (0.0)
binary: Binary (0.0)
large_binary: Binary (0.0)
fixed_size_binary: Binary (2.0)
decimal128: Real (7.3)
decimal256: Real (7.3)
list_boolean: IntegerList(Boolean) (0.0)
list_uint8: IntegerList (0.0)
list_int8: IntegerList (0.0)
list_uint16: IntegerList (0.0)
list_int16: IntegerList (0.0)
list_uint32: Integer64List (0.0)
list_int32: IntegerList (0.0)
list_uint64: RealList (0.0)
list_int64: Integer64List (0.0)
list_float32: RealList(Float32) (0.0)
list_float64: RealList (0.0)
list_string: StringList (0.0)
fixed_size_list_boolean: IntegerList(Boolean) (0.0)
fixed_size_list_uint8: IntegerList (0.0)
fixed_size_list_int8: IntegerList (0.0)
fixed_size_list_uint16: IntegerList (0.0)
fixed_size_list_int16: IntegerList (0.0)
fixed_size_list_uint32: Integer64List (0.0)
fixed_size_list_int32: IntegerList (0.0)
fixed_size_list_uint64: RealList (0.0)
fixed_size_list_int64: Integer64List (0.0)
fixed_size_list_float32: RealList(Float32) (0.0)
fixed_size_list_float64: RealList (0.0)
fixed_size_list_string: StringList (0.0)
struct_field.a: Integer64 (0.0)
struct_field.b: Real (0.0)
struct_field.c.d: String (0.0)
struct_field.c.f: String (0.0)
struct_field.h: Integer64List (0.0)
struct_field.i: Integer64 (0.0)
map_boolean: String(JSON) (0.0)
map_uint8: String(JSON) (0.0)
map_int8: String(JSON) (0.0)
map_uint16: String(JSON) (0.0)
map_int16: String(JSON) (0.0)
map_uint32: String(JSON) (0.0)
map_int32: String(JSON) (0.0)
map_uint64: String(JSON) (0.0)
map_int64: String(JSON) (0.0)
map_float32: String(JSON) (0.0)
map_float64: String(JSON) (0.0)
map_string: String(JSON) (0.0)
dict: Integer (0.0), domain name=dictDomain

C:\OSGeo4W>(
if exist out_GZIP.parquet del out_GZIP.parquet  
 ogr2ogr -lco COMPRESSION=GZIP out_GZIP.parquet test.parquet  
 ogrinfo -so out_GZIP.parquet out_GZIP 
) 
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_ECW_JP2ECW.dll using GDALRegister_ECW_JP2ECW.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_GEOR.dll using GDALRegister_GEOR.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_HDF5.dll using GDALRegister_HDF5.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_MrSID.dll using GDALRegister_MrSID.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_OCI.dll using RegisterOGROCI.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_SOSI.dll using RegisterOGRSOSI.
PARQUET: geo = {"version": "0.1.0", "primary_column": "geometry", "columns": {"geometry": {"crs": "GEOGCRS[\"WGS 84\",ENSEMBLE[\"World Geodetic System 1984 ensemble\",MEMBER[\"World Geodetic System 1984 (Transit)\"],MEMBER[\"World Geodetic System 1984 (G730)\"],MEMBER[\"World Geodetic System 1984 (G873)\"],MEMBER[\"World Geodetic System 1984 (G1150)\"],MEMBER[\"World Geodetic System 1984 (G1674)\"],MEMBER[\"World Geodetic System 1984 (G1762)\"],MEMBER[\"World Geodetic System 1984 (G2139)\"],ELLIPSOID[\"WGS 84\",6378137,298.257223563],ENSEMBLEACCURACY[2.0]],CS[ellipsoidal,2],AXIS[\"geodetic latitude (Lat)\",north],AXIS[\"geodetic longitude (Lon)\",east],UNIT[\"degree\",0.0174532925199433],USAGE[SCOPE[\"Horizontal component of 3D system.\"],AREA[\"World.\"],BBOX[-90,-180,90,180]],ID[\"EPSG\",4326]]", "bbox": [0, 2, 4, 2], "encoding": "WKB"}}}
GDAL: GDALOpen(test.parquet, this=000002BB7900BDB0) succeeds as Parquet.
GDAL: Using Parquet driver
GDAL: GDALDriver::Create(Parquet,out_GZIP.parquet,0,0,0,Unknown,0000000000000000)
GDALVectorTranslate: 5 features written in layer 'test'
ARROW: Memory pool: bytes_allocated = 14144
ARROW: Memory pool: max_memory = 35584
GDAL: GDALClose(test.parquet, this=000002BB7900BDB0)
ARROW: Memory pool (writer layer): bytes_allocated = 128
ARROW: Memory pool (writer layer): max_memory = 36160
GDAL: GDALClose(out_GZIP.parquet, this=000002BB7BD97130)
GDAL: In GDALDestroy - unloading GDAL shared library.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_ECW_JP2ECW.dll using GDALRegister_ECW_JP2ECW.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_GEOR.dll using GDALRegister_GEOR.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_HDF5.dll using GDALRegister_HDF5.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_MrSID.dll using GDALRegister_MrSID.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_OCI.dll using RegisterOGROCI.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_SOSI.dll using RegisterOGRSOSI.
PARQUET: geo = {"version":"0.4.0","primary_column":"geometry","columns":{"geometry":{"encoding":"WKB","bbox":[0.0,2.0,4.0,2.0],"orientation":"counterclockwise","geometry_type":"Point"}}}
PARQUET: gdal:schema = {"columns":{"boolean":{"type":"Integer","subtype":"Boolean"},"uint8":{"type":"Integer"},"int8":{"type":"Integer"},"uint16":{"type":"Integer"},"int16":{"type":"Integer","subtype":"Int16"},"uint32":{"type":"Integer64"},"int32":{"type":"Integer"},"uint64":{"type":"Real"},"int64":{"type":"Integer64"},"float32":{"type":"Real","subtype":"Float32"},"float64":{"type":"Real"},"string":{"type":"String"},"large_string":{"type":"String"},"timestamp_ms_gmt":{"type":"DateTime"},"timestamp_ms_gmt_plus_2":{"type":"DateTime"},"timestamp_ms_gmt_minus_0215":{"type":"DateTime"},"timestamp_s_no_tz":{"type":"DateTime"},"time32_s":{"type":"Time"},"time32_ms":{"type":"Time"},"time64_us":{"type":"Integer64"},"time64_ns":{"type":"Integer64"},"date32":{"type":"Date"},"date64":{"type":"Date"},"binary":{"type":"Binary"},"large_binary":{"type":"Binary"},"fixed_size_binary":{"type":"Binary","width":2},"decimal128":{"type":"Real","width":7,"precision":3},"decimal256":{"type":"Real","width":7,"precision":3},"list_boolean":{"type":"IntegerList","subtype":"Boolean"},"list_uint8":{"type":"IntegerList"},"list_int8":{"type":"IntegerList"},"list_uint16":{"type":"IntegerList"},"list_int16":{"type":"IntegerList"},"list_uint32":{"type":"Integer64List"},"list_int32":{"type":"IntegerList"},"list_uint64":{"type":"RealList"},"list_int64":{"type":"Integer64List"},"list_float32":{"type":"RealList","subtype":"Float32"},"list_float64":{"type":"RealList"},"list_string":{"type":"StringList"},"fixed_size_list_boolean":{"type":"IntegerList","subtype":"Boolean"},"fixed_size_list_uint8":{"type":"IntegerList"},"fixed_size_list_int8":{"type":"IntegerList"},"fixed_size_list_uint16":{"type":"IntegerList"},"fixed_size_list_int16":{"type":"IntegerList"},"fixed_size_list_uint32":{"type":"Integer64List"},"fixed_size_list_int32":{"type":"IntegerList"},"fixed_size_list_uint64":{"type":"RealList"},"fixed_size_list_int64":{"type":"Integer64List"},"fixed_size_list_float32":{"type":"RealList","subtype":"Float32"},"fixed_size_list_float64":{"type":"RealList"},"fixed_size_list_string":{"type":"StringList"},"struct_field.a":{"type":"Integer64"},"struct_field.b":{"type":"Real"},"struct_field.c.d":{"type":"String"},"struct_field.c.f":{"type":"String"},"struct_field.h":{"type":"Integer64List"},"struct_field.i":{"type":"Integer64"},"map_boolean":{"type":"String","subtype":"JSON"},"map_uint8":{"type":"String","subtype":"JSON"},"map_int8":{"type":"String","subtype":"JSON"},"map_uint16":{"type":"String","subtype":"JSON"},"map_int16":{"type":"String","subtype":"JSON"},"map_uint32":{"type":"String","subtype":"JSON"},"map_int32":{"type":"String","subtype":"JSON"},"map_uint64":{"type":"String","subtype":"JSON"},"map_int64":{"type":"String","subtype":"JSON"},"map_float32":{"type":"String","subtype":"JSON"},"map_float64":{"type":"String","subtype":"JSON"},"map_string":{"type":"String","subtype":"JSON"},"dict":{"type":"Integer"}}}
GDAL: GDALOpen(out_GZIP.parquet, this=00000229E5B6BF30) succeeds as Parquet.
ARROW: Memory pool: bytes_allocated = 0
ARROW: Memory pool: max_memory = 0
GDAL: GDALClose(out_GZIP.parquet, this=00000229E5B6BF30)
GDAL: In GDALDestroy - unloading GDAL shared library.
INFO: Open of `out_GZIP.parquet'
      using driver `Parquet' successful.

Layer name: out_GZIP
Geometry: Point
Feature Count: 5
Extent: (0.000000, 2.000000) - (4.000000, 2.000000)
Layer SRS WKT:
GEOGCRS["WGS 84",
    ENSEMBLE["World Geodetic System 1984 ensemble",
        MEMBER["World Geodetic System 1984 (Transit)"],
        MEMBER["World Geodetic System 1984 (G730)"],
        MEMBER["World Geodetic System 1984 (G873)"],
        MEMBER["World Geodetic System 1984 (G1150)"],
        MEMBER["World Geodetic System 1984 (G1674)"],
        MEMBER["World Geodetic System 1984 (G1762)"],
        MEMBER["World Geodetic System 1984 (G2139)"],
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]],
        ENSEMBLEACCURACY[2.0]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    USAGE[
        SCOPE["Horizontal component of 3D system."],
        AREA["World."],
        BBOX[-90,-180,90,180]],
    ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
Geometry Column = geometry
boolean: Integer(Boolean) (0.0)
uint8: Integer (0.0)
int8: Integer (0.0)
uint16: Integer (0.0)
int16: Integer(Int16) (0.0)
uint32: Integer64 (0.0)
int32: Integer (0.0)
uint64: Real (0.0)
int64: Integer64 (0.0)
float32: Real(Float32) (0.0)
float64: Real (0.0)
string: String (0.0)
large_string: String (0.0)
timestamp_ms_gmt: DateTime (0.0)
timestamp_ms_gmt_plus_2: DateTime (0.0)
timestamp_ms_gmt_minus_0215: DateTime (0.0)
timestamp_s_no_tz: DateTime (0.0)
time32_s: Time (0.0)
time32_ms: Time (0.0)
time64_us: Integer64 (0.0)
time64_ns: Integer64 (0.0)
date32: Date (0.0)
date64: Date (0.0)
binary: Binary (0.0)
large_binary: Binary (0.0)
fixed_size_binary: Binary (2.0)
decimal128: Real (7.3)
decimal256: Real (7.3)
list_boolean: IntegerList(Boolean) (0.0)
list_uint8: IntegerList (0.0)
list_int8: IntegerList (0.0)
list_uint16: IntegerList (0.0)
list_int16: IntegerList (0.0)
list_uint32: Integer64List (0.0)
list_int32: IntegerList (0.0)
list_uint64: RealList (0.0)
list_int64: Integer64List (0.0)
list_float32: RealList(Float32) (0.0)
list_float64: RealList (0.0)
list_string: StringList (0.0)
fixed_size_list_boolean: IntegerList(Boolean) (0.0)
fixed_size_list_uint8: IntegerList (0.0)
fixed_size_list_int8: IntegerList (0.0)
fixed_size_list_uint16: IntegerList (0.0)
fixed_size_list_int16: IntegerList (0.0)
fixed_size_list_uint32: Integer64List (0.0)
fixed_size_list_int32: IntegerList (0.0)
fixed_size_list_uint64: RealList (0.0)
fixed_size_list_int64: Integer64List (0.0)
fixed_size_list_float32: RealList(Float32) (0.0)
fixed_size_list_float64: RealList (0.0)
fixed_size_list_string: StringList (0.0)
struct_field.a: Integer64 (0.0)
struct_field.b: Real (0.0)
struct_field.c.d: String (0.0)
struct_field.c.f: String (0.0)
struct_field.h: Integer64List (0.0)
struct_field.i: Integer64 (0.0)
map_boolean: String(JSON) (0.0)
map_uint8: String(JSON) (0.0)
map_int8: String(JSON) (0.0)
map_uint16: String(JSON) (0.0)
map_int16: String(JSON) (0.0)
map_uint32: String(JSON) (0.0)
map_int32: String(JSON) (0.0)
map_uint64: String(JSON) (0.0)
map_int64: String(JSON) (0.0)
map_float32: String(JSON) (0.0)
map_float64: String(JSON) (0.0)
map_string: String(JSON) (0.0)
dict: Integer (0.0), domain name=dictDomain

C:\OSGeo4W>(
if exist out_ZSTD.parquet del out_ZSTD.parquet  
 ogr2ogr -lco COMPRESSION=ZSTD out_ZSTD.parquet test.parquet  
 ogrinfo -so out_ZSTD.parquet out_ZSTD 
) 
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_ECW_JP2ECW.dll using GDALRegister_ECW_JP2ECW.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_GEOR.dll using GDALRegister_GEOR.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_HDF5.dll using GDALRegister_HDF5.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_MrSID.dll using GDALRegister_MrSID.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_OCI.dll using RegisterOGROCI.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_SOSI.dll using RegisterOGRSOSI.
PARQUET: geo = {"version": "0.1.0", "primary_column": "geometry", "columns": {"geometry": {"crs": "GEOGCRS[\"WGS 84\",ENSEMBLE[\"World Geodetic System 1984 ensemble\",MEMBER[\"World Geodetic System 1984 (Transit)\"],MEMBER[\"World Geodetic System 1984 (G730)\"],MEMBER[\"World Geodetic System 1984 (G873)\"],MEMBER[\"World Geodetic System 1984 (G1150)\"],MEMBER[\"World Geodetic System 1984 (G1674)\"],MEMBER[\"World Geodetic System 1984 (G1762)\"],MEMBER[\"World Geodetic System 1984 (G2139)\"],ELLIPSOID[\"WGS 84\",6378137,298.257223563],ENSEMBLEACCURACY[2.0]],CS[ellipsoidal,2],AXIS[\"geodetic latitude (Lat)\",north],AXIS[\"geodetic longitude (Lon)\",east],UNIT[\"degree\",0.0174532925199433],USAGE[SCOPE[\"Horizontal component of 3D system.\"],AREA[\"World.\"],BBOX[-90,-180,90,180]],ID[\"EPSG\",4326]]", "bbox": [0, 2, 4, 2], "encoding": "WKB"}}}
GDAL: GDALOpen(test.parquet, this=000001BC8D2C4DA0) succeeds as Parquet.
GDAL: Using Parquet driver
GDAL: GDALDriver::Create(Parquet,out_ZSTD.parquet,0,0,0,Unknown,0000000000000000)
GDALVectorTranslate: 5 features written in layer 'test'
ARROW: Memory pool: bytes_allocated = 14144
ARROW: Memory pool: max_memory = 35584
GDAL: GDALClose(test.parquet, this=000001BC8D2C4DA0)
ARROW: Memory pool (writer layer): bytes_allocated = 128
ARROW: Memory pool (writer layer): max_memory = 36160
GDAL: GDALClose(out_ZSTD.parquet, this=000001BC8D2D8690)
GDAL: In GDALDestroy - unloading GDAL shared library.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_ECW_JP2ECW.dll using GDALRegister_ECW_JP2ECW.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_GEOR.dll using GDALRegister_GEOR.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_HDF5.dll using GDALRegister_HDF5.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_MrSID.dll using GDALRegister_MrSID.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_OCI.dll using RegisterOGROCI.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_SOSI.dll using RegisterOGRSOSI.
PARQUET: geo = {"version":"0.4.0","primary_column":"geometry","columns":{"geometry":{"encoding":"WKB","bbox":[0.0,2.0,4.0,2.0],"orientation":"counterclockwise","geometry_type":"Point"}}}
PARQUET: gdal:schema = {"columns":{"boolean":{"type":"Integer","subtype":"Boolean"},"uint8":{"type":"Integer"},"int8":{"type":"Integer"},"uint16":{"type":"Integer"},"int16":{"type":"Integer","subtype":"Int16"},"uint32":{"type":"Integer64"},"int32":{"type":"Integer"},"uint64":{"type":"Real"},"int64":{"type":"Integer64"},"float32":{"type":"Real","subtype":"Float32"},"float64":{"type":"Real"},"string":{"type":"String"},"large_string":{"type":"String"},"timestamp_ms_gmt":{"type":"DateTime"},"timestamp_ms_gmt_plus_2":{"type":"DateTime"},"timestamp_ms_gmt_minus_0215":{"type":"DateTime"},"timestamp_s_no_tz":{"type":"DateTime"},"time32_s":{"type":"Time"},"time32_ms":{"type":"Time"},"time64_us":{"type":"Integer64"},"time64_ns":{"type":"Integer64"},"date32":{"type":"Date"},"date64":{"type":"Date"},"binary":{"type":"Binary"},"large_binary":{"type":"Binary"},"fixed_size_binary":{"type":"Binary","width":2},"decimal128":{"type":"Real","width":7,"precision":3},"decimal256":{"type":"Real","width":7,"precision":3},"list_boolean":{"type":"IntegerList","subtype":"Boolean"},"list_uint8":{"type":"IntegerList"},"list_int8":{"type":"IntegerList"},"list_uint16":{"type":"IntegerList"},"list_int16":{"type":"IntegerList"},"list_uint32":{"type":"Integer64List"},"list_int32":{"type":"IntegerList"},"list_uint64":{"type":"RealList"},"list_int64":{"type":"Integer64List"},"list_float32":{"type":"RealList","subtype":"Float32"},"list_float64":{"type":"RealList"},"list_string":{"type":"StringList"},"fixed_size_list_boolean":{"type":"IntegerList","subtype":"Boolean"},"fixed_size_list_uint8":{"type":"IntegerList"},"fixed_size_list_int8":{"type":"IntegerList"},"fixed_size_list_uint16":{"type":"IntegerList"},"fixed_size_list_int16":{"type":"IntegerList"},"fixed_size_list_uint32":{"type":"Integer64List"},"fixed_size_list_int32":{"type":"IntegerList"},"fixed_size_list_uint64":{"type":"RealList"},"fixed_size_list_int64":{"type":"Integer64List"},"fixed_size_list_float32":{"type":"RealList","subtype":"Float32"},"fixed_size_list_float64":{"type":"RealList"},"fixed_size_list_string":{"type":"StringList"},"struct_field.a":{"type":"Integer64"},"struct_field.b":{"type":"Real"},"struct_field.c.d":{"type":"String"},"struct_field.c.f":{"type":"String"},"struct_field.h":{"type":"Integer64List"},"struct_field.i":{"type":"Integer64"},"map_boolean":{"type":"String","subtype":"JSON"},"map_uint8":{"type":"String","subtype":"JSON"},"map_int8":{"type":"String","subtype":"JSON"},"map_uint16":{"type":"String","subtype":"JSON"},"map_int16":{"type":"String","subtype":"JSON"},"map_uint32":{"type":"String","subtype":"JSON"},"map_int32":{"type":"String","subtype":"JSON"},"map_uint64":{"type":"String","subtype":"JSON"},"map_int64":{"type":"String","subtype":"JSON"},"map_float32":{"type":"String","subtype":"JSON"},"map_float64":{"type":"String","subtype":"JSON"},"map_string":{"type":"String","subtype":"JSON"},"dict":{"type":"Integer"}}}
GDAL: GDALOpen(out_ZSTD.parquet, this=000001719CF25650) succeeds as Parquet.
ARROW: Memory pool: bytes_allocated = 0
ARROW: Memory pool: max_memory = 0
GDAL: GDALClose(out_ZSTD.parquet, this=000001719CF25650)
GDAL: In GDALDestroy - unloading GDAL shared library.
INFO: Open of `out_ZSTD.parquet'
      using driver `Parquet' successful.

Layer name: out_ZSTD
Geometry: Point
Feature Count: 5
Extent: (0.000000, 2.000000) - (4.000000, 2.000000)
Layer SRS WKT:
GEOGCRS["WGS 84",
    ENSEMBLE["World Geodetic System 1984 ensemble",
        MEMBER["World Geodetic System 1984 (Transit)"],
        MEMBER["World Geodetic System 1984 (G730)"],
        MEMBER["World Geodetic System 1984 (G873)"],
        MEMBER["World Geodetic System 1984 (G1150)"],
        MEMBER["World Geodetic System 1984 (G1674)"],
        MEMBER["World Geodetic System 1984 (G1762)"],
        MEMBER["World Geodetic System 1984 (G2139)"],
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]],
        ENSEMBLEACCURACY[2.0]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    USAGE[
        SCOPE["Horizontal component of 3D system."],
        AREA["World."],
        BBOX[-90,-180,90,180]],
    ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
Geometry Column = geometry
boolean: Integer(Boolean) (0.0)
uint8: Integer (0.0)
int8: Integer (0.0)
uint16: Integer (0.0)
int16: Integer(Int16) (0.0)
uint32: Integer64 (0.0)
int32: Integer (0.0)
uint64: Real (0.0)
int64: Integer64 (0.0)
float32: Real(Float32) (0.0)
float64: Real (0.0)
string: String (0.0)
large_string: String (0.0)
timestamp_ms_gmt: DateTime (0.0)
timestamp_ms_gmt_plus_2: DateTime (0.0)
timestamp_ms_gmt_minus_0215: DateTime (0.0)
timestamp_s_no_tz: DateTime (0.0)
time32_s: Time (0.0)
time32_ms: Time (0.0)
time64_us: Integer64 (0.0)
time64_ns: Integer64 (0.0)
date32: Date (0.0)
date64: Date (0.0)
binary: Binary (0.0)
large_binary: Binary (0.0)
fixed_size_binary: Binary (2.0)
decimal128: Real (7.3)
decimal256: Real (7.3)
list_boolean: IntegerList(Boolean) (0.0)
list_uint8: IntegerList (0.0)
list_int8: IntegerList (0.0)
list_uint16: IntegerList (0.0)
list_int16: IntegerList (0.0)
list_uint32: Integer64List (0.0)
list_int32: IntegerList (0.0)
list_uint64: RealList (0.0)
list_int64: Integer64List (0.0)
list_float32: RealList(Float32) (0.0)
list_float64: RealList (0.0)
list_string: StringList (0.0)
fixed_size_list_boolean: IntegerList(Boolean) (0.0)
fixed_size_list_uint8: IntegerList (0.0)
fixed_size_list_int8: IntegerList (0.0)
fixed_size_list_uint16: IntegerList (0.0)
fixed_size_list_int16: IntegerList (0.0)
fixed_size_list_uint32: Integer64List (0.0)
fixed_size_list_int32: IntegerList (0.0)
fixed_size_list_uint64: RealList (0.0)
fixed_size_list_int64: Integer64List (0.0)
fixed_size_list_float32: RealList(Float32) (0.0)
fixed_size_list_float64: RealList (0.0)
fixed_size_list_string: StringList (0.0)
struct_field.a: Integer64 (0.0)
struct_field.b: Real (0.0)
struct_field.c.d: String (0.0)
struct_field.c.f: String (0.0)
struct_field.h: Integer64List (0.0)
struct_field.i: Integer64 (0.0)
map_boolean: String(JSON) (0.0)
map_uint8: String(JSON) (0.0)
map_int8: String(JSON) (0.0)
map_uint16: String(JSON) (0.0)
map_int16: String(JSON) (0.0)
map_uint32: String(JSON) (0.0)
map_int32: String(JSON) (0.0)
map_uint64: String(JSON) (0.0)
map_int64: String(JSON) (0.0)
map_float32: String(JSON) (0.0)
map_float64: String(JSON) (0.0)
map_string: String(JSON) (0.0)
dict: Integer (0.0), domain name=dictDomain

C:\OSGeo4W>(
if exist out_BROTLI.parquet del out_BROTLI.parquet  
 ogr2ogr -lco COMPRESSION=BROTLI out_BROTLI.parquet test.parquet  
 ogrinfo -so out_BROTLI.parquet out_BROTLI 
) 
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_ECW_JP2ECW.dll using GDALRegister_ECW_JP2ECW.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_GEOR.dll using GDALRegister_GEOR.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_HDF5.dll using GDALRegister_HDF5.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_MrSID.dll using GDALRegister_MrSID.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_OCI.dll using RegisterOGROCI.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_SOSI.dll using RegisterOGRSOSI.
PARQUET: geo = {"version": "0.1.0", "primary_column": "geometry", "columns": {"geometry": {"crs": "GEOGCRS[\"WGS 84\",ENSEMBLE[\"World Geodetic System 1984 ensemble\",MEMBER[\"World Geodetic System 1984 (Transit)\"],MEMBER[\"World Geodetic System 1984 (G730)\"],MEMBER[\"World Geodetic System 1984 (G873)\"],MEMBER[\"World Geodetic System 1984 (G1150)\"],MEMBER[\"World Geodetic System 1984 (G1674)\"],MEMBER[\"World Geodetic System 1984 (G1762)\"],MEMBER[\"World Geodetic System 1984 (G2139)\"],ELLIPSOID[\"WGS 84\",6378137,298.257223563],ENSEMBLEACCURACY[2.0]],CS[ellipsoidal,2],AXIS[\"geodetic latitude (Lat)\",north],AXIS[\"geodetic longitude (Lon)\",east],UNIT[\"degree\",0.0174532925199433],USAGE[SCOPE[\"Horizontal component of 3D system.\"],AREA[\"World.\"],BBOX[-90,-180,90,180]],ID[\"EPSG\",4326]]", "bbox": [0, 2, 4, 2], "encoding": "WKB"}}}
GDAL: GDALOpen(test.parquet, this=00000294A7A6B800) succeeds as Parquet.
GDAL: Using Parquet driver
GDAL: GDALDriver::Create(Parquet,out_BROTLI.parquet,0,0,0,Unknown,0000000000000000)
GDALVectorTranslate: 5 features written in layer 'test'
ARROW: Memory pool: bytes_allocated = 14144
ARROW: Memory pool: max_memory = 35584
GDAL: GDALClose(test.parquet, this=00000294A7A6B800)
ARROW: Memory pool (writer layer): bytes_allocated = 128
ARROW: Memory pool (writer layer): max_memory = 36160
GDAL: GDALClose(out_BROTLI.parquet, this=00000294A8695000)
GDAL: In GDALDestroy - unloading GDAL shared library.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_ECW_JP2ECW.dll using GDALRegister_ECW_JP2ECW.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_GEOR.dll using GDALRegister_GEOR.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_HDF5.dll using GDALRegister_HDF5.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_MrSID.dll using GDALRegister_MrSID.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_OCI.dll using RegisterOGROCI.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_SOSI.dll using RegisterOGRSOSI.
PARQUET: geo = {"version":"0.4.0","primary_column":"geometry","columns":{"geometry":{"encoding":"WKB","bbox":[0.0,2.0,4.0,2.0],"orientation":"counterclockwise","geometry_type":"Point"}}}
PARQUET: gdal:schema = {"columns":{"boolean":{"type":"Integer","subtype":"Boolean"},"uint8":{"type":"Integer"},"int8":{"type":"Integer"},"uint16":{"type":"Integer"},"int16":{"type":"Integer","subtype":"Int16"},"uint32":{"type":"Integer64"},"int32":{"type":"Integer"},"uint64":{"type":"Real"},"int64":{"type":"Integer64"},"float32":{"type":"Real","subtype":"Float32"},"float64":{"type":"Real"},"string":{"type":"String"},"large_string":{"type":"String"},"timestamp_ms_gmt":{"type":"DateTime"},"timestamp_ms_gmt_plus_2":{"type":"DateTime"},"timestamp_ms_gmt_minus_0215":{"type":"DateTime"},"timestamp_s_no_tz":{"type":"DateTime"},"time32_s":{"type":"Time"},"time32_ms":{"type":"Time"},"time64_us":{"type":"Integer64"},"time64_ns":{"type":"Integer64"},"date32":{"type":"Date"},"date64":{"type":"Date"},"binary":{"type":"Binary"},"large_binary":{"type":"Binary"},"fixed_size_binary":{"type":"Binary","width":2},"decimal128":{"type":"Real","width":7,"precision":3},"decimal256":{"type":"Real","width":7,"precision":3},"list_boolean":{"type":"IntegerList","subtype":"Boolean"},"list_uint8":{"type":"IntegerList"},"list_int8":{"type":"IntegerList"},"list_uint16":{"type":"IntegerList"},"list_int16":{"type":"IntegerList"},"list_uint32":{"type":"Integer64List"},"list_int32":{"type":"IntegerList"},"list_uint64":{"type":"RealList"},"list_int64":{"type":"Integer64List"},"list_float32":{"type":"RealList","subtype":"Float32"},"list_float64":{"type":"RealList"},"list_string":{"type":"StringList"},"fixed_size_list_boolean":{"type":"IntegerList","subtype":"Boolean"},"fixed_size_list_uint8":{"type":"IntegerList"},"fixed_size_list_int8":{"type":"IntegerList"},"fixed_size_list_uint16":{"type":"IntegerList"},"fixed_size_list_int16":{"type":"IntegerList"},"fixed_size_list_uint32":{"type":"Integer64List"},"fixed_size_list_int32":{"type":"IntegerList"},"fixed_size_list_uint64":{"type":"RealList"},"fixed_size_list_int64":{"type":"Integer64List"},"fixed_size_list_float32":{"type":"RealList","subtype":"Float32"},"fixed_size_list_float64":{"type":"RealList"},"fixed_size_list_string":{"type":"StringList"},"struct_field.a":{"type":"Integer64"},"struct_field.b":{"type":"Real"},"struct_field.c.d":{"type":"String"},"struct_field.c.f":{"type":"String"},"struct_field.h":{"type":"Integer64List"},"struct_field.i":{"type":"Integer64"},"map_boolean":{"type":"String","subtype":"JSON"},"map_uint8":{"type":"String","subtype":"JSON"},"map_int8":{"type":"String","subtype":"JSON"},"map_uint16":{"type":"String","subtype":"JSON"},"map_int16":{"type":"String","subtype":"JSON"},"map_uint32":{"type":"String","subtype":"JSON"},"map_int32":{"type":"String","subtype":"JSON"},"map_uint64":{"type":"String","subtype":"JSON"},"map_int64":{"type":"String","subtype":"JSON"},"map_float32":{"type":"String","subtype":"JSON"},"map_float64":{"type":"String","subtype":"JSON"},"map_string":{"type":"String","subtype":"JSON"},"dict":{"type":"Integer"}}}
GDAL: GDALOpen(out_BROTLI.parquet, this=000001D3C729F120) succeeds as Parquet.
ARROW: Memory pool: bytes_allocated = 0
ARROW: Memory pool: max_memory = 0
GDAL: GDALClose(out_BROTLI.parquet, this=000001D3C729F120)
GDAL: In GDALDestroy - unloading GDAL shared library.
INFO: Open of `out_BROTLI.parquet'
      using driver `Parquet' successful.

Layer name: out_BROTLI
Geometry: Point
Feature Count: 5
Extent: (0.000000, 2.000000) - (4.000000, 2.000000)
Layer SRS WKT:
GEOGCRS["WGS 84",
    ENSEMBLE["World Geodetic System 1984 ensemble",
        MEMBER["World Geodetic System 1984 (Transit)"],
        MEMBER["World Geodetic System 1984 (G730)"],
        MEMBER["World Geodetic System 1984 (G873)"],
        MEMBER["World Geodetic System 1984 (G1150)"],
        MEMBER["World Geodetic System 1984 (G1674)"],
        MEMBER["World Geodetic System 1984 (G1762)"],
        MEMBER["World Geodetic System 1984 (G2139)"],
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]],
        ENSEMBLEACCURACY[2.0]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    USAGE[
        SCOPE["Horizontal component of 3D system."],
        AREA["World."],
        BBOX[-90,-180,90,180]],
    ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
Geometry Column = geometry
boolean: Integer(Boolean) (0.0)
uint8: Integer (0.0)
int8: Integer (0.0)
uint16: Integer (0.0)
int16: Integer(Int16) (0.0)
uint32: Integer64 (0.0)
int32: Integer (0.0)
uint64: Real (0.0)
int64: Integer64 (0.0)
float32: Real(Float32) (0.0)
float64: Real (0.0)
string: String (0.0)
large_string: String (0.0)
timestamp_ms_gmt: DateTime (0.0)
timestamp_ms_gmt_plus_2: DateTime (0.0)
timestamp_ms_gmt_minus_0215: DateTime (0.0)
timestamp_s_no_tz: DateTime (0.0)
time32_s: Time (0.0)
time32_ms: Time (0.0)
time64_us: Integer64 (0.0)
time64_ns: Integer64 (0.0)
date32: Date (0.0)
date64: Date (0.0)
binary: Binary (0.0)
large_binary: Binary (0.0)
fixed_size_binary: Binary (2.0)
decimal128: Real (7.3)
decimal256: Real (7.3)
list_boolean: IntegerList(Boolean) (0.0)
list_uint8: IntegerList (0.0)
list_int8: IntegerList (0.0)
list_uint16: IntegerList (0.0)
list_int16: IntegerList (0.0)
list_uint32: Integer64List (0.0)
list_int32: IntegerList (0.0)
list_uint64: RealList (0.0)
list_int64: Integer64List (0.0)
list_float32: RealList(Float32) (0.0)
list_float64: RealList (0.0)
list_string: StringList (0.0)
fixed_size_list_boolean: IntegerList(Boolean) (0.0)
fixed_size_list_uint8: IntegerList (0.0)
fixed_size_list_int8: IntegerList (0.0)
fixed_size_list_uint16: IntegerList (0.0)
fixed_size_list_int16: IntegerList (0.0)
fixed_size_list_uint32: Integer64List (0.0)
fixed_size_list_int32: IntegerList (0.0)
fixed_size_list_uint64: RealList (0.0)
fixed_size_list_int64: Integer64List (0.0)
fixed_size_list_float32: RealList(Float32) (0.0)
fixed_size_list_float64: RealList (0.0)
fixed_size_list_string: StringList (0.0)
struct_field.a: Integer64 (0.0)
struct_field.b: Real (0.0)
struct_field.c.d: String (0.0)
struct_field.c.f: String (0.0)
struct_field.h: Integer64List (0.0)
struct_field.i: Integer64 (0.0)
map_boolean: String(JSON) (0.0)
map_uint8: String(JSON) (0.0)
map_int8: String(JSON) (0.0)
map_uint16: String(JSON) (0.0)
map_int16: String(JSON) (0.0)
map_uint32: String(JSON) (0.0)
map_int32: String(JSON) (0.0)
map_uint64: String(JSON) (0.0)
map_int64: String(JSON) (0.0)
map_float32: String(JSON) (0.0)
map_float64: String(JSON) (0.0)
map_string: String(JSON) (0.0)
dict: Integer (0.0), domain name=dictDomain

C:\OSGeo4W>dir *.parquet 
 Volume in Laufwerk C: hat keine Bezeichnung.
 Volumeseriennummer: 22C3-C828

 Verzeichnis von C:\OSGeo4W

07/31/2022  09:26 PM            37,888 out_BROTLI.parquet
07/31/2022  09:26 PM            39,955 out_GZIP.parquet
07/31/2022  09:26 PM            37,921 out_SNAPPY.parquet
07/31/2022  09:26 PM            38,893 out_ZSTD.parquet
07/31/2022  04:19 PM            57,756 test.parquet
               5 Datei(en),        212,413 Bytes
               0 Verzeichnis(se), 36,217,315,328 Bytes frei

comment:2 by andreaerdna, 2 years ago

Thanks for looking at this.

Actually in some cases the crash only happens when the GDAL/OGR tries to retrieve the features from the layer (see ​https://github.com/OSGeo/gdal/issues/6115#issuecomment-1200104043)

Could you please try if in your system also the following command (without "-so"): 

ogrinfo out_SNAPPY.parquet out_SNAPPY --debug ON

terminates properly listing all the features and if QGIS doesn't crash opening the out_SNAPPY.parquet layer?

in reply to:  2 comment:3 by jef, 2 years ago

Replying to jef:

Not reproducable here. Are you seing invalid instructions in eventlog?

Replying to andreaerdna:

Could you please try if in your system also the following command (without "-so"): terminates properly listing all the features and if QGIS doesn't crash opening the out_SNAPPY.parquet layer?

apparently works fine. 

C:\OSGeo4W> ogrinfo out_SNAPPY.parquet out_SNAPPY --debug on & echo %errorlevel%
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_ECW_JP2ECW.dll using GDALRegister_ECW_JP2ECW.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_GEOR.dll using GDALRegister_GEOR.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_HDF5.dll using GDALRegister_HDF5.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\gdal_MrSID.dll using GDALRegister_MrSID.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_OCI.dll using RegisterOGROCI.
GDAL: Auto register C:\OSGeo4W\apps\gdal\lib\gdalplugins\ogr_SOSI.dll using RegisterOGRSOSI.
PARQUET: geo = {"version":"0.4.0","primary_column":"geometry","columns":{"geometry":{"encoding":"WKB","bbox":[0.0,2.0,4.0,2.0],"orientation":"counterclockwise","geometry_type":"Point"}}}
PARQUET: gdal:schema = {"columns":{"boolean":{"type":"Integer","subtype":"Boolean"},"uint8":{"type":"Integer"},"int8":{"type":"Integer"},"uint16":{"type":"Integer"},"int16":{"type":"Integer","subtype":"Int16"},"uint32":{"type":"Integer64"},"int32":{"type":"Integer"},"uint64":{"type":"Real"},"int64":{"type":"Integer64"},"float32":{"type":"Real","subtype":"Float32"},"float64":{"type":"Real"},"string":{"type":"String"},"large_string":{"type":"String"},"timestamp_ms_gmt":{"type":"DateTime"},"timestamp_ms_gmt_plus_2":{"type":"DateTime"},"timestamp_ms_gmt_minus_0215":{"type":"DateTime"},"timestamp_s_no_tz":{"type":"DateTime"},"time32_s":{"type":"Time"},"time32_ms":{"type":"Time"},"time64_us":{"type":"Integer64"},"time64_ns":{"type":"Integer64"},"date32":{"type":"Date"},"date64":{"type":"Date"},"binary":{"type":"Binary"},"large_binary":{"type":"Binary"},"fixed_size_binary":{"type":"Binary","width":2},"decimal128":{"type":"Real","width":7,"precision":3},"decimal256":{"type":"Real","width":7,"precision":3},"list_boolean":{"type":"IntegerList","subtype":"Boolean"},"list_uint8":{"type":"IntegerList"},"list_int8":{"type":"IntegerList"},"list_uint16":{"type":"IntegerList"},"list_int16":{"type":"IntegerList"},"list_uint32":{"type":"Integer64List"},"list_int32":{"type":"IntegerList"},"list_uint64":{"type":"RealList"},"list_int64":{"type":"Integer64List"},"list_float32":{"type":"RealList","subtype":"Float32"},"list_float64":{"type":"RealList"},"list_string":{"type":"StringList"},"fixed_size_list_boolean":{"type":"IntegerList","subtype":"Boolean"},"fixed_size_list_uint8":{"type":"IntegerList"},"fixed_size_list_int8":{"type":"IntegerList"},"fixed_size_list_uint16":{"type":"IntegerList"},"fixed_size_list_int16":{"type":"IntegerList"},"fixed_size_list_uint32":{"type":"Integer64List"},"fixed_size_list_int32":{"type":"IntegerList"},"fixed_size_list_uint64":{"type":"RealList"},"fixed_size_list_int64":{"type":"Integer64List"},"fixed_size_list_float32":{"type":"RealList","subtype":"Float32"},"fixed_size_list_float64":{"type":"RealList"},"fixed_size_list_string":{"type":"StringList"},"struct_field.a":{"type":"Integer64"},"struct_field.b":{"type":"Real"},"struct_field.c.d":{"type":"String"},"struct_field.c.f":{"type":"String"},"struct_field.h":{"type":"Integer64List"},"struct_field.i":{"type":"Integer64"},"map_boolean":{"type":"String","subtype":"JSON"},"map_uint8":{"type":"String","subtype":"JSON"},"map_int8":{"type":"String","subtype":"JSON"},"map_uint16":{"type":"String","subtype":"JSON"},"map_int16":{"type":"String","subtype":"JSON"},"map_uint32":{"type":"String","subtype":"JSON"},"map_int32":{"type":"String","subtype":"JSON"},"map_uint64":{"type":"String","subtype":"JSON"},"map_int64":{"type":"String","subtype":"JSON"},"map_float32":{"type":"String","subtype":"JSON"},"map_float64":{"type":"String","subtype":"JSON"},"map_string":{"type":"String","subtype":"JSON"},"dict":{"type":"Integer"}}}
GDAL: GDALOpen(out_SNAPPY.parquet, this=00000211A70E3DA0) succeeds as Parquet.
INFO: Open of `out_SNAPPY.parquet'
      using driver `Parquet' successful.

Layer name: out_SNAPPY
Geometry: Point
Feature Count: 5
Extent: (0.000000, 2.000000) - (4.000000, 2.000000)
Layer SRS WKT:
GEOGCRS["WGS 84",
    ENSEMBLE["World Geodetic System 1984 ensemble",
        MEMBER["World Geodetic System 1984 (Transit)"],
        MEMBER["World Geodetic System 1984 (G730)"],
        MEMBER["World Geodetic System 1984 (G873)"],
        MEMBER["World Geodetic System 1984 (G1150)"],
        MEMBER["World Geodetic System 1984 (G1674)"],
        MEMBER["World Geodetic System 1984 (G1762)"],
        MEMBER["World Geodetic System 1984 (G2139)"],
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]],
        ENSEMBLEACCURACY[2.0]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    USAGE[
        SCOPE["Horizontal component of 3D system."],
        AREA["World."],
        BBOX[-90,-180,90,180]],
    ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
Geometry Column = geometry
boolean: Integer(Boolean) (0.0)
uint8: Integer (0.0)
int8: Integer (0.0)
uint16: Integer (0.0)
int16: Integer(Int16) (0.0)
uint32: Integer64 (0.0)
int32: Integer (0.0)
uint64: Real (0.0)
int64: Integer64 (0.0)
float32: Real(Float32) (0.0)
float64: Real (0.0)
string: String (0.0)
large_string: String (0.0)
timestamp_ms_gmt: DateTime (0.0)
timestamp_ms_gmt_plus_2: DateTime (0.0)
timestamp_ms_gmt_minus_0215: DateTime (0.0)
timestamp_s_no_tz: DateTime (0.0)
time32_s: Time (0.0)
time32_ms: Time (0.0)
time64_us: Integer64 (0.0)
time64_ns: Integer64 (0.0)
date32: Date (0.0)
date64: Date (0.0)
binary: Binary (0.0)
large_binary: Binary (0.0)
fixed_size_binary: Binary (2.0)
decimal128: Real (7.3)
decimal256: Real (7.3)
list_boolean: IntegerList(Boolean) (0.0)
list_uint8: IntegerList (0.0)
list_int8: IntegerList (0.0)
list_uint16: IntegerList (0.0)
list_int16: IntegerList (0.0)
list_uint32: Integer64List (0.0)
list_int32: IntegerList (0.0)
list_uint64: RealList (0.0)
list_int64: Integer64List (0.0)
list_float32: RealList(Float32) (0.0)
list_float64: RealList (0.0)
list_string: StringList (0.0)
fixed_size_list_boolean: IntegerList(Boolean) (0.0)
fixed_size_list_uint8: IntegerList (0.0)
fixed_size_list_int8: IntegerList (0.0)
fixed_size_list_uint16: IntegerList (0.0)
fixed_size_list_int16: IntegerList (0.0)
fixed_size_list_uint32: Integer64List (0.0)
fixed_size_list_int32: IntegerList (0.0)
fixed_size_list_uint64: RealList (0.0)
fixed_size_list_int64: Integer64List (0.0)
fixed_size_list_float32: RealList(Float32) (0.0)
fixed_size_list_float64: RealList (0.0)
fixed_size_list_string: StringList (0.0)
struct_field.a: Integer64 (0.0)
struct_field.b: Real (0.0)
struct_field.c.d: String (0.0)
struct_field.c.f: String (0.0)
struct_field.h: Integer64List (0.0)
struct_field.i: Integer64 (0.0)
map_boolean: String(JSON) (0.0)
map_uint8: String(JSON) (0.0)
map_int8: String(JSON) (0.0)
map_uint16: String(JSON) (0.0)
map_int16: String(JSON) (0.0)
map_uint32: String(JSON) (0.0)
map_int32: String(JSON) (0.0)
map_uint64: String(JSON) (0.0)
map_int64: String(JSON) (0.0)
map_float32: String(JSON) (0.0)
map_float64: String(JSON) (0.0)
map_string: String(JSON) (0.0)
dict: Integer (0.0), domain name=dictDomain
OGRFeature(out_SNAPPY):0
  boolean (Integer(Boolean)) = 1
  uint8 (Integer) = 1
  int8 (Integer) = -2
  uint16 (Integer) = 1
  int16 (Integer(Int16)) = -20000
  uint32 (Integer64) = 1
  int32 (Integer) = -2000000000
  uint64 (Real) = 1
  int64 (Integer64) = -200000000000
  float32 (Real(Float32)) = 1.5
  float64 (Real) = 1.5
  string (String) = abcd
  large_string (String) = abcd
  timestamp_ms_gmt (DateTime) = 2019/01/01 14:00:00+00
  timestamp_ms_gmt_plus_2 (DateTime) = 2019/01/01 14:00:00+02
  timestamp_ms_gmt_minus_0215 (DateTime) = 2019/01/01 14:00:00-0215
  timestamp_s_no_tz (DateTime) = 2019/01/01 14:00:00
  time32_s (Time) = 01:02:03
  time32_ms (Time) = 01:02:03.456
  time64_us (Integer64) = 3723000000
  time64_ns (Integer64) = 3723000000456
  date32 (Date) = 1970/01/02
  date64 (Date) = 1970/01/02
  binary (Binary) = 0001
  large_binary (Binary) = 0001
  fixed_size_binary (Binary) = 0001
  decimal128 (Real) = 1234.567
  decimal256 (Real) = 1234.567
  list_boolean (IntegerList(Boolean)) = (0:)
  list_uint8 (IntegerList) = (0:)
  list_int8 (IntegerList) = (0:)
  list_uint16 (IntegerList) = (0:)
  list_int16 (IntegerList) = (0:)
  list_uint32 (Integer64List) = (0:)
  list_int32 (IntegerList) = (0:)
  list_uint64 (RealList) = (0:)
  list_int64 (Integer64List) = (0:)
  list_float32 (RealList(Float32)) = (0:)
  list_float64 (RealList) = (0:)
  list_string (StringList) = (0:)
  fixed_size_list_boolean (IntegerList(Boolean)) = (2:1,0)
  fixed_size_list_uint8 (IntegerList) = (2:0,1)
  fixed_size_list_int8 (IntegerList) = (2:0,1)
  fixed_size_list_uint16 (IntegerList) = (2:0,1)
  fixed_size_list_int16 (IntegerList) = (2:0,1)
  fixed_size_list_uint32 (Integer64List) = (2:0,1)
  fixed_size_list_int32 (IntegerList) = (2:0,1)
  fixed_size_list_uint64 (RealList) = (2:0,1)
  fixed_size_list_int64 (Integer64List) = (2:0,1)
  fixed_size_list_float32 (RealList(Float32)) = (2:0.0,nan)
  fixed_size_list_float64 (RealList) = (2:0,nan)
  fixed_size_list_string (StringList) = (2:a,b)
  struct_field.a (Integer64) = 1
  struct_field.b (Real) = 2.5
  struct_field.c.d (String) = e
  struct_field.c.f (String) = g
  struct_field.h (Integer64List) = (2:5,6)
  struct_field.i (Integer64) = 3
  map_boolean (String(JSON)) = {"x":null,"y":true}
  map_uint8 (String(JSON)) = {"x":1,"y":null}
  map_int8 (String(JSON)) = {"x":1,"y":null}
  map_uint16 (String(JSON)) = {"x":1,"y":null}
  map_int16 (String(JSON)) = {"x":1,"y":null}
  map_uint32 (String(JSON)) = {"x":4000000000,"y":null}
  map_int32 (String(JSON)) = {"x":2000000000,"y":null}
  map_uint64 (String(JSON)) = {"x":4000000000000.0,"y":null}
  map_int64 (String(JSON)) = {"x":-2000000000000,"y":null}
  map_float32 (String(JSON)) = {"x":1.5,"y":null}
  map_float64 (String(JSON)) = {"x":1.5,"y":null}
  map_string (String(JSON)) = {"x":"x_val","y":null}
  dict (Integer) = 0
  POINT (0 2)

OGRFeature(out_SNAPPY):1
  boolean (Integer(Boolean)) = 0
  uint8 (Integer) = 2
  int8 (Integer) = -1
  uint16 (Integer) = 10001
  int16 (Integer(Int16)) = -10000
  uint32 (Integer64) = 1000000001
  int32 (Integer) = -1000000000
  uint64 (Real) = 100000000001
  int64 (Integer64) = -100000000000
  float32 (Real(Float32)) = 2.5
  float64 (Real) = 2.5
  string (String) = 
  large_string (String) = 
  timestamp_ms_gmt (DateTime) = 2019/01/01 14:00:00+00
  timestamp_ms_gmt_plus_2 (DateTime) = 2019/01/01 14:00:00+02
  timestamp_ms_gmt_minus_0215 (DateTime) = 2019/01/01 14:00:00-0215
  timestamp_s_no_tz (DateTime) = 2019/01/01 14:00:00
  time32_s (Time) = (null)
  time32_ms (Time) = 00:00:00.002
  time64_us (Integer64) = (null)
  time64_ns (Integer64) = 2
  date32 (Date) = 1970/01/03
  date64 (Date) = 1970/01/01
  binary (Binary) = 0001
  large_binary (Binary) = 0001
  fixed_size_binary (Binary) = 0001
  decimal128 (Real) = -1234.567
  decimal256 (Real) = -1234.567
  list_boolean (IntegerList(Boolean)) = (1:0)
  list_uint8 (IntegerList) = (1:0)
  list_int8 (IntegerList) = (1:0)
  list_uint16 (IntegerList) = (1:0)
  list_int16 (IntegerList) = (1:0)
  list_uint32 (Integer64List) = (1:0)
  list_int32 (IntegerList) = (1:0)
  list_uint64 (RealList) = (1:0)
  list_int64 (Integer64List) = (1:0)
  list_float32 (RealList(Float32)) = (1:nan)
  list_float64 (RealList) = (1:nan)
  list_string (StringList) = (1:A)
  fixed_size_list_boolean (IntegerList(Boolean)) = (2:0,1)
  fixed_size_list_uint8 (IntegerList) = (2:2,3)
  fixed_size_list_int8 (IntegerList) = (2:2,3)
  fixed_size_list_uint16 (IntegerList) = (2:2,3)
  fixed_size_list_int16 (IntegerList) = (2:2,3)
  fixed_size_list_uint32 (Integer64List) = (2:2,3)
  fixed_size_list_int32 (IntegerList) = (2:2,3)
  fixed_size_list_uint64 (RealList) = (2:2,3)
  fixed_size_list_int64 (Integer64List) = (2:2,3)
  fixed_size_list_float32 (RealList(Float32)) = (2:2.0,3.0)
  fixed_size_list_float64 (RealList) = (2:2,3)
  fixed_size_list_string (StringList) = (2:c,d)
  struct_field.a (Integer64) = 1
  struct_field.b (Real) = 2.5
  struct_field.c.d (String) = e
  struct_field.c.f (String) = g
  struct_field.h (Integer64List) = (2:5,6)
  struct_field.i (Integer64) = 3
  map_boolean (String(JSON)) = {"z":true}
  map_uint8 (String(JSON)) = {"z":3}
  map_int8 (String(JSON)) = {"z":3}
  map_uint16 (String(JSON)) = {"z":3}
  map_int16 (String(JSON)) = {"z":3}
  map_uint32 (String(JSON)) = {"z":3}
  map_int32 (String(JSON)) = {"z":3}
  map_uint64 (String(JSON)) = {"z":3.0}
  map_int64 (String(JSON)) = {"z":3}
  map_float32 (String(JSON)) = {"z":3.0}
  map_float64 (String(JSON)) = {"z":3.0}
  map_string (String(JSON)) = {"z":"z_val"}
  dict (Integer) = 1

OGRFeature(out_SNAPPY):2
  boolean (Integer(Boolean)) = (null)
  uint8 (Integer) = (null)
  int8 (Integer) = (null)
  uint16 (Integer) = (null)
  int16 (Integer(Int16)) = (null)
  uint32 (Integer64) = (null)
  int32 (Integer) = (null)
  uint64 (Real) = (null)
  int64 (Integer64) = (null)
  float32 (Real(Float32)) = (null)
  float64 (Real) = (null)
  string (String) = (null)
  large_string (String) = (null)
  timestamp_ms_gmt (DateTime) = 2019/01/01 14:00:00+00
  timestamp_ms_gmt_plus_2 (DateTime) = 2019/01/01 14:00:00+02
  timestamp_ms_gmt_minus_0215 (DateTime) = 2019/01/01 14:00:00-0215
  timestamp_s_no_tz (DateTime) = 2019/01/01 14:00:00
  time32_s (Time) = 00:00:03
  time32_ms (Time) = 00:00:00.003
  time64_us (Integer64) = 3
  time64_ns (Integer64) = 3
  date32 (Date) = 1970/01/04
  date64 (Date) = 1970/01/01
  binary (Binary) = 0001
  large_binary (Binary) = 0001
  fixed_size_binary (Binary) = 0001
  decimal128 (Real) = (null)
  decimal256 (Real) = (null)
  list_boolean (IntegerList(Boolean)) = (null)
  list_uint8 (IntegerList) = (null)
  list_int8 (IntegerList) = (null)
  list_uint16 (IntegerList) = (null)
  list_int16 (IntegerList) = (null)
  list_uint32 (Integer64List) = (null)
  list_int32 (IntegerList) = (null)
  list_uint64 (RealList) = (null)
  list_int64 (Integer64List) = (null)
  list_float32 (RealList(Float32)) = (null)
  list_float64 (RealList) = (null)
  list_string (StringList) = (null)
  fixed_size_list_boolean (IntegerList(Boolean)) = (2:1,0)
  fixed_size_list_uint8 (IntegerList) = (2:4,5)
  fixed_size_list_int8 (IntegerList) = (2:4,5)
  fixed_size_list_uint16 (IntegerList) = (2:4,5)
  fixed_size_list_int16 (IntegerList) = (2:4,5)
  fixed_size_list_uint32 (Integer64List) = (2:4,5)
  fixed_size_list_int32 (IntegerList) = (2:4,5)
  fixed_size_list_uint64 (RealList) = (2:4,5)
  fixed_size_list_int64 (Integer64List) = (2:4,5)
  fixed_size_list_float32 (RealList(Float32)) = (2:4.0,5.0)
  fixed_size_list_float64 (RealList) = (2:4,5)
  fixed_size_list_string (StringList) = (2:e,f)
  struct_field.a (Integer64) = 1
  struct_field.b (Real) = 2.5
  struct_field.c.d (String) = e
  struct_field.c.f (String) = g
  struct_field.h (Integer64List) = (2:5,6)
  struct_field.i (Integer64) = 3
  map_boolean (String(JSON)) = (null)
  map_uint8 (String(JSON)) = (null)
  map_int8 (String(JSON)) = (null)
  map_uint16 (String(JSON)) = (null)
  map_int16 (String(JSON)) = (null)
  map_uint32 (String(JSON)) = (null)
  map_int32 (String(JSON)) = (null)
  map_uint64 (String(JSON)) = (null)
  map_int64 (String(JSON)) = (null)
  map_float32 (String(JSON)) = (null)
  map_float64 (String(JSON)) = (null)
  map_string (String(JSON)) = (null)
  dict (Integer) = 2
  POINT (2 2)

OGRFeature(out_SNAPPY):3
  boolean (Integer(Boolean)) = 0
  uint8 (Integer) = 4
  int8 (Integer) = 1
  uint16 (Integer) = 30001
  int16 (Integer(Int16)) = 10000
  uint32 (Integer64) = 3000000001
  int32 (Integer) = 1000000000
  uint64 (Real) = 300000000001
  int64 (Integer64) = 100000000000
  float32 (Real(Float32)) = 4.5
  float64 (Real) = 4.5
  string (String) = c
  large_string (String) = c
  timestamp_ms_gmt (DateTime) = 2019/01/01 14:00:00+00
  timestamp_ms_gmt_plus_2 (DateTime) = 2019/01/01 14:00:00+02
  timestamp_ms_gmt_minus_0215 (DateTime) = 2019/01/01 14:00:00-0215
  timestamp_s_no_tz (DateTime) = 2019/01/01 14:00:00
  time32_s (Time) = 00:00:04
  time32_ms (Time) = 00:00:00.004
  time64_us (Integer64) = 4
  time64_ns (Integer64) = 4
  date32 (Date) = 1970/01/05
  date64 (Date) = 1970/01/01
  binary (Binary) = 0001
  large_binary (Binary) = 0001
  fixed_size_binary (Binary) = 0001
  decimal128 (Real) = 1234.567
  decimal256 (Real) = 1234.567
  list_boolean (IntegerList(Boolean)) = (3:0,0,1)
  list_uint8 (IntegerList) = (3:0,4,5)
  list_int8 (IntegerList) = (3:0,4,5)
  list_uint16 (IntegerList) = (3:0,4,5)
  list_int16 (IntegerList) = (3:0,4,5)
  list_uint32 (Integer64List) = (3:0,4,5)
  list_int32 (IntegerList) = (3:0,4,5)
  list_uint64 (RealList) = (3:0,4,5)
  list_int64 (Integer64List) = (3:0,4,5)
  list_float32 (RealList(Float32)) = (3:nan,4.5,5.5)
  list_float64 (RealList) = (3:nan,4.5,5.5)
  list_string (StringList) = (3:A,BC,CDE)
  fixed_size_list_boolean (IntegerList(Boolean)) = (2:0,1)
  fixed_size_list_uint8 (IntegerList) = (2:6,7)
  fixed_size_list_int8 (IntegerList) = (2:6,7)
  fixed_size_list_uint16 (IntegerList) = (2:6,7)
  fixed_size_list_int16 (IntegerList) = (2:6,7)
  fixed_size_list_uint32 (Integer64List) = (2:6,7)
  fixed_size_list_int32 (IntegerList) = (2:6,7)
  fixed_size_list_uint64 (RealList) = (2:6,7)
  fixed_size_list_int64 (Integer64List) = (2:6,7)
  fixed_size_list_float32 (RealList(Float32)) = (2:6.0,7.0)
  fixed_size_list_float64 (RealList) = (2:6,7)
  fixed_size_list_string (StringList) = (2:g,h)
  struct_field.a (Integer64) = 1
  struct_field.b (Real) = 2.5
  struct_field.c.d (String) = e
  struct_field.c.f (String) = g
  struct_field.h (Integer64List) = (2:5,6)
  struct_field.i (Integer64) = 3
  map_boolean (String(JSON)) = {}
  map_uint8 (String(JSON)) = {}
  map_int8 (String(JSON)) = {}
  map_uint16 (String(JSON)) = {}
  map_int16 (String(JSON)) = {}
  map_uint32 (String(JSON)) = {}
  map_int32 (String(JSON)) = {}
  map_uint64 (String(JSON)) = {}
  map_int64 (String(JSON)) = {}
  map_float32 (String(JSON)) = {}
  map_float64 (String(JSON)) = {}
  map_string (String(JSON)) = {}
  dict (Integer) = (null)
  POINT (3 2)

OGRFeature(out_SNAPPY):4
  boolean (Integer(Boolean)) = 1
  uint8 (Integer) = 5
  int8 (Integer) = 2
  uint16 (Integer) = 40001
  int16 (Integer(Int16)) = 20000
  uint32 (Integer64) = 4000000001
  int32 (Integer) = 2000000000
  uint64 (Real) = 400000000001
  int64 (Integer64) = 200000000000
  float32 (Real(Float32)) = 5.5
  float64 (Real) = 5.5
  string (String) = d
  large_string (String) = d
  timestamp_ms_gmt (DateTime) = 2019/01/01 14:00:00+00
  timestamp_ms_gmt_plus_2 (DateTime) = 2019/01/01 14:00:00+02
  timestamp_ms_gmt_minus_0215 (DateTime) = 2019/01/01 14:00:00-0215
  timestamp_s_no_tz (DateTime) = 2019/01/01 14:00:00
  time32_s (Time) = 00:00:05
  time32_ms (Time) = 00:00:00.005
  time64_us (Integer64) = 5
  time64_ns (Integer64) = 5
  date32 (Date) = 1970/01/06
  date64 (Date) = 1970/01/01
  binary (Binary) = 0001
  large_binary (Binary) = 0001
  fixed_size_binary (Binary) = 0001
  decimal128 (Real) = -1234.567
  decimal256 (Real) = -1234.567
  list_boolean (IntegerList(Boolean)) = (4:1,0,1,0)
  list_uint8 (IntegerList) = (4:0,7,8,9)
  list_int8 (IntegerList) = (4:0,7,8,9)
  list_uint16 (IntegerList) = (4:0,7,8,9)
  list_int16 (IntegerList) = (4:0,7,8,9)
  list_uint32 (Integer64List) = (4:0,7,8,9)
  list_int32 (IntegerList) = (4:0,7,8,9)
  list_uint64 (RealList) = (4:0,7,8,9)
  list_int64 (Integer64List) = (4:0,7,8,9)
  list_float32 (RealList(Float32)) = (4:nan,7.5,8.5,9.5)
  list_float64 (RealList) = (4:nan,7.5,8.5,9.5)
  list_string (StringList) = (4:A,BC,CDE,DEFG)
  fixed_size_list_boolean (IntegerList(Boolean)) = (2:1,0)
  fixed_size_list_uint8 (IntegerList) = (2:8,9)
  fixed_size_list_int8 (IntegerList) = (2:8,9)
  fixed_size_list_uint16 (IntegerList) = (2:8,9)
  fixed_size_list_int16 (IntegerList) = (2:8,9)
  fixed_size_list_uint32 (Integer64List) = (2:8,9)
  fixed_size_list_int32 (IntegerList) = (2:8,9)
  fixed_size_liARROW: Memory pool: bytes_allocated = 12800
ARROW: Memory pool: max_memory = 32896
GDAL: GDALClose(out_SNAPPY.parquet, this=00000211A70E3DA0)
GDAL: In GDALDestroy - unloading GDAL shared library.
st_uint64 (RealList) = (2:8,9)
  fixed_size_list_int64 (Integer64List) = (2:8,9)
  fixed_size_list_float32 (RealList(Float32)) = (2:8.0,9.0)
  fixed_size_list_float64 (RealList) = (2:8,9)
  fixed_size_list_string (StringList) = (2:i,j)
  struct_field.a (Integer64) = 1
  struct_field.b (Real) = 2.5
  struct_field.c.d (String) = e
  struct_field.c.f (String) = g
  struct_field.h (Integer64List) = (2:5,6)
  struct_field.i (Integer64) = 3
  map_boolean (String(JSON)) = {}
  map_uint8 (String(JSON)) = {}
  map_int8 (String(JSON)) = {}
  map_uint16 (String(JSON)) = {}
  map_int16 (String(JSON)) = {}
  map_uint32 (String(JSON)) = {}
  map_int32 (String(JSON)) = {}
  map_uint64 (String(JSON)) = {}
  map_int64 (String(JSON)) = {}
  map_float32 (String(JSON)) = {}
  map_float64 (String(JSON)) = {}
  map_string (String(JSON)) = {}
  dict (Integer) = 2
  POINT (4 2)

0

comment:4 by andreaerdna, 2 years ago

In my Windows 10 Home 64-bit system (AMD A8-6410 APU with AMD Radeon R5 Graphics), the ogrinfo.exe and arrow.dll crash is reported in Event Viewer as an Application Error for application ogrinfo.exe and module arrow.dll with Exception code: 0xc000001d (the same as reported at ​https://github.com/qgis/QGIS/issues/49148#issuecomment-1172286191 for tiledb.dll by another user).

Full Event Viewer errors report at ​https://gist.github.com/agiudiceandrea/1bdd64612dbecb793d885f2c966ae6d7

CPU-Z reports the following instructions sets for AMD A8-6410: MMX (+), SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, SSE4A, x86-64, AMD-V, AES, AVX

ogrinfo out_SNAPPY.parquet out_SNAPPY --debug ON crashes after the dict: Integer (0.0), domain name=dictDomain string is displayed, just before the first OGRFeature report, while it doesn't crash for out_BROTLI.parquet or out_GZIP.parquet or out_ZSTD.parquet.

The full log of run.bat (with ogrinfo -fid 0 out_%%c.parquet out_%%c instead of ogrinfo out_%%c.parquet out_%%c in the OSGeo4W Shell is available at ​https://gist.github.com/agiudiceandrea/87baa9e4c60ff5e4f3e1a1da6f31802b.

Version 0, edited 2 years ago by andreaerdna (next)

in reply to:  4 ; comment:5 by jef, 2 years ago

Replying to andreaerdna:

CPU-Z reports the following instructions sets for AMD A8-6410: MMX (+), SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, SSE4A, x86-64, AMD-V, AES, AVX

Does setting the environment variable ARROW_USER_SIMD_LEVEL to NONE make a difference on your machine?

in reply to:  5 ; comment:6 by andreaerdna, 2 years ago

Replying to jef:

Replying to andreaerdna:

CPU-Z reports the following instructions sets for AMD A8-6410: MMX (+), SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, SSE4A, x86-64, AMD-V, AES, AVX

Does setting the environment variable ARROW_USER_SIMD_LEVEL to NONE make a difference on your machine?

No difference with set ARROW_USER_SIMD_LEVEL=NONE and also with any other value specified at ​https://arrow.apache.org/docs/dev/cpp/env_vars.html#envvar-ARROW_USER_SIMD_LEVEL (apart from "SSE4.2" which should be actually "SSE4_2").

in reply to:  6 ; comment:7 by jef, 2 years ago

Replying to andreaerdna:

No difference with set ARROW_USER_SIMD_LEVEL=NONE and also with any other value specified at ​https://arrow.apache.org/docs/dev/cpp/env_vars.html#envvar-ARROW_USER_SIMD_LEVEL (apart from "SSE4.2" which should be actually "SSE4_2").

Please try arrow-cpp-7.0.0-2.

in reply to:  7 comment:8 by andreaerdna, 2 years ago

Replying to jef:

Please try arrow-cpp-7.0.0-2.

Unfortunately, the same issue occurs on my system also with arrow-cpp-7.0.0-2.

comment:9 by jef, 2 years ago

Resolution: → fixed
Status: new → closed

actually is was snappy itself (builds with BMI2 by default). Fixed in snappy-1.1.9-2 (and rebuilt arrow-cpp-7.0.0-5)

Last edited 2 years ago by jef (previous) (diff)

comment:10 by andreaerdna, 2 years ago

Thanks. Confirmed fixed with arrow-cpp-7.0.0-5.

Note: See TracTickets for help on using tickets.