Opened 5 weeks ago

Closed 5 weeks ago

Last modified 5 weeks ago

#7146 closed defect (fixed)

vsis3: VSIFStatL only works with key with unicode character if ReadDir is run first.

Reported by: tveastman Owned by: warmerdam
Priority: normal Milestone: 2.2.3
Component: default Version: unspecified
Severity: normal Keywords: vsis3 VSIFstatL unicode
Cc: robert.coup@…

Description (last modified by tveastman)

VSIFStatL() doesn't work on this path (An S3 key named b/ü.txt) :

>>> gdal.VSIStatL(u'/vsis3/public-bucket-gdal-vsis3-tests/b/\xfc.txt')
None

However, it *does* work if the bucket has been cached using ReadDirRecursive() first.

>>> gdal.ReadDirRecursive(u'/vsis3/public-bucket-gdal-vsis3-tests')
['a/', 'a/a.txt', 'b/', 'b/+.txt', 'b/_.txt', u'b/\xfc.txt']
>>> gdal.VSIStatL(u'/vsis3/public-bucket-gdal-vsis3-tests/b/\xfc.txt')
<osgeo.gdal.StatBuf; proxy of <Swig Object of type 'StatBuf *' at 0x7f16f72c6cf0> >

In the interest of helping shake out some more of these /vsis3/ bugs, I'm creating a pytest test suite that runs against a public (world readable) S3 bucket. I won't be able to keep it there forever, but I hope it's useful.

My test suite is at https://github.com/koordinates/gdal-vsis3-tests/

The bucket is called 'public-bucket-gdal-vsis3-tests'. You should be able to access it too.

I've also found that the VSIFOpenEx() method can't open files with similarly weird name S3 keys. I'll be adding a test case for that and filing a bug tomorrow, but I'll let you know now in case you can look into it before I've documented it better.

Cheers!

Change History (5)

comment:1 Changed 5 weeks ago by tveastman

Description: modified (diff)

comment:2 Changed 5 weeks ago by tveastman

Description: modified (diff)

comment:3 Changed 5 weeks ago by Even Rouault

Resolution: fixed
Status: newclosed

In 40694:

/vsis3/: properly handle unicode characters for VSIFOpenL() / VSIStatL() (fixes #7146)

comment:4 Changed 5 weeks ago by Even Rouault

In 40695:

/vsis3/: properly handle unicode characters for VSIFOpenL() / VSIStatL() (fixes #7146)

comment:5 Changed 5 weeks ago by Even Rouault

Milestone: 2.2.3
Note: See TracTickets for help on using tickets.