GDAL does not work when include 2-byte letters

[koba]

Both from QGIS v1.7.2 and SAGA v2.0.7,

GDAL module does not work if the command strings include 2-byte folder name or 2-byte file name.

[warmerdam]

Please be specific about the environment, and try to give us something specific we could use to reproduce the problem.

I'm guessing you are using GDAL 1.8.0 or newer on some variant of windows?

There was a substantial effort in gdal 1.8 to support filenames with unicode characters in them. On windows the default build configuration for GDAL now requires that filenames be passed in to GDLAOpen() in UTF8 and they are internally converted to double byte unicode characters and passed to a wide-char version of the open call. So one question is whether the proper utf-8 is getting passed in. It may be helpful to try and debug this with the commandline tools, and/or something (like a Python script) that allows you to use real unicode strings.

It would be helpful if you could attach a zip file with a small file you are unable to open but ought to be able to.

[koba]

I attach a zip file includes 2 raster files (1byte:English.tif and 2byte:日本語.tif) and 6 hard copies. I just downloaded and installed the newest QGIS v1.7.2 (setup version) without any plugins. The QGIS shows version of the GDAL is v1.81(fig.1). fig.2 and 3 are when I tried to warp an Japanese(2byte) name file. It fails. fig.4 and 5 are when I tried to warp an English(1byte) name file. It succeeds. fig.6 is the files list after warp. Only 1byte file was warped.

The GDAL module also fails if the target file exists under 2-byte folders.

Replying to warmerdam:

Please be specific about the environment, and try to give us something specific we could use to reproduce the problem. I'm guessing you are using GDAL 1.8.0 or newer on some variant of windows? There was a substantial effort in gdal 1.8 to support filenames with unicode characters in them. On windows the default build configuration for GDAL now requires that filenames be passed in to GDLAOpen() in UTF8 and they are internally converted to double byte unicode characters and passed to a wide-char version of the open call. So one question is whether the proper utf-8 is getting passed in. It may be helpful to try and debug this with the commandline tools, and/or something (like a Python script) that allows you to use real unicode strings. It would be helpful if you could attach a zip file with a small file you are unable to open but ought to be able to.

[warmerdam]

I've reviewed the screen snaps and I see that the gdalwarp command is failing when launched from QGIS. It is unfortunately not completely clear to me yet what operating system you are working on or what version of GDAL you are using.

Also, unfortunately the contents of 日本語.zip are corrupted (filename-wise) when I unzip them on my linux workstation. I am guessing this is an issue with the unzip command. The resulting file is:

  -rw-r----- 1 warmerdam eng 1180736 2011-07-20 15:52 ???{??.tif

It would be helpful if you could be specific about OS and demonstrate the issue directly invoking GDAL from the commandline instead of doing it through QGIS.

[koba]

I am using Windows7 Professional 64bit Japanese version.

I attach a hardcopy of command line trial.

I also attach not compressed tif files both with 1byte and 2byte names.