Opened 10 years ago

Closed 4 years ago

#4376 closed defect (fixed)

GDAL does not work when include 2-byte letters

Reported by: koba Owned by: warmerdam
Priority: normal Milestone:
Component: default Version: unspecified
Severity: normal Keywords:
Cc:

Description

Both from QGIS v1.7.2 and SAGA v2.0.7,

GDAL module does not work if the command strings include 2-byte folder name or 2-byte file name.

Attachments (7)

English.zip (852.5 KB) - added by koba 10 years ago.
日本語.zip (852.5 KB) - added by koba 10 years ago.
GDAL_2byte_1.zip (606.4 KB) - added by koba 10 years ago.
GDAL_2byte_6.zip (675.0 KB) - added by koba 10 years ago.
cmd.gif (24.1 KB) - added by koba 10 years ago.
English2.tif (480.7 KB) - added by koba 10 years ago.
日本語2.tif (480.7 KB) - added by koba 10 years ago.

Change History (12)

comment:1 Changed 10 years ago by warmerdam

Priority: highnormal

Please be specific about the environment, and try to give us something specific we could use to reproduce the problem.

I'm guessing you are using GDAL 1.8.0 or newer on some variant of windows?

There was a substantial effort in gdal 1.8 to support filenames with unicode characters in them. On windows the default build configuration for GDAL now requires that filenames be passed in to GDLAOpen() in UTF8 and they are internally converted to double byte unicode characters and passed to a wide-char version of the open call. So one question is whether the proper utf-8 is getting passed in. It may be helpful to try and debug this with the commandline tools, and/or something (like a Python script) that allows you to use real unicode strings.

It would be helpful if you could attach a zip file with a small file you are unable to open but ought to be able to.

Changed 10 years ago by koba

Attachment: English.zip added

Changed 10 years ago by koba

Attachment: 日本語.zip added

Changed 10 years ago by koba

Attachment: GDAL_2byte_1.zip added

Changed 10 years ago by koba

Attachment: GDAL_2byte_6.zip added

comment:2 in reply to:  1 Changed 10 years ago by koba

I attach a zip file includes 2 raster files (1byte:English.tif and 2byte:日本語.tif) and 6 hard copies. I just downloaded and installed the newest QGIS v1.7.2 (setup version) without any plugins. The QGIS shows version of the GDAL is v1.81(fig.1). fig.2 and 3 are when I tried to warp an Japanese(2byte) name file. It fails. fig.4 and 5 are when I tried to warp an English(1byte) name file. It succeeds. fig.6 is the files list after warp. Only 1byte file was warped.

The GDAL module also fails if the target file exists under 2-byte folders.

Replying to warmerdam:

Please be specific about the environment, and try to give us something specific we could use to reproduce the problem.

I'm guessing you are using GDAL 1.8.0 or newer on some variant of windows?

There was a substantial effort in gdal 1.8 to support filenames with unicode characters in them. On windows the default build configuration for GDAL now requires that filenames be passed in to GDLAOpen() in UTF8 and they are internally converted to double byte unicode characters and passed to a wide-char version of the open call. So one question is whether the proper utf-8 is getting passed in. It may be helpful to try and debug this with the commandline tools, and/or something (like a Python script) that allows you to use real unicode strings.

It would be helpful if you could attach a zip file with a small file you are unable to open but ought to be able to.

comment:3 Changed 10 years ago by warmerdam

I've reviewed the screen snaps and I see that the gdalwarp command is failing when launched from QGIS. It is unfortunately not completely clear to me yet what operating system you are working on or what version of GDAL you are using.

Also, unfortunately the contents of 日本語.zip are corrupted (filename-wise) when I unzip them on my linux workstation. I am guessing this is an issue with the unzip command. The resulting file is:

  -rw-r----- 1 warmerdam eng 1180736 2011-07-20 15:52 ???{??.tif

It would be helpful if you could be specific about OS and demonstrate the issue directly invoking GDAL from the commandline instead of doing it through QGIS.

Changed 10 years ago by koba

Attachment: cmd.gif added

Changed 10 years ago by koba

Attachment: English2.tif added

Changed 10 years ago by koba

Attachment: 日本語2.tif added

comment:4 Changed 10 years ago by koba

I am using Windows7 Professional 64bit Japanese version.

I attach a hardcopy of command line trial.

I also attach not compressed tif files both with 1byte and 2byte names.

comment:5 Changed 4 years ago by Jukka Rahkonen

Resolution: fixed
Status: newclosed

With gdal 2.3dev on Windows this command succeeds probably because of fix in #7065:

gdal_translate 日本語2.tif out.tif.

I had to copy-paste the filename from file manager into command window which shows the file name with my Finnish settings just as ???2.tif but that is obviously another problem.

Note: See TracTickets for help on using tickets.