Opened 14 years ago

Closed 13 years ago

#509 closed defect (fixed)

Invalid Charset Encoding breaks Win32 GUI

Reported by: pramsey Owned by: pramsey
Priority: medium Milestone: PostGIS 1.5.2
Component: postgis Version: 1.5.X
Keywords: Cc:


When running the attached shape file through the Win32 GUI, the charset has invalid characters (?) according to the log (though, really, something else is probably going wrong, like things not being transcoded at all, since all the bytes in the stream can be transcoded from WIN1252, they just might get transcoded to the wrong codepoint). Anyhow, things break. Probably we need to expose the new "ignore failures" option and also review whether things are even being transcoded at all, and possibly consider aborting the whole translation when a failure is hit: as it stands, the GUI just starts logging piles of errors.

Attachments (1) (195.6 KB ) - added by pramsey 14 years ago.
Problem shape file

Download all attachments as: .zip

Change History (11)

by pramsey, 14 years ago

Attachment: added

Problem shape file

comment:1 by pramsey, 14 years ago

Summary: Invalid Charset Encoding breaks GUIInvalid Charset Encoding breaks Win32 GUI

I just checked and this file works perfectly well in OS/X shp2pgsql-gui under X windows.

comment:2 by pramsey, 14 years ago

Works in OS/X quartz gtk too, though it takes 10x as long to load! Confusing!

comment:3 by robe, 14 years ago

This might be a different problem Paul, but this file crashes my windows shp2pgsql commandline even without any encoding args. Thought the gui doesn't crash just aborts the transaction.

I admit don't have the latest 1.5.2 SVN version handy at the moment.

Using with an older shppg2sql works fine.

Doing a diff between the 2 before the new version crashed and burned

It breaks at a NULL geometry

This line — well rather this is the next line in my old shp2pgsql that follows the new crash

The new shp2pgsql crashes as its trying to spit this out I guess.

INSERT INTO "nyc_bike_racks" ("id","housenumbe","street","rack_small","rack_large",the_geom) 
VALUES ('4756','295','Warren St','1','0',NULL);

comment:4 by pramsey, 14 years ago

That's good news, it should be easier to debug the command-line.

comment:5 by pramsey, 14 years ago

No such luck, runs fine w/ my shp2pgsql.exe from latest 1.5 branch. I think the NULL problem was fixed in an earlier patch, but that still seems to leave a different problem.

comment:6 by robe, 14 years ago

Paul, Okay I compiled from 1.5 branch and tested on my windows xp box and I don't seem to have any problems with either commandline or the GUI any more. (I think I was getting errors with my previous version of GUI wiht bunch of aborts but didn't really pay attention to the log)

GUI says successful and no comments about invalid character sets in the log.

When I look at my about box - it says Revision 5571. That's the one you are running too right?

Also the encoding says WINDOWS-1252 — that is the same one you have right?

comment:7 by robe, 14 years ago


You still consider this an issue? Like I said this dataset works fine for me on windows with the latest 1.5.2 SVN. Unless I'm missing something.

comment:8 by pramsey, 14 years ago

Yes, I need to test again, because my last build test using the latest SVN still yielded a crashing result. Another possibility is that the problem is the combination of the mingw built binary with the vcc build libiconv in the edb binary package.

comment:9 by robe, 14 years ago

I don't think its the vcc build libiconv at fault.

I think I had replaced my local postgresql bin with the mingw, but I just tried with the one that gets packaged with PgAdminIII which is the way I normally use it and it works fine with that one. Besides I would expect we would have similar issues with the command line if that were the case.

Are you sure your source directory isn't dirty with some old files.

For perspective I pulled out an old copy of a shp2pgsql-gui and it gives this error with your file.

Failed in pgui_exec(): ERROR:  invalid byte sequence for encoding "UTF8": 0xb3

I presume that is the error you are getting? I thought maybe it was the encoding of the database that would make a difference — but works fine for me against UTF8 encoded as well as win1252 encoded database.

comment:10 by pramsey, 13 years ago

Resolution: fixed
Status: newclosed

I haven't seen this one since in Win builds during our testing, so I'm shutting it down.

Note: See TracTickets for help on using tickets.