Invalid Charset Encoding breaks Win32 GUI

[pramsey]

When running the attached shape file through the Win32 GUI, the charset has invalid characters according to the log (though, really, something else is probably going wrong, like things not being transcoded at all, since all the bytes in the stream can be transcoded from WIN1252, they just might get transcoded to the wrong codepoint). Anyhow, things break. Probably we need to expose the new "ignore failures" option and also review whether things are even being transcoded at all, and possibly consider aborting the whole translation when a failure is hit: as it stands, the GUI just starts logging piles of errors.

[pramsey]

Problem shape file

[pramsey]

I just checked and this file works perfectly well in OS/X shp2pgsql-gui under X windows.

[pramsey]

Works in OS/X quartz gtk too, though it takes 10x as long to load! Confusing!

[robe]

This might be a different problem Paul, but this file crashes my windows shp2pgsql commandline even without any encoding args. Thought the gui doesn't crash just aborts the transaction.

I admit don't have the latest 1.5.2 SVN version handy at the moment.

Using with an older shppg2sql works fine.

Doing a diff between the 2 before the new version crashed and burned

It breaks at a NULL geometry

This line — well rather this is the next line in my old shp2pgsql that follows the new crash

The new shp2pgsql crashes as its trying to spit this out I guess.

INSERT INTO "nyc_bike_racks" ("id","housenumbe","street","rack_small","rack_large",the_geom) 
VALUES ('4756','295','Warren St','1','0',NULL);

[pramsey]

That's good news, it should be easier to debug the command-line.

[pramsey]

No such luck, runs fine w/ my shp2pgsql.exe from latest 1.5 branch. I think the NULL problem was fixed in an earlier patch, but that still seems to leave a different problem.

[robe]

Paul, Okay I compiled from 1.5 branch and tested on my windows xp box and I don't seem to have any problems with either commandline or the GUI any more. (I think I was getting errors with my previous version of GUI wiht bunch of aborts but didn't really pay attention to the log)

GUI says successful and no comments about invalid character sets in the log.

When I look at my about box - it says Revision 5571. That's the one you are running too right?

Also the encoding says WINDOWS-1252 — that is the same one you have right?

[robe]

Paul,

You still consider this an issue? Like I said this dataset works fine for me on windows with the latest 1.5.2 SVN. Unless I'm missing something.

[pramsey]

Yes, I need to test again, because my last build test using the latest SVN still yielded a crashing result. Another possibility is that the problem is the combination of the mingw built binary with the vcc build libiconv in the edb binary package.

[robe]

I don't think its the vcc build libiconv at fault.

I think I had replaced my local postgresql bin with the mingw, but I just tried with the one that gets packaged with PgAdminIII which is the way I normally use it and it works fine with that one. Besides I would expect we would have similar issues with the command line if that were the case.

Are you sure your source directory isn't dirty with some old files.

For perspective I pulled out an old copy of a shp2pgsql-gui and it gives this error with your file.

Failed in pgui_exec(): ERROR:  invalid byte sequence for encoding "UTF8": 0xb3

I presume that is the error you are getting? I thought maybe it was the encoding of the database that would make a difference — but works fine for me against UTF8 encoded as well as win1252 encoded database.

[pramsey]

I haven't seen this one since in Win builds during our testing, so I'm shutting it down.