SHP Coordinate Problems

[dengo]

Hi,

We have reports of a very strange problem with shapefiles.

After users have edited a shapefile and then reloaded it later on, the geometries of some features have bad coordinates.

This happened quite often in the last weeks, but we still have not been able to reproduce it. Therefore we took a closer look at the bad coordinates and noticed something interesting.

I've attached a shapefile with the problem. It contains 3 features with polygon geometries. The first feature has the coordinate problems in the second polygon of its two-part multipolygon (in other cases the feature with the problem was not a multipolygon, so this is not indicative of the problem).

feature1_problem1.PNG shows the first instance where the coordinates are problematic: Starting with vertex 58 (X) the coordinates have strange values and starting with vertex 61 (Y) seemingly normal coordinates follow but with X and Y flipped. At a closer look these are exactly the coordinates of the second feature in the shapefile (see feature2_coords.PNG).

feature1_problem2.PNG shows another problem starting at vertex 77 (Y). From vertex 81 (X) the coordinates of the third feature follow (see feature3_coords.PNG).

Having another look at the strange double values, we realized that those are in fact the bytes of the ShapeBin Header of the corresponding inserted geometries (except for the first strange double value).

Our best guess is that there is a bug within the shapefile driver, either when normally updating the shapefile or perhaps more likely when the repack is performed.

As I said, sadly we have not been able to find a way to reproduce it, but it happens quite often. The users might have done a very specific combination of feature inserts/updates and deletions. We have AUTO_REPACK disabled and the repack is only performed once (ExecuteSQL) when the layer is unloaded. I'm creating this issue in the hopes you might have any idea how something like this can happen.

I'll update this issue if we get any new information.

Kind regards, Dennis

[Even Rouault]

In 40016:

CPLStrtod(): parse string like '-1.#IND0000000' as NaN instead of -1 (seen when looking at refs #7031, but does not fix it)

[Even Rouault]

In 40017:

CPLStrtod(): parse string like '-1.#IND0000000' as NaN instead of -1 (seen when looking at refs #7031, but does not fix it)

[Even Rouault]

In 40018:

Shape: fix regression affecting GDAL 2.1.3 or later, 2.2.0 or later, when editing the last shape of a file and growing it, and then appending a new shape, which resulted in corruption of both shapes (fixes #7031)

[Even Rouault]

In 40019:

Shape: fix regression affecting GDAL 2.1.3 or later, 2.2.0 or later, when editing the last shape of a file and growing it, and then appending a new shape, which resulted in corruption of both shapes (fixes #7031)

[Even Rouault]

In 40020:

Shape: fix regression affecting GDAL 2.1.3 or later, 2.2.0 or later, when editing the last shape of a file and growing it, and then appending a new shape, which resulted in corruption of both shapes (fixes #7031)

[Even Rouault]

In 40021:

Shape: fix regression affecting GDAL 2.1.3 or later, 2.2.0 or later, when editing the last shape of a file and growing it, and then appending a new shape, which resulted in corruption of both shapes (fixes #7031)

[Even Rouault]

Dennis, reviewing the code, I found a reproducable scenario that resulted in corruption, which could potentially be the one that affected your users. The above commits fix it.

[dengo]

Thanks Even! I'm amazed how fast you find solutions for such issues. I'm very confident that this was the problem the users encountered. With your description it's pretty easy to reproduce and after the fix this problem no longer occurs.

Thanks again for the great work!