Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#5377 closed defect (fixed)

Invalid JSON output sometimes occurs with `"coordinates": [ , ]`

Reported by: Mr0grog Owned by: warmerdam
Priority: normal Milestone: 1.11.0
Component: OGR_SF Version: 1.10.1
Severity: normal Keywords: geojson
Cc:

Description

I tried running the shapefile found here:

https://github.com/CityOfPhiladelphia/phl-open-geodata/tree/master/bicycle_thefts

through ogr2ogr with the following command:

ogr2ogr -f GeoJSON output.json bicycle_thefts/shapefile/bicycle_thefts.shp

But get invalid JSON as output. Specifically, several features are missing numbers in their coordinates array, so they look like this:

... "geometry": { "type": "Point", "coordinates": [ , ] } ...

I'm not sure if there's bad data in the underlying shapefile or if something else is going wrong, but it would at least be nice to get a valid file out the other end (or an error message).

Attachments (1)

bicycle_thefts_shapefile.zip (348.4 KB) - added by Mr0grog 4 years ago.
Shapefile that results in bad JSON output

Download all attachments as: .zip

Change History (6)

Changed 4 years ago by Mr0grog

Shapefile that results in bad JSON output

comment:1 Changed 4 years ago by Jukka Rahkonen

Hi,

Do you mean that there should be { "geometry": null } in the result instead? GeoJSON specification says: "A feature object must have a member with the name "geometry". The value of the geometry member is a geometry object as defined above or a JSON null value.

Here is ogrinfo of one problematic point in the shapefile, DC_NUM='036081'

C:\temp\todelete>ogrinfo -al bicycle_thefts.shp -where DC_NUM='036081' INFO: Open of `bicycle_thefts.shp'

using driver `ESRI Shapefile' successful.

Layer name: bicycle_thefts Geometry: Point Feature Count: 1 Extent: (-75.273013, 39.884778) - (-74.959503, 40.131376) Layer SRS WKT: GEOGCS["GCS_WGS_1984",

DATUM["WGS_1984",

SPHEROID["WGS_84",6378137.0,298.257223563]],

PRIMEM["Greenwich",0.0], UNIT["Degree",0.0174532925199433]]

DC_NUM: String (6.0) DC_KEY: String (12.0) LOCATION_B: String (45.0) THEFT_DATE: Date (10.0) THEFT_YEAR: Integer (4.0) DC_DIST: Integer (4.0) STOLEN_VAL: Integer (4.0) THEFT_HOUR: Integer (4.0) UCR: Integer (4.0) LAT: Real (10.7) LNG: Real (19.11) OGRFeature(bicycle_thefts):7215

DC_NUM (String) = 036081 DC_KEY (String) = 201318036081 LOCATION_B (String) = S 219 S 52ND ST THEFT_DATE (Date) = 2013/06/21 THEFT_YEAR (Integer) = 2013 DC_DIST (Integer) = 18 STOLEN_VAL (Integer) = 200 THEFT_HOUR (Integer) = 14 UCR (Integer) = 615 LAT (Real) = 0.0000000 LNG (Real) = 0.00000000000 POINT (too_big too_big)

comment:2 Changed 4 years ago by Mr0grog

Do you mean that there should be { "geometry": null } in the result instead?

Probably! I wasn't really sure whether the correct output would be:

{ "geometry": null }

or

{ "geometry": { "type": "Point", "coordinates": [ 0, 0 ] } }

or something else entirely.

I don't have strong desire for a particular solution; mostly I'd just like the output to work in a JSON parser (the current output gives syntax errors—an "unexpected comma" because there's no actual number in the array). That said, I think null would be entirely reasonable here.

comment:3 Changed 4 years ago by Jukka Rahkonen

Hi,

In this shapefile it looks like LAN/LON coordinates with value 0 have converted into a very large negative integer. If I convert the original shapefile into a new one as ogr2ogr -f "ESRI Shapefile" clean.shp bicycle_thefts.shp ogrinfo shows extents of "clean.shp"

Layer name: clean Geometry: Point Feature Count: 7223 Extent: (-179769313486231570000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000.000000, -17976931348623157000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 00000000000000000000000000000000000000.000000) - (-74.959503, 40.131376)

Also other programs than GDAL has troubles with this shapefile, for example OpenJUMP. City of Philadelphia should really consider to change their process. But GDAL should also try to avoid creating invalid GeoJSON.

comment:4 Changed 4 years ago by Even Rouault

Component: defaultOGR_SF
Keywords: geojson added
Milestone: 2.0
Resolution: fixed
Status: newclosed

trunk r26926 "GeoJSON output: fix handling of huge coordinates (#5377)"

After fix, the following valid GeoJSON is produced :

$ ogr2ogr -f geojson /vsistdout/ /vsizip/bicycle_thefts_shapefile.zip  -fid 7215
{
"type": "FeatureCollection",
"features": [
{ "type": "Feature", "properties": { "DC_NUM": "036081", "DC_KEY": "201318036081", "LOCATION_B": "S 219 S 52ND ST", "THEFT_DATE": "2013\/06\/21", "THEFT_YEAR": 2013, "DC_DIST": 18, "STOLEN_VAL": 200, "THEFT_HOUR": 14, "UCR": 615, "LAT": 0.000000, "LNG": 0.000000 }, "geometry": { "type": "Point", "coordinates": [ -1.79769313486231571e+308, -1.79769313486231571e+308 ] } }
]
}

comment:5 Changed 4 years ago by Even Rouault

Milestone: 2.01.11.0
Note: See TracTickets for help on using tickets.