Opened 10 years ago

Closed 6 years ago

#2112 closed defect (fixed)

Field value string representation for NaN

Reported by: cbutlerami Owned by: warmerdam
Priority: normal Milestone: 1.8.1
Component: OGR_SF Version: 1.4.4
Severity: normal Keywords: ogdi
Cc:

Description

I have some standard DNC/VPF datasets that use NaN for some real field values. When the data is inserted into PostgreSQL by ogr2ogr, there is an error because Postgres does not recognize the string generated by OGRFeature::GetFieldAsString? for a real field type that has NaN as the value.

I believe PostgreSQL (at least) wants the string value 'NaN' (some variation of nan in single quotes).

Change History (4)

comment:1 Changed 9 years ago by warmerdam

Keywords: ogdi added

Clive,

Can you point me to a dataset we can download that demonstrates this? This should be caught at the OGR OGDI driver, and NaN's turned into NULL fields. But I'm hesitant to do the change without a way of testing it.

comment:2 Changed 9 years ago by cbutlerami

http://www.nga.mil/portal/site/dnc/

Select DNCDX017 from the region combobox, then choose the Coastal library and download the coa17a dataset (about 1MByte I think).

Unzip onto a DOS-compatible filesystem (case-insensitive) and try to insert into a PostgreSQL database ("ogr2ogr -f PostgreSQL PG:dbname=dnc gltp:/vrf/mnt/1/coa17a" or something like that)

I get:

GET_TABLE_ELEMENT: Invalid field number -1 ERROR 1: INSERT command for new feature failed. ERROR: column "nan" does not exist LINE 1: ...9688)'::TEXT) , 1, 'BE020', 1, 26, 1, nan, ...

Command: INSERT INTO "soundp@hyd(*)_point" (wkb_geometry , "id", "f_code", "acc", "dat", "exs", "hdh", "hdp", "snd", "svc", "val", "tile_id", "end_id") VALUES (GeomFromEWKT('SRID=32767;POINT (-75.005378723144531 37.962081909179688)'::TEXT) , 1, 'BE020', 1, 26, 1, nan, 23.700001, 10, 5, 0, 3, 1) ERROR 1: Terminating translation prematurely after failed translation of layer soundp@hyd(*)_point

Hacking OGRFeature::GetFieldAsString? to do an isNan() check and translate to the quoted string 'NaN' fixes the problem.

Hope this helps!

comment:3 Changed 6 years ago by cbutlerami

Ticket #3667 covers the same topic (NaN handling), but does not entirely fix it for me.

Specifically, #3667 fixes the Postgres driver to recognize 'nan' as a special value from GetFieldAsString?(). Unfortunately, GetFieldAsString?() applies a field width so that the actual returned value is ' nan', which is not recognized as 'nan' due to the use of strcmp().

Perhaps the correct fix is to have GetFieldAsString? test for NaN using CPLIsNan() and return the exact string that the Postgres driver expects, without allowing snprintf() to be involved.

i.e.:

... else if( poFDefn->GetType?() == OFTReal) {

if (CPLIsNan(pauFields[iField].Real)) {

return "nan";

}

...

Similar tests should probably also be added for the +/- infinity cases.

Alternatively, these tests could be moved to the Postgres driver. String comparisons should not be used due to the possibility that not all implementations of snprintf will return the same characters (i.e. NaN vs nan, Inf vs Infinity, etc.).

comment:4 Changed 6 years ago by Even Rouault

Milestone: 1.8.1
Resolution: fixed
Status: newclosed

Fixed in trunk (r21628) and in branches/1.8 (r21629)

Note: See TracTickets for help on using tickets.