Opened 10 years ago

Closed 5 years ago

#3409 closed defect (fixed)

BSB format: use correct +lat_ts value for MERCATOR from KNP/PP=

Reported by: hamish Owned by: warmerdam
Priority: normal Milestone:
Component: GDAL_Raster Version: svn-trunk
Severity: normal Keywords: bsb
Cc: chrisbarker, manimaul

Description

Hi,

trunk/frmts/bsb/bsbdataset.cpp has:

/* -------------------------------------------------------------------- */
/*      Can we derive a reasonable coordinate system definition for     */
/*      this file?  For now we keep it simple, just handling            */
/*      mercator. In the future we should consider others.              */
/* -------------------------------------------------------------------- */

(the above should really be on the frmt's help page)

    else if( EQUALN(pszPR,"PR=MERCATOR", 11) )
    {
        // We somewhat arbitrarily select our first GCPX as our 
        // central meridian.  This is mostly helpful to ensure 
        // that regions crossing the dateline will be contiguous 
        // in mercator.

Instead of using the first REF/ point, this should be using the PP= value in the KNP= section of the header.


thus ends the bug report and begins the RFE:


Likewise, for TRANSVERSE MERCATOR the PP= value represents the +lon_0 term.

I haven't figured out yet what the secondary terms for tmerc and LAMBERT CONFORMAL CONIC will be, mostly because I haven't found a NOAA chart which uses them yet.

Here are some observations:

I am thinking that the KNQ/P1= P2= P3= P4= variables must be there for the other projection definition terms.

Looking at NOAA chart 12300 as a random sample,

  • For mercator, KNQ/P2= is the same as PP= at 40.0, which will be +lat_ts. This matches what is printed along with the map title if I view the .KAP file in QGIS.

For reference, here's it in context:

KNP/SC=400000,GD=NAD83,PR=MERCATOR,PP=40.000,PI=30.000,SP=,SK=0.0000000
    TA=90.0000000,UN=FATHOMS,SD=MEAN LOWER LOW WATER,DX=40.00,DY=40.00
KNQ/EC=RF,GD=NARC,VC=UNKNOWN,SC=MLLW,PC=MC,P1=UNKNOWN,P2=40.000
    P3=NOT_APPLICABLE,P4=NOT_APPLICABLE,GC=NOT_APPLICABLE,RM=INVERSE

I think KNP/PI= has something to do with roughly how much distance 1 pixel covers, anyway here is a scale : PI ratio for a number of NOAA charts:

SC=400000,PI=30.000
SC=80000,PI=10.000
SC=40000,PI=5.000
SC=15000,PI=1.000
SC=10000,PI=1.000

perhaps that's (mean_extent / dpi*0.0254) ??? shrug

here is an example of +proj=poly from NOAA chart 14500:

KNP/SC=1500000,GD=NAD83,PR=POLYCONIC,PP=-83.814,PI=60.000,SP=,SK=359.8576421
    TA=89.8576421,UN=FATHOMS,SD=GREAT LAKES LOW WATER,DX=150.00,DY=150.00
KNQ/EC=RF,GD=NARC,VC=UNKNOWN,SC=GLLW,PC=PH,P1=-83.814,P2=UNKNOWN
    P3=NOT_APPLICABLE,P4=NOT_APPLICABLE,GC=NOT_APPLICABLE,RM=INVERSE

(PP= P1= +lon_0, and PC=PH)

I'm using the GPL OpenCPN.org software for tests vs. the GDAL code, both that only handles +proj=merc right now as well.

a little more info at

http://libbsb.sourceforge.net/bsb_file_format.html

and by reading the c++ code of mc2bsbh ver 0.0.9: (GPL3)

http://www.dacust.com/inlandwaters/mapcal/index.html

I'm still searching for any NOAA lcc and tmerc charts to understand them.

Hamish

Change History (18)

comment:1 Changed 10 years ago by hamish

jonasaberg provided this link to a demo chart from Sweden which uses tmerc.

http://www.hydrographica.se/images/BSB-demo.zip

KNP/SC=10000,GD=WGS84,PR=TRANSVERSE MERCATOR
    PP=18.0582833333333,PI=UNKNOWN,TA=90.0,UN=METERS
    SD=AMSL,SK=0.0,DX=1.25,DY=1.25
KNQ/EC=WE,GD=WGE,VC=MSL,SC=AMSL,GC=UB,RM=INVERSE
    PC=TC,P1=18.0582833333333,P2=1.00,P3=0.00,P4=NOT_APPLICABLE

So PP and P1 look like +lon_0; PC="TC"; P2=(scale factor??); P3=(+lat_0??)

also, this tool from NOAA might be able to manufacture headers for us: http://www.csc.noaa.gov/digitalcoast/tools/chartreprojector/

(BSBv3 has many more options, BSBv2 is restricted to just merc,tmerc,lcc, and poly)

Hamish

comment:2 Changed 10 years ago by hamish

the NOAA reprojection tool does not change the KNP/ and KNQ/ lines but it adds a ARC/ line in the header comments.

a minute or two of reverse engineering later,

tmerc:

  • P3= in the ARC/ line is scale factor
  • P4= 0 (??)
  • P5= central meridian
  • P6= origin of latitude
  • P7= +x_0
  • P8= +y_0

lcc:

  • P3= 1st Standard Parallel
  • P4= 2nd Standard Parallel
  • P5= Central Meridian
  • P6= Origin of Latitude
  • P7= +x_0
  • P8= +y_0

I'm not sure how useful that is; it would be good to have verification.

Hamish

comment:3 Changed 10 years ago by hamish

for tmerc P3 in the KNQ/ line conflicts with P3 in the ARC/ line, +lat_0 vs. +k_0 ?? so don't trust the ARC/ variables in comment:2 to be valid for what you find in the KNQ/ line.

comment:4 Changed 10 years ago by hamish

the initial report is a bit wrong, this has nothing to do with the central meridian. It's about how to set +lat_ts from "PP" on the KNP/ header line(s). That is currently hardcoded in bsbdataset.cpp as PARAMETER[\"standard_parallel_1\",0],.

I looked at making a quick patch but the trouble extracting PP= from the header is that each new metadata line starts with "aaa/" and can line wrap, but the current code assumes each individual line in the file is its own metadata record. So it got messier than I wanted to submit.

even if +lat_ts is not critical (KNP/SC=scale is currently ignored), PP='s meaning is context dependent and for poly & tmerc it gives +lon_0. So for enhanced bsb projection support in the future it becomes important to parse that term.

Hamish

comment:5 Changed 9 years ago by warmerdam

Cc: chrisbarker added
Status: newassigned

Starting work on this.

comment:6 Changed 9 years ago by hamish

fyi, much of what I've been able to figure out so far can be found in this script:

https://trac.osgeo.org/grass/browser/grass-addons/raster/r.out.kap_template/r.out.kap_template

starting about line 310.

thanks, Hamish

comment:7 Changed 9 years ago by warmerdam

I'm finding image data of HG-DEMO_1.KAP to be corrupt at scanline 128. Is this just me? Is there any reason to believe this should be a valid and readable file?

Hamish, I see use of a variety of parameters. Have you actually found files which vary these parameters? For instance, scale and latitude of origin for transverse mercator?

comment:8 Changed 9 years ago by warmerdam

OK, I have updated the header reading code to merge extension lines into the main header line.

I have also added preliminary handling of transverse mercator based on the KNP line. The old BSB docs I have only list the KNP line and the only alterable value for TM is PP (central meridian).

These changes are in trunk (r20801). I have only been able to do rough validation since the image appears to be unreadable. I'll try to find other TM images from NOAA.

I am hesitant to go much further without examples of different behavior.

I'm going to try and work thruogh

comment:9 Changed 9 years ago by warmerdam

Ok, I can't find any noaa transverse mercator maps, but there are some UTM maps. For instance http://www.charts.noaa.gov/RNCs/14996.zip has

KNP/SC=25000,GD=NAD83,PR=UNIVERSAL TRANSVERSE MERCATOR,PP=-92.7,PI=2.000,SP=
    SK=0.0000000,TA=90.0000000,UN=FEET,SD=GREAT LAKES LOW WATER,DX=2.50,DY=2.50
KNQ/EC=RF,GD=NARC,VC=UNKNOWN,SC=GLLW,PC=*U,P1=UNKNOWN,P2=UNKNOWN,P3=UNKNOWN
    P4=UNKNOWN,GC=NOT_APPLICABLE,RM=POLYNOMIAL

I do not understand how this can be UTM with a central meridian of -92.7. That does not correspond to any UTM zone. Is this really supposed to just be transverse mercator with UTM parameters for everything *except* the central meridian which is arbitrary?

OK, I've confirmed that this is essentially just the transverse mercator case again. Incorporated in trunk (r20802). Moving on to Polyconic and/or LCC.

comment:10 Changed 9 years ago by hamish

As far as I can remember, I've never seen an official NOAA chart in anything other than Merc or Polyconic.

#3175 comment 14: "for testing, NOAA chart 14500 is Polyconic not Merc."

I had never heard of RNCs in UTM before. Is that BSBv3? AFAIR the projection options exploded from the base 4 supported by BSBv2 to an unknown amount of whole bunch of them.. Anyway -92.7 is not an official UTM +lon_0 for sure. And without a zone associated with it... well it makes me scratch my head as to if the "UNIVERSAL" & PC=*U really correct. I'll download that chart and have a look.

All the examples I've seen for tmerc have been for small harbours, and I seem to recall that the Germans had been using it within rivers? (maybe I'm getting confused with the rotation angle though) And apparently the Sweedish too... I get the same scanline error with the Sweedish HG_DEMO_1.KAP file.

All the terms I found (unless commented otherwise with something like "this is just a wild guess") would have been derived from inspecting govt issued charts from here or there, to a level that convinced me it was correct.

fwiw tmerc charts produced from my script do load and render correctly in OpenCPN, but there may be a bit of autocorrelation going on there. I'd have to go out to the boat to test it with 3rd party (non-foss) software.

Hamish

comment:11 Changed 9 years ago by warmerdam

I have added support for LCC and POLYCONIC based on NOAA chart examples (r20803).

Lots of scaling and offset projection parameters are still set arbitrarily but this doesn't matter too much since we just need the right shape of projection. The BSB file has no actual projected coordinates - just lat/long ones.

At this point I've made all the samples I have ready access to work. I would suggest that new tickets be created if files are found that don't work.

---

Hamish, Chris Barker did a full review of NOAA RNCs and found examples of POLYCONIC, LAMBERT CONFORMAL CONIC, UNIVERSAL TRANSVERSE MERCATOR and MERCATOR. Example charts include:

POLYCONIC
  11516_1.KAP
LAMBERT CONFORMAL CONIC
  14982_1.KAP
UNIVERSAL TRANSVERSE MERCATOR
  14996_1.KAP
MERCATOR
  11006_1.KAP

Fetch from urls of the form: http://www.charts.noaa.gov/RNCs/14982.zip

comment:12 Changed 9 years ago by hamish

(I'll slowly play catch-up)

Replying to warmerdam:

I'm finding image data of HG-DEMO_1.KAP to be corrupt at scanline

  1. Is this just me?

no, I see the same.

Is there any reason to believe this should be a valid and readable file?

only that the header source sounds like they are in the business:

http://www.hydrographica.se

Hamish, I see use of a variety of parameters. Have you actually found files which vary these parameters? For instance, scale and latitude of origin for transverse mercator?

sample files are the only source I've had to decipher these, I've never seen any authoritative documentation. A grep of my old data dir has just found a German example of tmerc:

KNP/SC=200000,GD=Potsdam
    PR=TRANSVERSE MERCATOR,PP=9.000000,PI=UNKNOWN
    P1=0.000000,P2=1.000000,P3=0.000000,P4=500000.000000
    SP=UNKNOWN,SK=0,UN=NONE,SD=NONE,DX=16.71,DY=16.71

Hamish

comment:13 in reply to:  12 Changed 9 years ago by hamish

Replying to hamish:

Replying to warmerdam:

I'm finding image data of HG-DEMO_1.KAP to be corrupt at scanline 128. Is this just me?

no, I see the same.

oops, I had chart outlines switched off. Yes, I can view the Swedish HG-DEMO_1.KAP chart in OpenCPN 2.1.624a (http://www.opencpn.org), and the raster lat/lon grid lines seem to be georegistered correctly.

funny that it works, as I thought/assumed/guessed that OpenCPN was just using GDAL's reader code as you are mentioned in the copyrights.. perhaps that's just the georef.c stuff and not the BSB reading charting.cpp though.

hmmm now I seem to recall Dave mentioning something about a common bug in the format, something to do with colors. Will have to dig that out.

Hamish

comment:14 Changed 9 years ago by hamish

Cc: manimaul added

taking the liberty of cc'ing manimaul as the source of the apparently buggy Swedish tmerc chart.

comment:15 Changed 9 years ago by hamish

fyi, "NAV": http://www.cruisersforum.com/forums/f134/charts-31346-52.html#post532827

points out this NOAA tool: http://www.csc.noaa.gov/digitalcoast/tools/chartreprojector/

which may be of interest. I'd used earlier versions which just added an ARC/ metadata line to the header comments of the BSB (see comment:2 above), not output a TIFF.

Hamish

comment:16 Changed 9 years ago by Even Rouault

bsb: fix typo of r20803

comment:17 in reply to:  11 Changed 9 years ago by hamish

Replying to warmerdam:

I have added support for LCC and POLYCONIC based on NOAA chart examples (r20803).

thanks,

Lots of scaling and offset projection parameters are still set arbitrarily but this doesn't matter too much since we just need the right shape of projection.

The BSB file has no actual projected coordinates - just lat/long ones.

fyi, GRASS's new wxPythonGUI "location" (projection params) setup wizard* has an option to set up a new project based on a georeferenced data file. So if you have a shapefile with a .prj file, or a geotiff in UTM12N as spec'd in the WKT and read by libgdal, it will create the new project and set up the proj terms from the file automatically. If GDAL is ignoring some of those terms, and the user doesn't pick up the problem by scanning the resulting +proj terms by eye, then things silently don't go to plan when they try to do other work.

point being that there may be other downstream reasons for passing on that metedata that are not directly linked to the correct reading of the original raster array.

[*] (the new location wizard is a rather nice bit of work & has come along quite nicely. already very usable.)

Replying to rouault:

bsb: fix typo of r20803

(with r20809)

re. r20813 NEWS:

  • Added UNIVERSAL TRANSVERSE MERCATOR, LCC and POLYCONIC handling (#3409)

suggest s/UNIVERSAL TRANSVERSE MERCATOR/TRANSVERSE MERCATOR/

Hamish

comment:18 Changed 5 years ago by Jukka Rahkonen

Resolution: fixed
Status: assignedclosed

This ticket was listed as fixed in GDAL 1.8.0 release news (r20813) and without further investigations I believe that this one is fixed.

Note: See TracTickets for help on using tickets.