Move IGNORE_MISSING_DATA to run-time configuration

[pramsey]

Currently Mapserver has only compile-time control of behavior when files referenced in tile indexes are missing, via the IGNORE_MISSING_DATA define, and that behavior is applied globally.

Under this RFC, the missing data behavior will be defined:

• at run time
• on a per-layer basis

As well, the option to "fail on a missing layer" will be added to the WMS client code, which currently defaults to an "ignore on a missing layer".

All changes will preserve the current default behavior in cases where new behavior is not requested by the user: missing data in tileindexes will continue to cause failure and missing WMS layers will continue to be ignored.

A PROCESSING instruction "IGNORE_MISSING_DATA" will be added, and checked at run time in place of the current #ifndef IGNORE_MISSING_DATA blocks.

At the same time, this ticket (#2722) can probably be tracked down and resolved.

People who previously compiled with --enable-ignore-missing-data will have to add IGNORE_MISSING_DATA directives to their map files instead.

[woodbri]

Paul,

Can you please make sure that the directive can be set at the MAP object level or the LAYER object. At the MAP object it will define the default behavior for all LAYER objects and at the LAYER object it will explicitly set the value for that LAYER possible overriding the default set at the MAP level.

This is important because many of us have 100s of mapfiles and it will be a pain to add the directive to just the MAP object let alone all the LAYERs. In fact, better yet keep the compile time directive, but make it only set the default behavior of the MAP object's IGNORE_MISSING_DATA property. Then you would have NO mapfile regression issues, and no one would have to change mapfiles, but you would get all the features you want to add.

[pramsey]

Hm, there is no PROCESSING block at the map file level, so perhaps it's time to actually add a map file directive.

ON_MISSING IGNORE|LOG|FAIL

yes?

[pramsey]

The mechanics in the proposal have been changed to be a single CONFIG entry that controls behavior for the whole mapfile CONFIG "ON_MISSING_DATA", with three modes "IGNORE", "LOG", "FAIL". The --ignore-missing-data compile switch will be left in place and alter the default behavior between LOG and FAIL.

[pramsey]

Er, all details are at ​http://svn.osgeo.org/mapserver/trunk/mapserver/rfc/ms-rfc-47.txt

[pramsey]

All existing IGNORE_MISSING_DATA has been swapped to run-time.

[jmckenna]

paul i am trying to implement your patch, with r8000 on windows, but I get these messages:

mapserver-trunk>patch -i ignore_missing_1.patch patching file maprasterquery.c Hunk #1 FAILED at 719. Hunk #2 FAILED at 778. 2 out of 2 hunks FAILED -- saving rejects to file maprasterquery.c.rej patching file mapshape.c Hunk #3 FAILED at 1676. Hunk #5 FAILED at 1906. Hunk #6 succeeded at 1917 with fuzz 1. Hunk #7 FAILED at 1927. Hunk #9 FAILED at 1973. Hunk #10 FAILED at 1985. Hunk #12 FAILED at 2028. Hunk #13 succeeded at 2062 with fuzz 1. Hunk #15 FAILED at 2108. Hunk #16 FAILED at 2140. Hunk #17 FAILED at 2238. 9 out of 18 hunks FAILED -- saving rejects to file mapshape.c.rej patching file mapraster.c Hunk #1 succeeded at 1588 with fuzz 1. patching file mapshape.h patching file mapserver.h Hunk #2 succeeded at 1524 with fuzz 1. patching file mapobject.c

---

hmm am i doing something wrong??

[jmckenna]

it must be some unix-windows encoding issue :(

[pramsey]

it is possible your checkout is in windows crlf form?

[jmckenna]

(my patch problem was caused by some editor magic)

I successfully tested the patch on windows, first having set the 'IGNORE_MISSING_DATA' switch at compile-time, and then not setting it. I tested with an index with a missing file, using shp2img...and the new "ON_MISSING_DATA" config setting in the mapfile worked as expected.

Very nice addition Paul. Let me know when it needs to be documented.

[pramsey]

Should apply cleanly and include shptree and mapscript cases.

[pramsey]

Committed to trunk at r8015.

[jmckenna]

I've opened ticket:2809 for documenting this addition.

[jmckenna]

reopening as the WMS Client changes have not yet been implemented...

[mko]

When using wms client connections, remote services sometimes return a 0-byte response with content-type image/png. msDrawRasterLayerLow() will fail to draw the layer. Can we treat corrupt files as missing data too and fail|log|ignore? For comparison, when getting a http timeout, mapserv works similar to MS_MISSING_DATA_LOG (msDebug and msSetError).

[pramsey]

WMS client corrupt response patch

[pramsey]

Try the patch for the WMS zero-length problem, hopefully this catches it.

[pramsey]

If the patch doesn't get tested, I can't apply it, as I don't have access to a rig that produces this error.

[dmorissette]

A good way to produce a 0-byte image/png response is to try to connect to the demis.nl WMS service (it refuses to serve requests to MapServer and produces a 0-byte response instead). Ticket #3243 discusses this and even includes a mapfile for testing.

Reassign to me if you want and I'll test and apply the patch.

[pramsey]

Thanks Daniel, that map file made testing very very easy. Applied the patch to trunk at r9753.