Opened 11 years ago

Closed 9 years ago

#2785 closed enhancement (fixed)

Move IGNORE_MISSING_DATA to run-time configuration

Reported by: pramsey Owned by: pramsey
Priority: normal Milestone: 6.0 release
Component: MapServer C Library Version: unspecified
Severity: normal Keywords:
Cc: woodbri, dmorissette, jmckenna, mko

Description

Currently Mapserver has only compile-time control of behavior when files referenced in tile indexes are missing, via the IGNORE_MISSING_DATA define, and that behavior is applied globally.

Under this RFC, the missing data behavior will be defined:

  • at run time
  • on a per-layer basis

As well, the option to "fail on a missing layer" will be added to the WMS client code, which currently defaults to an "ignore on a missing layer".

All changes will preserve the current default behavior in cases where new behavior is not requested by the user: missing data in tileindexes will continue to cause failure and missing WMS layers will continue to be ignored.

A PROCESSING instruction "IGNORE_MISSING_DATA" will be added, and checked at run time in place of the current #ifndef IGNORE_MISSING_DATA blocks.

At the same time, this ticket (#2722) can probably be tracked down and resolved.

People who previously compiled with --enable-ignore-missing-data will have to add IGNORE_MISSING_DATA directives to their map files instead.

Attachments (3)

ignore_missing_1.patch (20.8 KB) - added by pramsey 11 years ago.
All existing IGNORE_MISSING_DATA has been swapped to run-time.
ignore_missing_2.patch (22.7 KB) - added by pramsey 11 years ago.
Should apply cleanly and include shptree and mapscript cases.
2785-1.patch (1.4 KB) - added by pramsey 10 years ago.
WMS client corrupt response patch

Download all attachments as: .zip

Change History (24)

comment:1 Changed 11 years ago by woodbri

Paul,

Can you please make sure that the directive can be set at the MAP object level or the LAYER object. At the MAP object it will define the default behavior for all LAYER objects and at the LAYER object it will explicitly set the value for that LAYER possible overriding the default set at the MAP level.

This is important because many of us have 100s of mapfiles and it will be a pain to add the directive to just the MAP object let alone all the LAYERs. In fact, better yet keep the compile time directive, but make it only set the default behavior of the MAP object's IGNORE_MISSING_DATA property. Then you would have NO mapfile regression issues, and no one would have to change mapfiles, but you would get all the features you want to add.

comment:2 Changed 11 years ago by woodbri

Cc: woodbri added

comment:3 Changed 11 years ago by pramsey

Hm, there is no PROCESSING block at the map file level, so perhaps it's time to actually add a map file directive.

ON_MISSING IGNORE|LOG|FAIL

yes?

comment:4 Changed 11 years ago by dmorissette

Cc: dmorissette added

comment:5 Changed 11 years ago by pramsey

The mechanics in the proposal have been changed to be a single CONFIG entry that controls behavior for the whole mapfile CONFIG "ON_MISSING_DATA", with three modes "IGNORE", "LOG", "FAIL". The --ignore-missing-data compile switch will be left in place and alter the default behavior between LOG and FAIL.

Changed 11 years ago by pramsey

Attachment: ignore_missing_1.patch added

All existing IGNORE_MISSING_DATA has been swapped to run-time.

comment:7 Changed 11 years ago by jmckenna

Cc: jmckenna added

comment:8 Changed 11 years ago by jmckenna

paul i am trying to implement your patch, with r8000 on windows, but I get these messages:

mapserver-trunk>patch -i ignore_missing_1.patch patching file maprasterquery.c Hunk #1 FAILED at 719. Hunk #2 FAILED at 778. 2 out of 2 hunks FAILED -- saving rejects to file maprasterquery.c.rej patching file mapshape.c Hunk #3 FAILED at 1676. Hunk #5 FAILED at 1906. Hunk #6 succeeded at 1917 with fuzz 1. Hunk #7 FAILED at 1927. Hunk #9 FAILED at 1973. Hunk #10 FAILED at 1985. Hunk #12 FAILED at 2028. Hunk #13 succeeded at 2062 with fuzz 1. Hunk #15 FAILED at 2108. Hunk #16 FAILED at 2140. Hunk #17 FAILED at 2238. 9 out of 18 hunks FAILED -- saving rejects to file mapshape.c.rej patching file mapraster.c Hunk #1 succeeded at 1588 with fuzz 1. patching file mapshape.h patching file mapserver.h Hunk #2 succeeded at 1524 with fuzz 1. patching file mapobject.c


hmm am i doing something wrong??

comment:9 Changed 11 years ago by jmckenna

it must be some unix-windows encoding issue :(

comment:10 Changed 11 years ago by pramsey

it is possible your checkout is in windows crlf form?

comment:11 Changed 11 years ago by jmckenna

(my patch problem was caused by some editor magic)

I successfully tested the patch on windows, first having set the 'IGNORE_MISSING_DATA' switch at compile-time, and then not setting it. I tested with an index with a missing file, using shp2img...and the new "ON_MISSING_DATA" config setting in the mapfile worked as expected.

Very nice addition Paul. Let me know when it needs to be documented.

Changed 11 years ago by pramsey

Attachment: ignore_missing_2.patch added

Should apply cleanly and include shptree and mapscript cases.

comment:12 Changed 11 years ago by pramsey

Committed to trunk at r8015.

comment:13 Changed 11 years ago by jmckenna

Resolution: fixed
Status: newclosed

I've opened ticket:2809 for documenting this addition.

comment:14 Changed 11 years ago by jmckenna

Resolution: fixed
Status: closedreopened

reopening as the WMS Client changes have not yet been implemented...

comment:15 Changed 11 years ago by mko

Cc: mko added

When using wms client connections, remote services sometimes return a 0-byte response with content-type image/png. msDrawRasterLayerLow() will fail to draw the layer. Can we treat corrupt files as missing data too and fail|log|ignore? For comparison, when getting a http timeout, mapserv works similar to MS_MISSING_DATA_LOG (msDebug and msSetError).

Changed 10 years ago by pramsey

Attachment: 2785-1.patch added

WMS client corrupt response patch

comment:16 Changed 10 years ago by pramsey

Try the patch for the WMS zero-length problem, hopefully this catches it.

comment:17 Changed 10 years ago by pramsey

Milestone: 5.6 release6.0 release

comment:18 Changed 10 years ago by pramsey

If the patch doesn't get tested, I can't apply it, as I don't have access to a rig that produces this error.

comment:19 Changed 10 years ago by dmorissette

A good way to produce a 0-byte image/png response is to try to connect to the demis.nl WMS service (it refuses to serve requests to MapServer and produces a 0-byte response instead). Ticket #3243 discusses this and even includes a mapfile for testing.

Reassign to me if you want and I'll test and apply the patch.

comment:20 Changed 10 years ago by pramsey

Thanks Daniel, that map file made testing very very easy. Applied the patch to trunk at r9753.

comment:21 Changed 9 years ago by pramsey

Resolution: fixed
Status: reopenedclosed
Note: See TracTickets for help on using tickets.