Opened 19 years ago
Last modified 13 years ago
#1305 assigned enhancement
(Prelim) Gradient Support for Mapserver 4.4.1
Reported by: | Owned by: | sdlime | |
---|---|---|---|
Priority: | normal | Milestone: | 5.6 release |
Component: | MapServer CGI | Version: | 4.6 |
Severity: | minor | Keywords: | |
Cc: | woodbri@…, sgillies@…, mapserver@…, mko, sholl, havatv, jmckenna |
Description (last modified by )
As I posted on the mapserver-users list, I have put together a hack to get gradient colors from a single style. Here is the post: ... Regarding the gradient coloring, it works like this: you choose a numeric field in your data (I've tested with PostGIS, but anything you can use for classitem or labelitem should work), and create a style like this: STYLE COLOR 60 60 60 MINCOLOR 0 0 0 MAXCOLOR 255 255 0 MINVALUE 0.0 MAXVALUE 300000.0 GRADIENTITEM "sale_price" END That takes the sale_price field from the shapes values, maps its value to a percentage between MINVALUE and MAXVALUE and then picks the color that's appropriate from the color range. For a quick snapshot of how this looks, I've mapped relative size of parcels (as a percentage) to these colors: MINCOLOR 127 29 200 #Purple MAXCOLOR 255 255 0 #Yellow You can see the results here: http://www.binko.net/gradient.png As I said, this is still in the proof of concept stage, and I haven't tested against any rendering except GD (although in theory it should work). I have gotten some feedback from the -users group, but I'd appreciate any from the core developers. I will post a patch against 4.4.1 as well. Bill
Attachments (8)
Change History (59)
comment:1 by , 19 years ago
Cc: | added |
---|
by , 19 years ago
Attachment: | gradient.patch.gz added |
---|
comment:2 by , 19 years ago
Cc: | added |
---|
comment:3 by , 19 years ago
Just a couple of developer comments here. The way I did this was: 1) add the elements to the mapfile lexer etc (yuk!) 2) grab them in mapfile.c and put them on the styleObj 3) In msDrawShape, before we do anything, we call a new msMapGradient(style, shape) function for each style. That function changes the style's color according the the shape value (defined in GRADIENTITEM) and the parameters. It also must reset the pen, since that will be cached if it isn't set to MS_PEN_UNSET This probably is not very thread-friendly, at least not if multiple threads will access the same styleObj.
comment:4 by , 19 years ago
Feedback I've gotten from -users: 1) Add support for MINSYMBOLSIZE and MAXSYMBOLSIZE that uses the same GRADIENTITEM . This seems straightforward and useful: I think it can be done easily - just add the file elements and everything else is in msMapGradient(). 2) Add support for Some kind of increments. Personally, I like simple things, so something like "INCREMENT 10.0" would work for me. Other solutions have been discussed. 3) Add support for legends. This is needed, but seems to be beyond me for right now. If anyone who has worked in legends wants my input, please let me know. BTW: if someone will send me the connect string for CVS (even read only) I will post a 4.5 patch.
comment:5 by , 19 years ago
Hi All, was on vacation last week, sorry for the delayed comments. Looks like a sweet addition. This should be a 4.5 enhancement as opposed to 4.4 but it looks like that's were it's headed anyway. The CVS access information is at: http://cvs.gis.umn.edu/cvs.html I gotta think about where parameters should live a bit though. Something feels awkward to me about all this living in a style object. What you're really talking about is a creating a bunch of virtual classes right? Steve
comment:6 by , 19 years ago
I've heard both sides of that question: whether these should be "virtual classes" or not. To me, this is more of a Dynamic Color. In fact, I originally named one of the properties I added "DYNCOLOR". The fundemental problem I was trying to solve (and for my purposes, I feel I succeeded) was that I had features with an attribute that was a non-discrete value, and I wanted to map it simply. It turns out that its equally valid for any attribute that has a large number of discrete values. Because there are not a fixed number of "classes" (unless we add the "INCREMENT" option or an equivalent), I don't really think that virtual class describes it well. But what do I know: I just got here :) Bill
comment:7 by , 19 years ago
Couldn't the gradient be used to build classes with styles etc... someplace outside msDrawShape? Would be better performance-wise potentially. Could be done in msDrawShape or even msDrawMap or msPrepareMap... Just thinking out loud at the moment. This gets particularly hairy with annotation and markers since the markers and properties get cached along with a label. Steve
comment:8 by , 19 years ago
Doing it higher may be difficult, as it requires the value from the shape itself to calculate the color. If you want a set number of classes, you can probably get away with it. What's nice about this is that it give you continuous color. Is it potentially a performance issue? Perhaps. I haven't tried with more than a few thousand shapes. All it really does is replace the pen color on each shape. I'm not sure what the performance hit on that would be. The arithmetic involved in calculating the color should not be a real factor.
comment:9 by , 19 years ago
Steve, For what it's worth, I also think of it as a single class with a dynamic color. I think this becomes significant when we look at how this class/layer should be represented in the legend. In particular, we want the layer to appear to have a continuous color, and we don't want the legend gummed up with dozens or hundreds of classes. So there is the dilema. Is this a specialized style with variable color, or is it a bunch of classes. I would like to stress that this concept of treating a variable (raster pixel or vector attribute) as a continous value and coloring appropriately is not a fringe capability. It is quite core, especially with the science crowd.
comment:10 by , 19 years ago
I was thinking of it sort of like the color ramp support in something like arcview. That is, request 5 classes and let the software automatically classify data based on an attribute. I see the difference now. Let's go ahead with a patch against 4.5 as is and think about adding an autoclass feature as a totally seperate effort... Steve
comment:11 by , 19 years ago
(From update of attachment 315) this applies the same GRADIENT abilities to CVS HEAD as of 4/13/05
comment:12 by , 19 years ago
Please don't forget to update the code that copies and writes styleObj.
comment:14 by , 19 years ago
I have a new patch ready to test with the copy/write code in there. However, the move to GD >= 2.0.16 has kindof hammered me. My Mandrake 10.0 Official system doesn't have an update beyond 2.0.15, so I will need to upgrade to 10.1. Basically, I had a patch that seemed to work, until I got the update on 1225 earlier today. I can update as of this morning, and re-apply the patch and use that, or it will be tomorrow (at least) before I can move up to 10.1. Bill
comment:15 by , 19 years ago
Maybe I'm missing something but the patch doesn't look like a patch, rather a binary file... Steve
comment:16 by , 19 years ago
Steve, It looks like it is gzipped. Just save it as patch.gz and then do "gzip -d patch.gz".
comment:17 by , 19 years ago
Status: | new → assigned |
---|
Guess I was missing something then, thanks Frank. I've applied the patch and verified that at least everything still compiles fine. The changes have been committed to 4.5. Probably should create (or change this one) a documentation bug. Steve
comment:18 by , 19 years ago
Very sorry! I gzipped them out of habit: I should have asked the convention here. I just use: zcat patch.1305.gz | patch from within the mapserver directory. I won't gzip any new ones. As for my status on this, I need to grab the latest libgd2, build it (but not install it) and test my new patch with that. I upgraded my system libgd2 to the latest Mandrake 10.0 will support, and it broke my mapserver binary (with segfaults). Not sure why, but I'm sick of <expletive>ing with it. :) I'll get that done this weekend, test the new patch and have it up by Sunday if that's ok. I will also move to MDK 10.1 soon, but I have "real" work on this box that can't be interrupted for that kind of a change. Happy Tax Day! (Oh Joy!)
comment:19 by , 19 years ago
I saw the CVS commit cross the wire. No sign of the needed change to mapcopy.c.
by , 19 years ago
Attachment: | 1305CopySupport.patch added |
---|
This patch adds support for copying etc. (Against 4.5)
comment:20 by , 19 years ago
I managed to get libgd2.0.33 build and installed. The patch seems to work fine. However, I don't know how to exercise the copy/write code. If someone could test those, I'd appreciate it.
comment:21 by , 19 years ago
I did the stuff in the last patch by hand. Specifically: - added copy support (also added item index copying for all styleObj items) - added a line to freeStyle to clean up gradientitem if set - added writing support, although I set it so all (mincolor/maxcolor included) gradient parameters are output only if gradientitem is set Steve
comment:22 by , 19 years ago
Great Steve, Are you going to commit those? I'm still seeing my changes locally with no changes for Copy/Write support on mapcopy.c and mapfile.c
comment:23 by , 19 years ago
I committed those changes last week, here are the revisions from mapfile.c: * $Log: mapfile.c,v $ * Revision 1.299 2005/04/15 19:32:33 julien * Bug 1103: Set the default tolerance value based on the layer type. * * Revision 1.298 2005/04/15 18:52:01 sdlime * Added write support for the gradient parameters to writeStyle. Parameters are only written if a gradientitem is set. * * Revision 1.297 2005/04/15 17:52:47 sdlime * Updated freeStyle to free the gradientitem if set. * * Revision 1.296 2005/04/15 17:10:36 sdlime * Applied Bill Benko's patch for bug 1305, gradient support. I don't have any outstanding commits and it looks fine to me. Steve
comment:24 by , 19 years ago
One thing that would be nice would be to take advantage of this from MapScript. Unfortunately since this is item driven that's not possible with dynamic features. Now, I've not gone to look at the source. Might it be possible to expose a method in MapScript that would take 1) a style with the gradient min/max values and colors and 2) a value and have it return a color? That way you could use your code to compute colors and MapScript would do the assignment rather than the code in msDrawShape. This could be easy to do depending on your implementation. Thoughts? I ask because I have an immediate need to do something like this. Steve
comment:25 by , 19 years ago
Sure, Steve This would mean a trivial change to msMapGradient() to split it into two parts: one that gets the value off of the shape, and the other that maps the value to the gradient. Exposing to Mapscript is something you'd have to manage: I've no idea on it. I can make that change and send you a patch against current CVS HEAD. Do you want it on this bug?
by , 19 years ago
Attachment: | 1305.split.patch added |
---|
Splits the graident mapping function for Mapscripts support
comment:26 by , 19 years ago
In a discussion with Frank, he suggested that I bring this as an option for this bug as several people are still concerned with the number of options and the work Gradient: > As for your first two concerns, I'm fine with any modifications to those. > I was thinking recently that it could be simplified to: > > STYLE > COLORRANGE 0 0 0 255 255 0 # black to yellow > DATARANGE 0.0 100.0 > RANGEITEM "foobar" > END This would simplify things: would "RANGE" be better? It would allow things like "ALPHACOLORRANGE", "OUTLINECOLORRANGE" and "SYMBOLSIZERANGE" to only need one more element if people wanted to add them. What about having two colors on one element: is that ok? -- too confusing? Just thoughts Bill
comment:27 by , 19 years ago
Hmmm... I guess when I see Frank's suggestion I like it. I can't see where folks would get that confused, switching the terminology to "RANGE" helps with that. I'd support the change (coupled with the function spliting changes mentioned in previous comments). Steve
comment:28 by , 19 years ago
Steve, Just so you don't put Frank's reputation behind that idea, I suggested it and he suggested posting it here (although I think he liked the changes). I'm going to go ahead and make those modifications and stop messing with the legends. If anyone can give feedback on the mockup, I'll get those working eventually. I don't want the basic functionality held up with the new release pending etc. Bill
by , 19 years ago
Attachment: | 1305.colorrange.patch added |
---|
Patch to use new ColorRange syntax and split function
comment:29 by , 19 years ago
Ok, thanks for the clarification, now we can blame you! I've applied the patch and will work on the MapScript extension, should be trivial. Steve
comment:30 by , 19 years ago
Cc: | added |
---|
comment:31 by , 19 years ago
Cc: | added |
---|---|
Version: | 4.4 → 4.6 |
This is a short thread I had with Sean regarding the gradient/color ramp support in 4.6. Didn't want to lose the content. Steve ------- Yup, I see what you mean. As I was using it I was wishing for a non-linear interpolation, perhaps a log-based function. I suppose in map file terms we could do something like: COLORRAMP NAME 'blue2red' METHOD LINEAR MINCOLOR 0 0 255 MAXCOLOR 255 0 0 MINVALUE 0 MAXVALUE 100 INTERVALS 8 END and then refer to the ramp by name in a STYLE: STYLE ... COLOR 'blue2red' END with RANGEITEM moving to a LAYER level parameter, into the COLORRAMP itself (as ITEM), or perhaps even encoding it in the color: COLOR 'blue2red:DEPTH' What do you guys think? I don't have a good feeling for what the demand for this functionality, like I said as I used it I found myself wanting more. Steve >>> Sean Gillies <sgillies@frii.com> 5/11/2005 9:29:58 AM >>> OK, now I understand, although your example doesn't technically need the color ramp feature. I think Bill's use case is crying out for a ColorRange or ColorMap class, rather than extension of the Style class. Defining a new class now would allow for room to grow in the future when Bill wants to implement color ramps that are not just linear interpolations between 2 colors. Maybe we'll want something other than a straight line in color space, or want to dither to web-safe colors. Here's an example of how such a thing would be used in your example: ramp = ColorRamp(firstcolor=colorObj(255,0,0), lastcolor=colorObj(0,0,255), item="TEMP" minvalue=0 maxvalue=10) while (1): po = pointObj(x, y) style.color.setRGB(ramp.getColor(val)) po.draw(...) Of course, for mapserv, we'll need a place for the ColorRamp to live. I think it's much better to have this in the Layer than in the style. Maybe the style references the color ramp in the same way it currently references symbols. I like the styleObj as it is, just a "dumb" bag of properties. That's my $.02. My other $.02 is that Bill's feature is something that should be tried out in MapServer 4.7 for a while to get it just right. We should be fixing performance bugs and improving documentation at this point, not trying to polish a brand new feature. Sean On May 11, 2005, at 1:33 AM, Steve Lime wrote: > Bill's stuff assumes that features come from regular data sources (e.g. > shapefiles) and have an attribute with values that fall within a range > (e.g. temperature). If you're working with features build in MapScript > you don't have that luxury. Code might look like: > > $layer = ... > $class = $layer->getClass(0); > $style = $class->getStyle(0); > > while(...) { > # $x, $y and $val come from a database for instance > $point->{x} = $x; > $point->{y} = $y; > $style->setRangeColor($val); > $point->draw(...); > } > > Steve > >>>> Sean Gillies <sgillies@frii.com> 05/10/05 11:24 PM >>> > Steve, > > I haven't been following development of this feature and so I don't > really understand the difference between your way and Bill's. Can you > show me a usage example? > > Sean > >
comment:32 by , 19 years ago
In general terms, I like moving the colormap out into it's own object, partly because at some point I would like to be able to load pre-defined color ramps from a file.
comment:33 by , 19 years ago
How big a deal would it be to not release this as part of 4.6? I don't think we'd have to back it out, just not document it as an *official* feature. Steve
comment:34 by , 19 years ago
Guys, I have no problem with this change -- I don't see the need to reuse the ColorRamps/ColorRange/Gradient (any other names?), as they are generally specific to the range of the values on your Class -- however, moving them doesn't reduce their usefulness, and I've always been flexible on where they live. In general, making a "grouping" that contains all of the parameters looks more well defined, and was what I orignally had. I'd be careful with the word ColorRamp, as it seems to have a definition in ESRI products (although this might be exactly what they mean -- I don't know). As for putting it in 4.6/4.7, I don't care. I work off of CVS HEAD and will for a while. It makes sense to me that this shouldn't go into a release until the legend support is done. (Speaking of which, I'd like more feedback on the attached mockup.) The only thing I'd ask is this: If this support is removed from CVS HEAD, I will need to maintain it as a patch -- I have maps that will need to continue to function that use this. I don't mind changing their mapfiles to match the new format (even if its unstable for a while). I'm even willing to make the changes once we actually agree on a format. However, maintaining the patches is a pain in the ass -- if you want me to support this going forward, I'd appreciate being able to commit. If not, I'll be happy to work with whoever will be owning it. As far as the linear nature, I agree that applying a logarithmic interpolation would be nice, and I'm sure there are others. I have a working layer that does just that: however, I just apply the LOG() inside of PostGIS :-) Bill
comment:35 by , 19 years ago
I haven't looked at the way this is implemented in details, but I'd like to second the idea of using separate objects for the colorramp parameters instead of bloating the style object. With respect to releasing this or not in 4.6, well, even if it's there and not documented people will use it and may rely on it, it might be best to document that the feature is experimental and that there are plans to remove/change it in the next release.
comment:36 by , 19 years ago
Just to make sure we're all on the same page, the current CVS HEAD accepts ColorRange attributes in the following format: > STYLE > COLORRANGE 0 0 0 255 255 0 # black to yellow > DATARANGE 0.0 100.0 > RANGEITEM "foobar" > END I'm fine with any format that captures the following information: 1) Start and End Colors 2) Start and End of Data Range 3) Layer Attribute to use to calculate the color for the current shape To support Non-linear interpolation, additional attributes may be needed. We might find that putting all of the attributes EXCEPT for the "RANGEITEM" (perhaps renamed to RAMPITEM or whatever) outside of the class. That way you could use the same definitions for different layers just changing which attribute you're keying off of. Perhaps one item at the CLASS (or STYLE) level defined as such: COLORRAMP "blue2red" "Attribute1" and everything else like this (at a higher level): COLORRAMP NAME 'blue2red' # METHOD LINEAR -- not implemented yet MINCOLOR 0 0 255 MAXCOLOR 255 0 0 MINVALUE 0 MAXVALUE 100 # INTERVALS 8 -- not implemented yet END
comment:37 by , 19 years ago
I would agree that we should document this feature as experimental with the expectation that it will change (and hopefully stabilize) for 4.8. The feature type in general is needed in my opinion, but I for one haven't quite settled my mind on how the final result should be accomplished.
comment:38 by , 19 years ago
So, is there a rollback or does the code stay in? My preference would be a rollback until it's done the way we are agreeing it should be done. Because once it's in, the incentive to do it right diminishes.
comment:39 by , 19 years ago
My opinion is that the feature is already quite useful, and that it will be hard to get feedback if is rolledback. Furthermore, rolling it back may well be messy. So, I vote for leave it in, mark it as experimental/transitional.
comment:41 by , 19 years ago
Cc: | added |
---|
comment:42 by , 16 years ago
Description: | modified (diff) |
---|---|
Milestone: | → 5.4 release |
Perhaps we can address formally again in 5.4...
Steve
comment:43 by , 15 years ago
I am interested in using this funcionality, since I would like to display grayscale rasters (that are computed on the fly) with values in the range [-255 255].
I tried it with 5.2.0, and it seems to work. Here is the relevant snippet from the map file:
CLASS NAME "img diff" STYLE COLORRANGE 0 0 0 255 255 0 DATARANGE -255 255 END END
Now, is it possible to display a legend like in the example above, with a continuous colorbar?
follow-up: 45 comment:44 by , 15 years ago
This would also be beautiful from the WMS GetLegendGraphic side of things, especially for raster. I would also suggest this as a command line utility to generate gradient-ed legends offline, then being able to ref them in KEYIMAGE.
comment:45 by , 15 years ago
Replying to tomkralidis:
This would also be beautiful from the WMS GetLegendGraphic side of things, especially for raster. I would also suggest this as a command line utility to generate gradient-ed legends offline, then being able to ref them in KEYIMAGE.
Thank you Tom--this is exactly what I meant. I would like to be able to generate a similar graphic as a result of a GetLegendGraphic request, using a COLORRANGE directive, instead of defining a separate class for each group of values.
comment:46 by , 15 years ago
Cc: | added |
---|
comment:47 by , 13 years ago
Cc: | added |
---|
Ping,
is there any momentum within this issue? At least it seems to work, but the docs do not reflect anything about it. I even did not know about this possibility though.
Could this nice feature be somehow documented?
TIA for your commets.
Stephan
comment:48 by , 13 years ago
Cc: | added |
---|
I have created a documentation ticket (#3917). I hope it is possible to document this useful feature in some way, even if it is still preliminary.
comment:49 by , 13 years ago
Note that I had worked on this documentation long ago, but was told by the MapServer PSC not to add it, as the feature has not been approved by the PSC (see mapserver-dev mailing list archives).
comment:50 by , 13 years ago
Cc: | added |
---|
comment:51 by , 13 years ago
Jeff's correct. We had talked about some changes we'd like to see back at the sprint but didn't feel it was appropriate to make them so close to the release. They'll be in 6.2 for sure though.
Steve
Prelim Gradient Patch against mapserver-4.4.1