Change default color table

[wenzeslaus]

The default color table in 7.0 is rainbow. It is known that it is not ideal. I think we should change it before the next release.

For example, matplotlib ​changed the default to ​viridis. But there are other options as well.

[mlennert]

Replying to wenzeslaus:

The default color table in 7.0 is rainbow. It is known that it is not ideal. I think we should change it before the next release.

For example, matplotlib ​changed the default to ​viridis. But there are other options as well.

Choice of color table is one of the most important parts of cartography. I always tell my students to never use the default colors of any software. And that even though I don't know why rainbow was chosen, I like it because it is so blatantly bad that it forces the user to make a deliberate choice of another color table ;-).

So, I'm not sure I see the need to change that.

If we do change, I think that my favorite candidate for default would then be grey. This avoids color blindness issues, works for most applications and at the same time, in our modern color-oriented computing world, does provoke users to chose another one.

[neteler]

Replying to mlennert:

I always tell my students to never use the default colors of any software.

Agreed - but most users will just take the default (sigh).

If we do change, I think that my favorite candidate for default would then be grey. This avoids color blindness issues, works for most applications and at the same time, in our modern color-oriented computing world, does provoke users to chose another one.

This might be counter-productive and scare away users ("... oh, it cannot even show a color map!").

[wenzeslaus]

In 68506:

add viridis color table (see #3043)

[wenzeslaus]

comparison of viridis, gray, rainbow and bcyr using elevation from NC sample dataset

[wenzeslaus]

Overview of different rasters visualized with proposed color tables with color blidness simulated by ImageMagic

[wenzeslaus]

viridis is now available in trunk (try r.colors). This color table was developed for Matplotlib based on color properties and color perception and then it won the community voting. See the attachments to see how it behaves including gray scale print and simulated color blindness (I can share also the full resolution images).

The number one unresolved thing from my point of view is that viridis has 256 entries (most of the other color tables in GRASS has less than 20) and some test should be done to see if there is some performance difference. Also two pairs of colors are actually the same for the same values. (These issues can be resolved any time later.) Finally, it seems that on some screens viridis starts with more blueish color which doesn't look that good while on some other with more violet color which probably the right way (and looks better) but that's more a local issue.

Is anybody opposed to the change of default color table from rainbow to viridis in GRASS GIS for release 7.2?

[mlennert]

Replying to wenzeslaus:

viridis is now available in trunk (try r.colors). This color table was developed for Matplotlib based on color properties and color perception and then it won the community voting. See the attachments to see how it behaves including gray scale print and simulated color blindness (I can share also the full resolution images).

Without some kind of legend, it is difficult to understand what is what in your color_tables_overview... Generally, it seems to me that rainbow and bcyr provide greater contrasts.

Is anybody opposed to the change of default color table from rainbow to viridis in GRASS GIS for release 7.2?

I would vote -0, i.e. I'm opposed to this change, but not enough to stop it if others all agree on it.

I'm not convinced that viridis is better. More so, I think that if we provide a default color table of which we say that it is good (better than rainbow), we should make our criteria more explicit. Why do you find viridis good (better than other color tables) ? Just aesthetics ? What is good for matplotlib, might not be good for GRASS...

More fundamentally, and to repeat myself: IMHO there is no good default color table. Color tables qualities are context and target dependent. I personally thus prefer not to pretend that we can provide a good color table by default.

Moritz

[wenzeslaus]

Published papers about why rainbow color table is bad (incomplete list):

• D. Borland and R. M. Taylor Ii, "Rainbow Color Map (Still) Considered Harmful," in IEEE Computer Graphics and Applications, vol. 27, no. 2, pp. 14-17, March-April 2007. doi: 10.1109/MCG.2007.323435
• Stauffer, Reto, et al. "Somewhere over the rainbow: how to make effective use of colors in meteorological visualizations." Bulletin of the American Meteorological Society 96.2 (2015): 203-216.
• Rougier, Nicolas P., Michael Droettboom, and Philip E. Bourne. "Ten simple rules for better figures." PLOS Comput Biol 10.9 (2014): e1003833.

[wenzeslaus]

A search of the Internet yields many websites and blog posts dealing with rainbow color table and the message seems to be quite clear: avoid using it. See for example ​How The Rainbow Color Map Misleads.

The issue with having it as a default is that it is a sign of an approval. Sometimes people even consider rainbow color table appealing which makes it even more dangerous default because "it is default selected by the smart software creators and I like it, so let's not change it."

[wenzeslaus]

Replying to mlennert:

Replying to wenzeslaus:

See the attachments to see how it behaves including gray scale print and simulated color blindness (I can share also the full resolution images).

Without some kind of legend, it is difficult to understand what is what in your color_tables_overview...

The legend is in the full size image (shared personally). The layout of images is the following. First 2x2 images are:

viridis	rainbow
grey	bcyr

Next 2x2 to the right are the same with less RGB colors used for representation (as it happens in videos or GIFs). The next 2x2 to the right are the 4 original images in gray scale (as if printed on paper; it also shows how one perceives intensity of that image).

normal (2x2)

reduced colors (2x2)

gray scale (2x2)

Next two rows are again the original 4 images in 2x2 layout with 3 color blindness variations simulated by ImageMagic.

protanope (2x2)

deuteranope (2x2)

tritanope (2x2)

The same is repeated also for random surface created by r.surf.fractal and landclass96 categorical raster from the NC dataset (the first row is NC elevation raster).

elevation (4x6)

fractal (4x6)

landclass (4x6)

[wenzeslaus]

Wave tilted surface represented with viridis, rainbow, bcyr and gray color tables

[wenzeslaus]

Replying to mlennert:

Generally, it seems to me that rainbow and bcyr provide greater contrasts.

This might be true is some cases, or at least it seems this way. However, the opposite is true in the example I just posted which is a slightly titled plane with waves on it created using:

g.region cols=500 rows=300 n=-300 s=-600 w=0 e=500
r.mapcalc "x = col() / 10 + 2 * cos(col() * 100)" && r.colors map=x color=grey

The waves are visible with viridis (first) and gray (last) but rainbow (second) shows them a lot in the middle while not at all in the green region. bcyr (third) shows waves on the sides but not so much in the middle, they seem to be completely lost in light blue and yellow regions.

See more in work by ​Peter Kovesi (Colour Maps with Uniform Perceptual Contrast).

In some other cases rainbow overly emphasizes features or even creates (suggests) features which are not there. The example above doesn't show this except for the region between light and dark blue where the lines (waves) are much more prominent.

It must be noted that a lot of color tables, even some rainbow-like ones when they are perceptually uniform, can perform good in this test.

[wenzeslaus]

Replying to mlennert:

I think that if we provide a default color table (here map) of which we say that it is good (better than rainbow), we should make our criteria more explicit.

Article on Gnuplotting (citing Moreland, 2009) lists these features for a default color table and I'm adding my notes in the nested lists:

• The map yields images that are aesthetically pleasing
  • Although some people consider rainbow pretty or are simply used to it, my claim is that it is objectively worse because things which contain less colors or less styles in general is considered a better graphical design.
  • grey is OK but black and white is a bit boring and then there is the "show color feature" thing Markus mentioned in comment:2
  • In cartography black and white have usually special purpose (like text and its halo), so this excludes color tables which start or end with one of them.
  • The black-body radiation color tables which include or are similar to color tables like sepia in GRASS GIS, or hot, inferno, plasma, and magma in Matplotlib are often recommend, but I haven't seen them used that much. There might a actually a aesthetic reason for it. The Better Default Colormap for Matplotlib video suggests that the reason is that these don't have a green color in them. However, in general, I think they are nice. Gnuplot default is actually similar to inferno, plasma, or magma.
  • The cubehelix color tables are often regarded as "rotten melon", although some variations look nice and some are very much like viridis, plasma, and others. See seaborn and palettable for examples.
  • inferno, plasma, magma, and viridis were considered for Matplotlib and viridis was selected because it was the one most people liked. As mentioned above, it was likely visual because unlike others, it had green in (the measurable/theoretical features were similar with the other three).
• The map has a maximal perceptual resolution
  • In comment:9 showed that our rainbow doesn't always have it. Some revised rainbow-like color tables may have it quite high.
• Interference with the shading of 3D surfaces is minimal
  • This rules out the grey and also the ones which go from very low brightness (black) to very high brightness (white).
• The map is not sensitive to vision deficiencies
  • Although there is many different color blindness variations and sometimes the percentages of affected people are relatively low, this is often a requirement and my guess is that, similarly to web design, everybody at the end benefits.
  • This rules out anything which contains red and green at the same time like rainbow or gyr.
  • This also likely rules out anything which does not look good when converted to gray scale as the intensity is probably the last thing which is preserved. (I would need to do more here.) However, there is the benefit of printing in gray scale without much loss.
• The order of the colors should be intuitively the same for all people
  • I guess the usual issue here is who can name colors of rainbow in the right order? Additionally, does the color table include all colors or does it skip some?
  • This likely rules out anything with a lot of colors, so probably also the rainbow-like color tables which fix some of the classic rainbow color table problems.
  • Simpler color tables with one or two colors like summer from Matplotlib doesn't require any knowledge of order.
  • As the order of colors is not clear even few colors may create confusion. Exception might black-body radiation color tables where the order might be intuitive (but that might be just result of the brightness as discussed in the following point).
  • I think this is also where the diverging color tables like differences or curvature fail here as defaults because the choice of order of individual colors is arbitrary and the often pronounced middle would be unsuitable. However, ParaView uses diverging blue-white-red as a default and there are is some literature which suggests them.
  • Brightness seems to be only generally accepted natural ordering. This is in favor of grey and all color tables which look like grey when printing in black and white. This is the case for cubehelix color tables and inferno, plasma, magma, and viridis.
• The perceptual interpolation matches the underlying scalars of the map
  • (I would need to do more reading here but) I think that it means that human should be able to guess what is the color in between two colors in the color table and the result should be the same as what is a more detailed color table would actually say. Simple color tables with predictable changes in brightness would fulfill this.

Why do you find viridis good (better than other color tables) ?

As far as I can tell, the general trend in default color tables is simplicity, i.e. one, two or three colors, excluding white and black, clear brightness increase and black and white printing friendly, and color blind friendliness. Matplotlib's plasma and viridis fulfill those well together with some color tables from the cubelix and black-body radiation families (and perhaps some others designed in some other way but intentionally or unintentionally fulfilling the criteria - a lot of work would be needed, however, to find and check those).

Just aesthetics ?

I think it is the final factor after the other things are fulfilled since many color tables would fulfill the other requirements, but wouldn't be nice (some of the cubehelix ones, for example). There are two things which support viridis as color table accepted by people for aesthetic reasons. viridis has won voting over inferno, plasma, and magma preceding its inclusion to Matplotlib and Matlab actually has similar color table to viridis called parula (and I don't think they would select something they wouldn't consider nice). viridis' authors actually claim (in the video) that viridis is little better than parula according to its behavior in some color space (default of both in the past was jet which is similar to rainbow); some people actually mention aesthetics as well, but that's quite subtle difference in case of viridis and parula.

What is good for matplotlib, might not be good for GRASS...

They actually considered even terrain with shading as a use case. So, the question should be why it wouldn't be good for GRASS when it is good for Matplotlib?

More fundamentally, and to repeat myself: IMHO there is no good default color table. Color tables qualities are context and target dependent. I personally thus prefer not to pretend that we can provide a good color table by default.

However, people simply use the default anywhere from tutorials to articles and the reasons range from trusting the defaults (because why would authors put there bad defaults?) to liking the rainbow color tables (which are considered bad for several reasons). Thus, I think we need to provide the best default we can. This doesn't mean that we should not promote usage of a proper color table for a given case. For example, digital elevation models, temperatures or water depth are good examples where the default is not appropriate.

(I'm not including any links because Trac spam filter won't let me.)

[mlennert]

Replying to wenzeslaus:

Replying to mlennert:

I think that if we provide a default color table (here map) of which we say that it is good (better than rainbow), we should make our criteria more explicit.

Article on Gnuplotting (citing Moreland, 2009) lists these features for a default color table and I'm adding my notes in the nested lists:

Wow, you're really putting a lot of research into this. Thanks !

I agree with all you are saying (I never defended rainbow as a good color table !), and if you all think that having a better (not good, as I don't think that exists, but better) default color table, then go for it. As said, if we change, then personally I would prefer grey (as this would force people to chose if they want color), but I hear what you are saying about the marketing aspect.

One aspect your review doesn't seem to take into account is categorized maps. Many of the above arguments only relate to continuous, quantitative values, not qualitative data (and the landuse map in viridis is not very nice... ;-)), but I guess this is impossible to detect automatically, so different defaults for different types of maps does not make sense.

So, for me, go ahead with whatever you feel is right...

[wenzeslaus]

Replying to mlennert:

One aspect your review doesn't seem to take into account is categorized maps. Many of the above arguments only relate to continuous, quantitative values, not qualitative data (and the landuse map in viridis is not very nice... ;-)),

Right, I did not address them explicitly and I didn't looked at those specifically, but most of the things apply to categorical (qualitative) as well. Also I think that in case of categorical as well as diverging data (e.g. difference), it is easier for user to realize that a special color table is needed. viridis is not ideal but would work better than gray I think.

but I guess this is impossible to detect automatically, so different defaults for different types of maps does not make sense.

Well, in the current code, there are in fact separate calls for CELL and FCELL+DCELL (Rast_map_is_fp() is used). So, if we say that CELL map are categorical, they could be separate. I'm not sure how big assumption that would be. If CELL==categorical, separate tables would be easy to add unless I'm missing something. (source:grass/trunk/lib/raster/color_read.c?rev=68564#L55​)

[mlennert]

Replying to wenzeslaus:

Replying to mlennert:

One aspect your review doesn't seem to take into account is categorized maps. Many of the above arguments only relate to continuous, quantitative values, not qualitative data (and the landuse map in viridis is not very nice... ;-)),

Right, I did not address them explicitly and I didn't looked at those specifically, but most of the things apply to categorical (qualitative) as well. Also I think that in case of categorical as well as diverging data (e.g. difference), it is easier for user to realize that a special color table is needed. viridis is not ideal but would work better than gray I think.

You really do like viridis, it seems ;-)

but I guess this is impossible to detect automatically, so different defaults for different types of maps does not make sense.

Well, in the current code, there are in fact separate calls for CELL and FCELL+DCELL (Rast_map_is_fp() is used). So, if we say that CELL map are categorical, they could be separate. I'm not sure how big assumption that would be. If CELL==categorical, separate tables would be easy to add unless I'm missing something. (source:grass/trunk/lib/raster/color_read.c?rev=68564#L55​)

I thought about this. I think there are sufficient quantitative variables that are integers (just imagine a DEM with 1m vertical precision) to make this assumption (CELL = categorical) a bit dangerous.

[wenzeslaus]

Replying to mlennert:

You really do like viridis, it seems ;-)

There are alternatives if you go though the related papers and websites, but viridis seems to me as a really strong candidate. (Anybody please suggest others!)

Also, another open source community (Matplotlib) already did the hard work of identifying viridis and its relatives as good candidates for default.

Moreover, I think there is value in choosing what some other open source project uses.

I think there are sufficient quantitative variables that are integers (just imagine a DEM with 1m vertical precision) to make this assumption (CELL = categorical) a bit dangerous.

This was why I originally rejected the idea. Another good example would be (most of?) imagery data or count of occurrences (e.g. r.in.lidar method=n). The counter argument would be that the categorical color table would actually work for them in the same way as rainbow worked so far, but the actual categorical data would benefit. However, I don't think it is a string argument.

BTW, it would make sense to track this in the metadata (G7:t.create has semantictype=min|max|sum|mean).

[wenzeslaus]

In 68586:

change default color table for 2D and 3D rasters (see #3043)

Introduces DEFAULT_COLOR_TABLE define for
an easy change and global control.

Depends on r68506 which includes the
viridis color table used as a new default.

Also removes (obsolete?) calls to the
color table specific function and
uses the general function instead.

[wenzeslaus]

I made the change in r68586, but note that the commit also makes it easy to change the color table now - change at one place in include/gis.h:

#define DEFAULT_COLOR_TABLE "viridis"

So, we can of course still do that. Of course, I put viridis there for now.

[vincent]

I suggested a generic color ramp based on the GRASS GIS green, hoping this odd choice for a map would persuade users to set their own color tables. The grass-green table used in the attached file grass_chart.jpg is this one :

0% 230:244:230
50% 0:140:0
100% 0:60:0
nv 255:255:255

[vincent]

matplotlib/viscm grass color ramp

[vincent]

matplotlib/viscm grass color ramp properties

[vincent]

tinkering with matplotlib/viscm tool I tried to reproduce a colorramp close to the former (comment 17). Here is the result:

• properties shown in the attached file capture_grass_ramp.jpg
• editable python file grass_ramp.py

[wenzeslaus]

This (comment:18) looks pretty good. I like the colors and the fact that viscm shows it (creates it) so well. I would definitively include it (not necessarily as new greens (#3055) but perhaps as grass), but I still think that viridis is a better default because it has more colors. Although I can see that these green colors as default would be an interesting branding choice.

I've extended r.colors.matplotlib in r68724. Now you can use the file saved in viscm tool as a color table in GRASS GIS.

[vincent]

Replying to wenzeslaus:

I would definitively include it (not necessarily as new greens (#3055) but perhaps as grass)

The point is : to preserve constant deltas in the ramp we need to keep the right progress of colors i.e. the rule file cannot be of proportional type as understood by GRASS. The output from viscm has 256 steps, one just needs to turn RGB percents into 0-255 values.

[wenzeslaus]

Replying to vincent:

Replying to wenzeslaus:

I would definitively include it (not necessarily as new greens (#3055) but perhaps as grass)

The point is : to preserve constant deltas in the ramp we need to keep the right progress of colors

Yes.

i.e. the rule file cannot be of proportional type as understood by GRASS.

Can you please provide an example to help me understand? Ideally using r.colors.matplotlib input and output.

Anyway, what we would need is a color table which is constructed using the model in viscm, but that's just too complicated (I guess) and I don't think ​Matplotlib is doing anything like this, although I'm not sure what is the difference between ListedColormap and LinearSegmentedColormap.from_list. It seems that ​BIDS/colormap is using them in a same way (​ListedColormap, ​LinearSegmentedColormap.from_list).

In r.colors.matplotlib I try to do the conversion from the Matploitlib objects to GRASS color rules. The GRASS rules are linearly interpolated in between (when used) which is what happens in the Matplotlib objects I think. This is not ideal but with enough steps, it is a smaller problem. I struggle with what is enough and how to implement it. It seems that six colors give visually the same color tables as originals (so default is ncolors=6), however for reading the viscm generated files, I'm using whatever number is in the file (this way is also easier to implement).

The output from viscm has 256 steps,

That's right. It's 256 steps, not values from 1 to 256. In r.colors.matplotlib (in case of viscm output) I convert ListedColormap which is created from the following list:

[[ 0.02099905, 0.14497527, 0.14510218],
 [ 0.02128463, 0.14850087, 0.147124  ],
 [ 0.02135256, 0.15204182, 0.14916189],
...
 [ 0.9776095 , 0.92497705, 0.69352479],
 [ 0.9815921 , 0.92876498, 0.70167807],
 [ 0.98557933, 0.93256261, 0.70987881]]

to GRASS color rules:

0.000% 5:37:37
0.392% 5:38:37
0.784% 5:38:38
...
99.216% 250:236:177
99.608% 251:237:179
100.000% 252:238:181

one just needs to turn RGB percents into 0-255 values.

As far as I understand, that's unrelated to the above. This is RGB triplet format issue (0-1 -> 0-255). I hope I'm doing it right in r.colors.matplotlib:

# taken from color_rules.c
r1 = int(r1 * 255.999)
g1 = int(g1 * 255.999)
b1 = int(b1 * 255.999)

[vincent]

Replying to wenzeslaus:

In r.colors.matplotlib I try to do the conversion from the Matploitlib objects to GRASS color rules. The GRASS rules are linearly interpolated in between (when used) which is what happens in the Matplotlib objects I think.

Sorry I did not read entirely your manual page, the way you proceed (a proportional mode rule file) is definitely better than what I tried to do (translating the viscm color ramp to 255 RGB values).

As far as I understand, that's unrelated to the above. This is RGB triplet format issue (0-1 -> 0-255). I hope I'm doing it right in r.colors.matplotlib:

Anyhow, isn't it a minor issue ? IIUC it's a matter of approximating via int() rather than round() or considering 256 steps rather than 255. As a colorblind I don't feel concerned by a difference of 1/255 of a value !-)

To put it in a nutshell : r.color.matplotlib does the job the way it should.

[vincent]

Scilab instructions file

[vincent]

proposal for a new color ramp

[vincent]

Replying to vincent:

To put it in a nutshell : r.colors.matplotlib does the job the way it should.

r.colors.matplotlib does not provide exact percent values (did I miss something?). So starting back from my own grass_ramp.py file, I extracted the cm_data array, then did the 0-255 > 0-100% interpolation via Scilab.

For those interested, I give in attachment the cm_data array (grass_ramp.txt),and the sequence of commands run in Scilab (viscmtograss.sce). It outputs a file which needs some cosmetics, which I did it in one go :

scilab-cli -f viscmtograss.sce && cat rgbtemp | awk '{FS=" "}''{printf "%s%s %s:%s:%s\n", NR-1,"%",$1,$2,$3}'>grass_0_100 && rm rgbtemp

Finally I propose to add the new "grass" color rule to the existing list of default ramps.

[wenzeslaus]

In 68755:

add GRASS green color table by Vincent Bain (see #3043 and #3055)

[wenzeslaus]

A new grass color table is now available. I've removed the nv color specification and I'm leaving that on the other defaults. I added description GRASS GIS green (perceptually uniform). I've noticed (with inverted colors which sometimes show more) that the color table does not contain the exact color from the logo. That's a known issue (feature) I suppose (it is hard to get exact RGB value in viscm).

[wenzeslaus]

Replying to vincent:

r.colors.matplotlib does not provide exact percent values (did I miss something?).

Well, it provides exactly the values which are on the input only converted to percents. If you provide it with 256 steps, it will give you that. To have all whole percents from 0 to 100, you would need to give it 101 values (speaking about the current (and simple) implementation of the file input). In other words, it does not produce files which are nice to look at by human but the library function does not care. It is worth noting that both 101 and 256 colors (steps) are arbitrary choices and I don't know how to determine sufficient number of steps for a color table.

[wenzeslaus]

viridis and grass color tables applied to output of r.surf.fractal

[wenzeslaus]

Comparison of viridis and grass color tables:

Both are available in trunk (7.3), so you can easily try them yourself.

[martinl]

What is status of this ticket? Any backports? Time to close? Decrease the priority? Thanks.

[wenzeslaus]

I'll change the default in 7.2 to virdis during GRASS meetup today. We can consider backporting the color tables (only) to 7.0.

[martinl]

Replying to wenzeslaus:

I'll change the default in 7.2 to virdis during GRASS meetup today. We can consider backporting the color tables (only) to 7.0.

Thanks, Martin

[wenzeslaus]

In 68907:

add GRASS green color table by Vincent Bain (see #3043 and #3055, backport r68755)

[wenzeslaus]

In 68908:

change default color table to viridis for 7.2.0 (backports 68586, finishes r68631, closes #3043)