Painting a picture of color spaces

When we turn on our computers and cell phones, we are able to see all sorts of colors displayed on our screens. If we zoom in far enough, we’ll discover that each pixel consists of three subpixels pumping amounts of red, green and blue light into our eyes. This is a case where the RGB color model, which is one mathematical model of representing colors, is very useful. In fact, the RGB model is the most commonly used model for representing images on electronic systems; the light the subpixels pump creates an illusion of different colors as the rgb values of each pixel are mixed by our brains. Interestingly, this means that other animals with different photoreceptors than us would perceive images on a screen differently from how they would perceive that object in real life. However, the RGB color model isn’t the only way to represent colors, and different representations can be useful for different applications.

  

[1]  Close up of an LCD screen

First, let’s take a detour into color mixing. The two types of color mixing that color spaces use are additive and subtractive, where additive color mixing describes the color created from adding different wavelengths of light together while subtractive color mixing corresponds to the color created from subtracting wavelengths of light. Subtractive color mixing is the more intuitive mixing for humans, since it corresponds to the mixing of physical substances, such as paint. This is due to how humans perceive color; an apple is seen as red because the apple absorbs all wavelengths of light except for red, which is reflected. When we mix physical substances of different colors, each substance will absorb some wavelength the other paint reflects, leaving us with fewer wavelengths than we started with, hence the term subtractive color mixing.

                                                  

[2] Additive color mixing     [3] Subtractive color mixing

Now, back to what color models and color spaces actually are. Color models are additive or subtractive mathematical models of how colors are represented. Color spaces are defined by the combination of the color model and the mapping function to an absolute color space. Depending on the application, it may be useful to convert between different color spaces, which we can easily do for commonly used color spaces by using the existing OpenCV libraries. Some commonly used color spaces include RGB, LAB, YCrCb, and HSV.


Revisiting the RGB color space with more context: the sRGB color space is a color space based on the RGB color model, which is an additive color model that mixes red, green, and blue light to produce an array of colors. The three color channels each range from 0 to 255. Although the RGB color model is based on the way humans perceive color with three kinds of photoreceptor cells, this model is unable to represent every color the human eye can see, especially the highly saturated colors. In [5], the gray region is the range of colors a human can see, while the smaller triangle is the range of colors representable by the RGB model.

                                               

                     [4] RGB color model     [5] Colors representable by RGB model

In comparison, the LAB color space is useful for picking out subtle differences in color. LAB encodes lightness in one channel (L) and color in the other two channels, with A ranging from green to red and B ranging from blue to yellow. This color space is a much more accurate approximation of how humans see colors, since it is based on the opponent color model of human vision (red/green and yellow/blue form opponent pairs). In addition, it is a larger color space compared to RGB, covering the entire range of human color perception.



Another color space is the YCrCb color space, which is good for data compression. Y is the luminance (brightness) component, Cr describes the red chroma component and Cb describes the blue chroma component. Humans are more sensitive to black and white information, which YCrCb separates into the Y component and is usually stored in high resolution. In contrast, humans are less sensitive to the Cr and Cb chroma components, so they don’t need to be as accurate and thus can be compressed for more efficient processing.

                                                    

  [6] LAB color space                                                                                 [7] Cr and Cb components at constant luma  

The HSV color space is an alternative representation of the RGB color space, created to better represent how humans perceive color. Instead of red green blue channels, the HSV color space has hue, saturation, and value channels, where hue describes the predominant wavelength (color), saturation describes the intensity of the color (how much of other hues are in the color), and value describes the lightness or darkness of the color. Hue ranges from 0 to 360 degrees and saturation and value range from 0 to 100%.

      [8] HSV color model

The biggest advantage of the HSV color space is that it only uses one channel (hue) to specify color, making it very easy to find objects that are a certain color, despite lighting variations. For instance, our lab currently has an ant tracking software pipeline which automatically detects ants walking through certain regions of the video. The pipeline first needs to detect these regions of interest (outlined in red) utilizing the HSV color space. Since the lighting for the videos we are analyzing isn’t always consistent, the regions of interest are perceived as different shades of red. These different shades would make it very difficult to find our regions of interest throughout all videos if we used the RGB color space, since different shades of red would have very different RGB values. However, in HSV color space, pure red corresponds to a hue of 0, and we can look for red as pixels with a hue of around 0 with varying saturation and value values.

                                                                                                     

RGB value [255, 102, 102]       RGB value [255, 0, 0]                               RGB value [153, 0, 0]

HSV value [0°, 60%, 100%]                    HSV value [0°, 100%, 100%]                       HSV value [0°, 100%, 60%]

However, there is one drawback to HSV -- it doesn’t capture the true brightness of a color. Changes in hue cause changes in perceived brightness, even when saturation and value are held constant, due to how our computer screens display colors. Colors such as yellow, cyan and magenta are additive combinations of the rgb subpixels and are thus actually emitting twice as much light as red, green, and blue. This difference in perceived brightness is amplified by the way we interpret red green and blue light; our brain perceives red to be 62% brighter than blue and green to be 127% brighter than blue. This makes it very difficult to compare the contrast ratio of two arbitrary colors. Another color space, HCL (hue chroma luminance), exists to fix this problem HSV has with hue and brightness; a change in hue in this color space in no way affects brightness. However, a library to convert to this color space doesn’t currently exist in OpenCV. 

[10] HCL color space

In conclusion, there are many color spaces available for image processing, with some of the commonly used ones being RGB, LAB, YCrCb, and HSV. Depending on the application, a certain color space may be more useful compared to the other spaces; for example, the HSV color space is very useful for our ant tracking pipeline since we are trying to single out regions corresponding to the hue red. OpenCV allows us to easily convert from one color space to another.

Further reading:
This site has some sliders for RGB and HSV color models.
More on HSV versus HCL: Why I Love HSV and Why It's Totally Useless
Additional color spaces and their uses

Media Credits:
[1] Photo by Peter Halasz https://flic.kr/p/4n9P7n
[2] Image by Wikipedia user Pko source
[3] Public domain image source
[4] Image by Wikipedia user SharkD source
[5] Image by Wikipedia user Dicklyon source
[6] Image by Wikipedia user Nilsjohan source
[7] Image by Wikipedia user Simon Eugster source
[8] Image by Wikipedia user SharkD source
[9] Image by Wikipedia user SharkD source