From the V1 blobs, color information is sent to cells in the second visual area, V2. The cells in V2 that are most strongly color tuned are clustered in the "thin stripes" that, like the blobs in V1, stain for the enzyme cytochrome oxidase (separating the thin stripes are interstripes and thick stripes, which seem to be concerned with other visual information like motion and high-resolution form). Neurons in V2 then synapse onto cells in area V4. Area V4 is a relatively large visual area, the largest by far cortical area outside V1, encompassing almost as much cortex as V1. Neurons in V4 were originally proposed by Semir Zeki to be exclusively dedicated to color, but this has since been shown not to be the case.

Quantitative studies have argued that there is no higher concentration of color cells in V4 than in primary visual cortex, although this remains controversial. Independent of color sensitivity, V4 neurons have been shown to be very sensitive to the shape of stimuli, curvature, and stereo-scopic depth. V4 neurons have also been shown to be modulated by attention. The role of V4 neurons in color vision remains to be better characterized: indeed the vast majority of scientific papers examining the function of V4 do not concern color processing.

Anatomical studies have shown that neurons in V4 provide input to the inferior temporal lobe . "IT" cortex is thought to integrate color information with shape and form, although it has been difficult to define the appropriate criteria for this claim. Despite this murkiness, it has been useful to characterize this pathway (V1 > V2 > V4 > IT) as the ventral stream or the "what pathway", distinguished from the dorsal stream ("where pathway") that is thought to analyze motion, among many other features.

Other animals enjoying three, four or even five color vision systems include tropical fish and birds. In the latter case tetrachromacy is achieved through up to four cone types, depending on species. Brightly colored oil droplets inside the cones shift the spectral sensitivity of the cell. (Some species of bird such as the pigeon in fact possess five distinct types of droplet and may thus be pentachromats). Mammals other than primates generally have less effective two-receptor color perception systems, allowing only dichromatic color vision; marine mammals have only a single cone type and are thus monochromats.

Color perception mechanisms are highly dependent on evolutionary factors, of which the most prominent is thought to be satisfactory recognition of food sources. In herbivorous primates, color perception is essential for finding proper (mature) leaves. In hummingbirds, particular flower types are often recognized by color as well. On the other hand, nocturnal mammals have less-developed color vision, since adequate light is needed for cones to function properly. There is evidence that ultraviolet light plays a part in color perception in many branches of the animal kingdom, especially insects.