This is one of those things that doesn't seem obvious at first, but when you think about it, you realize that it makes complete sense.
I already figured this out in my own head. Why would "red" or "purple" actually be a thing floating around out there in the world? Especially when you factor in deer can see ultraviolet. Nope, our brains stick the colors on.