Why is it so difficult?

Finding out the real color of a planet is not a trivial thing, the images we have are calibrated in many different ways to help scientists do their jobs.
Sometimes, images are taken in different wavelenghts far from the ones human eyes are able to see, like infrared. To make them visible, they are converted to human visible colors by algorithms of many sorts.
Some other times images are retouched simply to make them look better in websites, newspapers or TV. They're literally known as "false color images"

What is real color, anyway?

There's no such thing as "real" color, as it depends on the eyes of who is watching. Sounds like a beautiful philosophical thought, right?
The human eye is capable of seeing light in wavelenghts ranging from 390 to 700 nanometers. That's enough for a beautiful sunset, but scientists crave for much more.
Light from the Universe comes in many sorts of wavelenghts, scientists often calibrate their telescopes for exotic wavelenghts that allow them to discover much more. It's just like having superhero vision.

We've traversed the Internet searching for the best sources of real color images, we've found many other studies and great results (references below), so we've crossed them and compiled the results here in a beautiful way for students, artists and curious people, and also in an official CSS repository for geek programmers.
We've open sourced our findings via the spacecolors Github repository, check out there the most up-to-date, accurate colors and contribute!
By the way, turns out the average color of the Universe it's not black! Actually, they've called the color "Cosmic Latte", and this text is written in it!

The book "Pale blue dot" by Carl Sagan was a strong inspiration for this project.
Gustav Holst's "The Planets" suite interpreted by the Berlin Philarmonic Orchestra and directed by Herbert von Karajan is recommended when reading "Pale Blue Dot".