We finally went to the Coliseum, Forum and Palatine Hill.  I thought the Palatine Hill was the most interesting (and it's the least crowded.)  The Coliseum looks like....a coliseum.  The Forum, you can see from various places in Rome.  But the Palatine Hill is where the emperors lived, and it's sort of a hidden treasure trove of interesting ruins.

Circus Maximus with Palatine in background