On-Premise or Cloud?

Cloud (or off-premise hosting) versus the classic on-premise server?   The language and array of possibilities can be confusing: clouds, private clouds, on-premise private, off-premise private, hybrid cloud services, etc.  How do you choose the best IT solution for...