Double NAT'ing a Bad thing?

I only use it in setting up hardware (such as prepping a network at our office before deploying to client), or in a few rare cases where only basic internet usage is needed. I try to avoid it in a busier production environment, because certain types of traffic are not fond of being molested by NAT, not to mention double molested. Some examples…certain VPN clients, or some VoIP. It is an extra hop, technically does add a tiny bit of latency. Can lose some QoS.