Off-diagonal mixed state phases based upon a concept of orthogonality adapted to unitary evolution and a proper normalisation condition are introduced. Some particular instances are analysed and parallel transport leading to the off-diagonal mixed state geometric phase is delineated. A complete experimental realisation of the off-diagonal mixed state geometric phases in the qubit case using polarisation-entangled two-photon interferometry is proposed.