We propose a mechanism for N & eacute;el switching via suppressed heating in a hybrid magnon-phonon cavity. A terahertz-driven cavity mode couples to a spin-phonon chain, and all particles dissipate energy through baths. Mean-field analysis shows that cavity photons enable switching by imbalancing the spin-density on sublattices- a symmetry-breaking effect absent without the cavity. Switching occurs at low fields (1-5 V/mu m) and selectively targets low magnon energies and perpendicular magnon modes. Laser fluence, damping, and photon loss tune the switching, advancing cavity-assisted optospintronics.