Heparan sulfate (HS) is a linear polysaccharide attached to a core protein, forming heparan sulfate proteoglycans (HSPGs) that are ubiquitously expressed on the surface of almost all mammalian cells and the extracellular matrix. HS orchestrates the binding of various signal molecules to their receptors, thus regulating many biological processes, including homeostasis, metabolism, and various pathological processes. Due to its wide distribution and negatively charged properties, HS is exploited by many viruses as a cofactor to attach to host cells. Therefore, inhibition of the interaction between virus and HS is proposed as a promising approach to mitigate viral infection, including SARS-CoV-2. In this review, we summarize the interaction manners of HS with viruses with focus on significant pathogenic RNA viruses, including alphaviruses, flaviviruses, and coronaviruses. We also provide an overview of the challenges we may face when using HS mimetics as antivirals for clinical treatment. More studies are needed to provide a further understanding of the interplay between HS and viruses both in vitro and in vivo, which will favor the development of specific antiviral inhibitors.