Transistors operating at high frequencies are the basic building blocks of millimeter-wave communication and sensor systems. The high charge-carrier mobility and saturation velocity in graphene can open way for ultra-fast field-effect transistors with a performance even better than what can be achieved with III-V-based semiconductors. However, the progress of high-speed graphene transistors has been hampered by fabrication issues, influence of adjacent materials, and self-heating effects. Here, we report on the improved performance of graphene field-effect transistors (GFETs) obtained by using a diamond substrate. An extrinsic maximum frequency of oscillation fmax of up to 54 GHz was obtained for a gate length of 500 nm. Furthermore, the high thermal conductivity of diamond provides an efficient heat-sink, and the relatively high optical phonon energy of diamond contributes to an increased charge-carrier saturation velocity in the graphene channel. Moreover, we show that GFETs on diamond exhibit excellent scaling behavior for different gate lengths. These results promise that the GFET-on-diamond technology has the potential of reaching sub-terahertz frequency performance.