Understanding the binding mechanism between probe-functionalized magnetic nanoparticles (MNPs) and DNA targets or amplification products thereof is essential in the optimization of magnetic biosensors for the detection of DNA. Herein, the molecular interaction forming hybrid structures upon hybridization between DNA-functionalized magnetic nanoparticles, exhibiting Brownian relaxation, and rolling circle amplification products (DNA-coils) is investigated by the use of atomic force microscopy in a liquid environment and magnetic biosensors measuring the frequency-dependent magnetic response and the frequency-dependent modulation of light transmission. This approach reveals the qualitative and quantitative correlations between the morphological features of the hybrid structures with their magnetic response. The suppression of the high-frequency peak in the magnetic response and the appearance of a new peak at lower frequencies match the formation of larger sized assemblies upon increasing the concentration of DNA-coils. Furthermore, an increase of the DNA-coil concentration induces an increase in the number of MNPs per hybrid structure. This study provides new insights into the DNA-MNP binding mechanism, and its versatility is of considerable importance for the mechanistic characterization of other DNA-nanoparticle biosensor systems.