Automatic electrocardiogram (ECG) analysis using deep neural networks has seen promising results in recent years. However, these models are susceptible to domain shifts when they are applied to new data distributions not previously trained on. We investigate how training on worst-case artificial samples through adversarial training can help promote models that can easily be molded through fine-tuning to new datasets. We compare the area under the precision-recall curve (AUPRC) for the classification of atrial fibrillation using two cohorts: we use PTB-XL for training and CODE-15% for fine-tuning and evaluating the models. Our results show that adversarially trained models on ECG data yield higher transferability when fine-tuned on new datasets compared to normally trained models (0.732 vs. 0.685 AUPRC). We also note that they even have the ability to supersede models solely trained on the new dataset using more total time and data (0.732 vs. 0.685 AUPRC). Our work thus paves the way for the training of general models that can be applied to different types of new settings with high performance. © 2024 IEEE Computer Society. All rights reserved.