Convolutional neural networks are the method of choice for many medical imaging tasks, in particular segmentation. Recently, efforts have been made to include distance measures in the network training, as for example the introduction of boundary loss, calculated via a signed distance transform. Using boundary loss for segmentation can alleviate issues with imbalance and irregular shapes, leading to a better segmentation boundary. It is originally based on the Euclidean distance transform. In this paper we investigate the effects of employing various definitions of distance when using the boundary loss for medical image segmentation. Our results show a promising behaviour in training with non-Euclidean distances, and suggest a possible new use of the boundary loss in segmentation problems.