Our paper "Do Different Deep Metric Learning Losses Lead to Similar Learned Features?" has been accepted at ICCV 2021
07/23/2021We analyze the learned features of models trained with different deep metric learning loss functions on two scales (pixel and image property level) and find that ranking and classification based loss functions tend to differ significantly.
We are happy to announce that our paper "Do Different Deep Metric Learning Losses Lead to Similar Learned Features?" got accepted at ICCV 2021, one of the largest conferences for computer vision research. In our analytical work, we provide evidence that ranking and classification based loss functions for deep metric learning differ significantly in the way they embed inputs.
Deep metric learning has the task of training a neural network that maps input items to a vector space, where similar inputs are closer together than different inputs. This way, it is possible to represent new inputs and find similar items by searching for the closest items in vector space. Previous work has shown that a wide variety of proposed deep metric learning loss functions perform similarly under fair experimental conditions. This made us wonder, if the learned visual features of the input images are similar for all loss functions as well. We thus propose two methods, one analyzing the differences in losses on pixel level and one analyzing the influences of high-level image properties on the embeddings. For the pixel level analysis, we introduce a gradient based explainer method that highlights areas that were important for the network to embed the input. We can the compare such explainations quantitatively between different losses. For the high-level analysis, we create a dataset of realistic 3D car renders with randomly sampled properties, such as car color, background color, or sun elevation. We then quantify how much influence each property has on the resulting embedding. We find that there are significant differences between ranking and classification based loss functions, two categories of deep metric learning methods. Also, we identify that most models are prone to changes in seemingly irrelevant high-level image properties, such as object color or illumination.
We will update this news when the final paper is ready to be published.
