Deep Representation and Metric Learning

• 
  Global Vegetation Modeling with Pre-Trained Weather Transformers. Janetzky, Pascal; Gallusser, Florian; Hentschel, Simon; Hotho, Andreas; Krause, Anna. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Accurate vegetation models can produce further insights into the complex inter- action between vegetation activity and ecosystem processes. Previous research has established that long-term trends and short-term variability of temperature and precipitation affect vegetation activity. Motivated by the recent success of Transformer-based Deep Learning models for medium-range weather forecast- ing, we adapt the publicly available pre-trained FourCastNet to model vegetation activity while accounting for the short-term dynamics of climate variability. We investigate how the learned global representation of the atmosphere’s state can be transferred to model the normalized difference vegetation index (NDVI). Our model globally estimates vegetation activity at a resolution of 0.25◦ while relying only on meteorological data. We demonstrate that leveraging pre-trained weather models improves the NDVI estimates compared to learning an NDVI model from scratch. Additionally, we compare our results to other recent data-driven NDVI modeling approaches from machine learning and ecology literature. We further provide experimental evidence on how much data and training time is necessary to turn FourCastNet into an effective vegetation model. Code and models will be made available upon publication.

  @misc{janetzky2024global, abstract = {Accurate vegetation models can produce further insights into the complex inter- action between vegetation activity and ecosystem processes. Previous research has established that long-term trends and short-term variability of temperature and precipitation affect vegetation activity. Motivated by the recent success of Transformer-based Deep Learning models for medium-range weather forecast- ing, we adapt the publicly available pre-trained FourCastNet to model vegetation activity while accounting for the short-term dynamics of climate variability. We investigate how the learned global representation of the atmosphere’s state can be transferred to model the normalized difference vegetation index (NDVI). Our model globally estimates vegetation activity at a resolution of 0.25◦ while relying only on meteorological data. We demonstrate that leveraging pre-trained weather models improves the NDVI estimates compared to learning an NDVI model from scratch. Additionally, we compare our results to other recent data-driven NDVI modeling approaches from machine learning and ecology literature. We further provide experimental evidence on how much data and training time is necessary to turn FourCastNet into an effective vegetation model. Code and models will be made available upon publication.}, author = {Janetzky, Pascal and Gallusser, Florian and Hentschel, Simon and Hotho, Andreas and Krause, Anna}, keywords = {author:KRAUSE}, title = {Global Vegetation Modeling with Pre-Trained Weather Transformers}, year = 2024 }

• 
  Liquor-HGNN: A heterogeneous graph neural network for leakage detection in water distribution networks. Schaller, Melanie; Steininger, Michael; Dulny, Andrzej; Schlör, Daniel; Hotho, Andreas. In LWDA’23: Lernen, Wissen, Daten, Analysen. October 09--11, 2023, Marburg, Germany, M. Leyer (ed.). 2023.
  • [ BibTeX ]
  • [ Download ]
  @article{schaller2023liquorhgnn, author = {Schaller, Melanie and Steininger, Michael and Dulny, Andrzej and Schlör, Daniel and Hotho, Andreas}, editor = {Leyer, M.}, journal = {LWDA'23: Lernen, Wissen, Daten, Analysen. October 09–11, 2023, Marburg, Germany}, keywords = {author:manli}, month = {Oktober}, title = {Liquor-HGNN: A heterogeneous graph neural network for leakage detection in water distribution networks}, year = 2023 }
• 
  Optimizing Medical Service Request Processes through Language Modeling and Semantic Search. Schlör, Daniel; Pfister, Jan; Hotho, Andreas. In 2023 the 7th International Conference on Medical and Health Informatics (ICMHI), of ICMHI 2023, pp. 136–141. Association for Computing Machinery, Kyoto, Japan, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Medical service requests are a crucial part of the workflow in hospitals and healthcare organizations. However, the process of requesting medical services can be time consuming and can require physicians and medical personnel to navigate complex interfaces and enter detailed information about the requested service. In this paper, we propose a system that uses machine learning techniques such as large language models and semantic search to optimize the process of requesting medical services. Our approach enables physicians to request medical services using natural language rather than navigating complex interfaces, allowing for more efficient and flexible interactions with hospital information systems. We evaluate our approach on real-world data and discuss the implications of our work for the future of digital health care. Our results suggest that our approach has the potential to streamline the process of requesting medical services and reduce the time and manual effort required in the daily hospital routine.

  @inproceedings{10.1145/3608298.3608324, abstract = {Medical service requests are a crucial part of the workflow in hospitals and healthcare organizations. However, the process of requesting medical services can be time consuming and can require physicians and medical personnel to navigate complex interfaces and enter detailed information about the requested service. In this paper, we propose a system that uses machine learning techniques such as large language models and semantic search to optimize the process of requesting medical services. Our approach enables physicians to request medical services using natural language rather than navigating complex interfaces, allowing for more efficient and flexible interactions with hospital information systems. We evaluate our approach on real-world data and discuss the implications of our work for the future of digital health care. Our results suggest that our approach has the potential to streamline the process of requesting medical services and reduce the time and manual effort required in the daily hospital routine.}, address = {New York, NY, USA}, author = {Schlör, Daniel and Pfister, Jan and Hotho, Andreas}, booktitle = {2023 the 7th International Conference on Medical and Health Informatics (ICMHI)}, keywords = {app_medical}, pages = {136–141}, publisher = {Association for Computing Machinery}, series = {ICMHI 2023}, title = {Optimizing Medical Service Request Processes through Language Modeling and Semantic Search}, year = 2023 }

• 
  Do Different Deep Metric Learning Losses Lead to Similar Learned Features?. Kobs, Konstantin; Steininger, Michael; Dulny, Andrzej; Hotho, Andreas. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10644–10654. 2021.
  • [ BibTeX ]
  • [ Download ]
  @article{noauthororeditor, author = {Kobs, Konstantin and Steininger, Michael and Dulny, Andrzej and Hotho, Andreas}, journal = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)}, keywords = {core-ml}, month = 10, pages = {10644-10654}, title = {Do Different Deep Metric Learning Losses Lead to Similar Learned Features?}, year = 2021 }

• 
  SimLoss: Class Similarities in Cross Entropy. Kobs, Konstantin; Steininger, Michael; Zehe, Albin; Lautenschlager, Florian; Hotho, Andreas. 2020.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  One common loss function in neural network classification tasks is Categorical Cross Entropy (CCE), which punishes all misclassifications equally. However, classes often have an inherent structure. For instance, classifying an image of a rose as "violet" is better than as "truck". We introduce SimLoss, a drop-in replacement for CCE that incorporates class similarities along with two techniques to construct such matrices from task-specific knowledge. We test SimLoss on Age Estimation and Image Classification and find that it brings significant improvements over CCE on several metrics. SimLoss therefore allows for explicit modeling of background knowledge by simply exchanging the loss function, while keeping the neural network architecture the same. Code and additional resources can be found at https://github.com/konstantinkobs/SimLoss.

  @misc{kobs2020simloss, abstract = {One common loss function in neural network classification tasks is Categorical Cross Entropy (CCE), which punishes all misclassifications equally. However, classes often have an inherent structure. For instance, classifying an image of a rose as "violet" is better than as "truck". We introduce SimLoss, a drop-in replacement for CCE that incorporates class similarities along with two techniques to construct such matrices from task-specific knowledge. We test SimLoss on Age Estimation and Image Classification and find that it brings significant improvements over CCE on several metrics. SimLoss therefore allows for explicit modeling of background knowledge by simply exchanging the loss function, while keeping the neural network architecture the same. Code and additional resources can be found at https://github.com/konstantinkobs/SimLoss.}, author = {Kobs, Konstantin and Steininger, Michael and Zehe, Albin and Lautenschlager, Florian and Hotho, Andreas}, keywords = {author:zehe}, note = {cite arxiv:2003.03182Comment: This paper is going to be published in the proceedings of the 25th International Symposium on Methodologies for Intelligent Systems (ISMIS)}, title = {SimLoss: Class Similarities in Cross Entropy}, year = 2020 }

• 
  On the Right Track! Analysing and Predicting Navigation Success in Wikipedia. Koopmann, Tobias; Dallmann, Alexander; Hettinger, Lena; Niebler, Thomas; Hotho, Andreas. In Proceedings of the 30th ACM Conference on Hypertext and Social Media, of HT ’19, pp. 143–152. ACM, Hof, Germany, 2019.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{Koopmann:2019:RTA:3342220.3343650, address = {New York, NY, USA}, author = {Koopmann, Tobias and Dallmann, Alexander and Hettinger, Lena and Niebler, Thomas and Hotho, Andreas}, booktitle = {Proceedings of the 30th ACM Conference on Hypertext and Social Media}, keywords = {author:hotho}, pages = {143–152}, publisher = {ACM}, series = {HT '19}, title = {On the Right Track! Analysing and Predicting Navigation Success in Wikipedia}, year = 2019 }
• 
  Classification of text-types in german novels. Schlör, D; Schöch, C; Hotho, A. In Digital Humanities 2019: Conference Abstracts. 2019.
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{schlor2019classification, author = {Schlör, D and Schöch, C and Hotho, A}, booktitle = {Digital Humanities 2019: Conference Abstracts}, keywords = {from:daschloer}, title = {Classification of text-types in german novels}, year = 2019 }

• 
  Learning Word Embeddings from Tagging Data: A methodological comparison. Niebler, Thomas; Hahn, Luzian; Hotho, Andreas. In Proceedings of the LWDA. 2017.
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{niebler2017learning_2, author = {Niebler, Thomas and Hahn, Luzian and Hotho, Andreas}, booktitle = {Proceedings of the LWDA}, keywords = {bibsonomy}, title = {Learning Word Embeddings from Tagging Data: A methodological comparison}, year = 2017 }
• 
  Learning Semantic Relatedness From Human Feedback Using Metric Learning. Niebler, Thomas; Becker, Martin; Pölitz, Christian; Hotho, Andreas. 2017.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Assessing the degree of semantic relatedness between words is an important task with a variety of semantic applications, such as ontology learning for the Semantic Web, semantic search or query expansion. To accomplish this in an automated fashion, many relatedness measures have been proposed. However, most of these metrics only encode information contained in the underlying corpus and thus do not directly model human intuition. To solve this, we propose to utilize a metric learning approach to improve existing semantic relatedness measures by learning from additional information, such as explicit human feedback. For this, we argue to use word embeddings instead of traditional high-dimensional vector representations in order to leverage their semantic density and to reduce computational cost. We rigorously test our approach on several domains including tagging data as well as publicly available embeddings based on Wikipedia texts and navigation. Human feedback about semantic relatedness for learning and evaluation is extracted from publicly available datasets such as MEN or WS-353. We find that our method can significantly improve semantic relatedness measures by learning from additional information, such as explicit human feedback. For tagging data, we are the first to generate and study embeddings. Our results are of special interest for ontology and recommendation engineers, but also for any other researchers and practitioners of Semantic Web techniques.

  @misc{niebler2017learning, abstract = {Assessing the degree of semantic relatedness between words is an important task with a variety of semantic applications, such as ontology learning for the Semantic Web, semantic search or query expansion. To accomplish this in an automated fashion, many relatedness measures have been proposed. However, most of these metrics only encode information contained in the underlying corpus and thus do not directly model human intuition. To solve this, we propose to utilize a metric learning approach to improve existing semantic relatedness measures by learning from additional information, such as explicit human feedback. For this, we argue to use word embeddings instead of traditional high-dimensional vector representations in order to leverage their semantic density and to reduce computational cost. We rigorously test our approach on several domains including tagging data as well as publicly available embeddings based on Wikipedia texts and navigation. Human feedback about semantic relatedness for learning and evaluation is extracted from publicly available datasets such as MEN or WS-353. We find that our method can significantly improve semantic relatedness measures by learning from additional information, such as explicit human feedback. For tagging data, we are the first to generate and study embeddings. Our results are of special interest for ontology and recommendation engineers, but also for any other researchers and practitioners of Semantic Web techniques.}, author = {Niebler, Thomas and Becker, Martin and Pölitz, Christian and Hotho, Andreas}, keywords = {postsii}, note = {cite arxiv:1705.07425}, title = {Learning Semantic Relatedness From Human Feedback Using Metric Learning}, year = 2017 }

• 
  Straight Talk! Automatic Recognition of Direct Speech in Nineteenth-Century French Novels. Schöch, Christof; Schlör, Daniel; Popp, Stefanie; Brunner, Annelen; Henny, Ulrike; Tello, Jos’e} Calvo. In DH, pp. 346–353. 2016.
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{schoch2016straight, author = {Schöch, Christof and Schlör, Daniel and Popp, Stefanie and Brunner, Annelen and Henny, Ulrike and Tello, Jos{\'e} Calvo}, booktitle = {DH}, keywords = {directspeech}, pages = {346–353}, title = {Straight Talk! Automatic Recognition of Direct Speech in Nineteenth-Century French Novels.}, year = 2016 }
• 
  Extracting Semantics from Unconstrained Navigation on Wikipedia. Niebler, Thomas; Schlör, Daniel; Becker, Martin; Hotho, Andreas. In KI, 30(2), pp. 163–168. 2016.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @article{journals/ki/NieblerS0H16, author = {Niebler, Thomas and Schlör, Daniel and Becker, Martin and Hotho, Andreas}, journal = {KI}, keywords = {selected}, number = 2, pages = {163-168}, title = {Extracting Semantics from Unconstrained Navigation on Wikipedia}, volume = 30, year = 2016 }

• 
  ConDist: A Context-Driven Categorical Distance Measure. Ring, Markus; Otto, Florian; Becker, Martin; Niebler, Thomas; Landes, Dieter; Hotho, Andreas. ECMLPKDD2015 (ed.). 2015.
  • [ BibTeX ]
  @inproceedings{ring2015condist, author = {Ring, Markus and Otto, Florian and Becker, Martin and Niebler, Thomas and Landes, Dieter and Hotho, Andreas}, editor = {ECMLPKDD2015}, keywords = {from:hotho}, title = {ConDist: A Context-Driven Categorical Distance Measure}, year = 2015 }

• 
  Computing semantic relatedness from human navigational paths on Wikipedia. Singer, Philipp; Niebler, Thomas; Strohmaier, Markus; Hotho, Andreas. In Proceedings of the 22nd international conference on World Wide Web companion, of WWW ’13 Companion, ACM (ed.), pp. 171–172. International World Wide Web Conferences Steering Committee, Rio de Janeiro, Brazil, 2013.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  This paper presents a novel approach for computing semantic relatedness between concepts on Wikipedia by using human navigational paths for this task. Our results suggest that human navigational paths provide a viable source for calculating semantic relatedness between concepts on Wikipedia. We also show that we can improve accuracy by intelligent selection of path corpora based on path characteristics indicating that not all paths are equally useful. Our work makes an argument for expanding the existing arsenal of data sources for calculating semantic relatedness and to consider the utility of human navigational paths for this task.

  @inproceedings{singer2013computing, abstract = {This paper presents a novel approach for computing semantic relatedness between concepts on Wikipedia by using human navigational paths for this task. Our results suggest that human navigational paths provide a viable source for calculating semantic relatedness between concepts on Wikipedia. We also show that we can improve accuracy by intelligent selection of path corpora based on path characteristics indicating that not all paths are equally useful. Our work makes an argument for expanding the existing arsenal of data sources for calculating semantic relatedness and to consider the utility of human navigational paths for this task.}, address = {Republic and Canton of Geneva, Switzerland}, author = {Singer, Philipp and Niebler, Thomas and Strohmaier, Markus and Hotho, Andreas}, booktitle = {Proceedings of the 22nd international conference on World Wide Web companion}, editor = {ACM}, keywords = {selected}, pages = {171–172}, publisher = {International World Wide Web Conferences Steering Committee}, series = {WWW '13 Companion}, title = {Computing semantic relatedness from human navigational paths on Wikipedia}, year = 2013 }
• 
  Computing Semantic Relatedness from Human Navigational Paths: A Case Study on Wikipedia. Singer, Philipp; Niebler, Thomas; Strohmaier, Markus; Hotho, Andreas. In International Journal on Semantic Web and Information Systems (IJSWIS), 9(4), pp. 41–70. IGI Global, 2013.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  In this article, the authors present a novel approach for computing semantic relatedness and conduct a large-scale study of it on Wikipedia. Unlike existing semantic analysis methods that utilize Wikipedia’s content or link structure, the authors propose to use human navigational paths on Wikipedia for this task. The authors obtain 1.8 million human navigational paths from a semi-controlled navigation experiment – a Wikipedia-based navigation game, in which users are required to find short paths between two articles in a given Wikipedia article network. The authors’ results are intriguing: They suggest that (i) semantic relatedness computed from human navigational paths may be more precise than semantic relatedness computed from Wikipedia’s plain link structure alone and (ii) that not all navigational paths are equally useful. Intelligent selection based on path characteristics can improve accuracy. The authors’ work makes an argument for expanding the existing arsenal of data sources for calculating semantic relatedness and to consider the utility of human navigational paths for this task.

  @article{singer2013computing, abstract = {In this article, the authors present a novel approach for computing semantic relatedness and conduct a large-scale study of it on Wikipedia. Unlike existing semantic analysis methods that utilize Wikipedia’s content or link structure, the authors propose to use human navigational paths on Wikipedia for this task. The authors obtain 1.8 million human navigational paths from a semi-controlled navigation experiment – a Wikipedia-based navigation game, in which users are required to find short paths between two articles in a given Wikipedia article network. The authors’ results are intriguing: They suggest that (i) semantic relatedness computed from human navigational paths may be more precise than semantic relatedness computed from Wikipedia’s plain link structure alone and (ii) that not all navigational paths are equally useful. Intelligent selection based on path characteristics can improve accuracy. The authors’ work makes an argument for expanding the existing arsenal of data sources for calculating semantic relatedness and to consider the utility of human navigational paths for this task.}, author = {Singer, Philipp and Niebler, Thomas and Strohmaier, Markus and Hotho, Andreas}, journal = {International Journal on Semantic Web and Information Systems (IJSWIS)}, keywords = {from:hotho}, number = 4, pages = {41–70}, publisher = {IGI Global}, title = {Computing Semantic Relatedness from Human Navigational Paths: A Case Study on Wikipedia}, volume = 9, year = 2013 }