Deep Representation & Metric Learning

ModernGBERT: German-only 1B Encoder Model Trained from Scratch. Ehrmanntraut, Anton; Wunderle, Julia; Pfister, Jan; Jannidis, Fotis; Hotho, Andreas. 2025.

[ BibTeX ]
[ URL ]
[ Download ]

@misc{ehrmanntraut2025moderngbertgermanonly1bencoder,
  author = {Ehrmanntraut, Anton and Wunderle, Julia and Pfister, Jan and Jannidis, Fotis and Hotho, Andreas},
  keywords = {author:pfister},
  title = {ModernGBERT: German-only 1B Encoder Model Trained from Scratch},
  year = 2025
}

{O}tterly{O}bsessed{W}ith{S}emantics at {S}em{E}val-2024 Task 4: Developing a Hierarchical Multi-Label Classification Head for Large Language Models. Wunderle, Julia; Schubert, Julian; Cacciatore, Antonella; Zehe, Albin; Pfister, Jan; Hotho, Andreas. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), A. K. Ojha, A. S. Do{\u{g}}ru{\"o}z, H. Tayyar Madabushi, G. Da San Martino, S. Rosenthal, A. Ros{\’a} (eds.), pp. 602–612. Association for Computational Linguistics, Mexico City, Mexico, 2024.

[ Abstract ]
[ BibTeX ]
[ URL ]
[ Download ]

For our submission for Subtask 1, we developed a custom classification head that is designed to be applied atop of a Large Language Model. We reconstructed the hierarchy across multiple fully connected layers, allowing us to incorporate previous foundational decisions in subsequent, more fine-grained layers. To find the best hyperparameters, we conducted a grid-search and to compete in the multilingual setting, we translated all documents to English.

@inproceedings{wunderle-etal-2024-otterlyobsessedwithsemantics,
  abstract = {For our submission for Subtask 1, we developed a custom classification head that is designed to be applied atop of a Large Language Model. We reconstructed the hierarchy across multiple fully connected layers, allowing us to incorporate previous foundational decisions in subsequent, more fine-grained layers. To find the best hyperparameters, we conducted a grid-search and to compete in the multilingual setting, we translated all documents to English.},
  address = {Mexico City, Mexico},
  author = {Wunderle, Julia and Schubert, Julian and Cacciatore, Antonella and Zehe, Albin and Pfister, Jan and Hotho, Andreas},
  booktitle = {Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)},
  editor = {Ojha, Atul Kr. and Do{\u{g}}ru{\"o}z, A. Seza and Tayyar Madabushi, Harish and Da San Martino, Giovanni and Rosenthal, Sara and Ros{\'a}, Aiala},
  keywords = {author:pfister},
  month = {06},
  pages = {602–612},
  publisher = {Association for Computational Linguistics},
  title = {{O}tterly{O}bsessed{W}ith{S}emantics at {S}em{E}val-2024 Task 4: Developing a Hierarchical Multi-Label Classification Head for Large Language Models},
  year = 2024
}

$LL\"aMmlein: Compact and ...$

LL\"aMmlein: Compact and Competitive German-Only Language Models from Scratch. Pfister, Jan; Wunderle, Julia; Hotho, Andreas. 2024.

[ BibTeX ]
[ URL ]
[ Download ]

@misc{pfister2024llammleincompactcompetitivegermanonly,
  author = {Pfister, Jan and Wunderle, Julia and Hotho, Andreas},
  keywords = {author:pfister},
  title = {LL\"aMmlein: Compact and Competitive German-Only Language Models from Scratch},
  year = 2024
}

BibSonomy Meets ChatLLMs for Publication Management: From Chat to Publication Management: Organizing your related work using BibSonomy & LLMs. Völker, Tom; Pfister, Jan; Koopmann, Tobias; Hotho, Andreas. 2024.

[ Abstract ]
[ BibTeX ]
[ URL ]
[ DOI ]
[ Download ]

The ever-growing corpus of scientific literature presents significant challenges for researchers with respect to discovery, management, and annotation of relevant publications. Traditional platforms like Semantic Scholar, BibSonomy, and Zotero offer tools for literature management, but largely require manual laborious and error-prone input of tags and metadata. Here, we introduce a novel retrieval augmented generation system that leverages chat-based large language models (LLMs) to streamline and enhance the process of publication management. It provides a unified chat-based interface, enabling intuitive interactions with various backends, including Semantic Scholar, BibSonomy, and the Zotero Webscraper. It supports two main use-cases: (1) Explorative Search & Retrieval - leveraging LLMs to search for and retrieve both specific and general scientific publications, while addressing the challenges of content hallucination and data obsolescence; and (2) Cataloguing & Management - aiding in the organization of personal publication libraries, in this case BibSonomy, by automating the addition of metadata and tags, while facilitating manual edits and updates. We compare our system to different LLM models in three different settings, including a user study, and we can show its advantages in different metrics.

@misc{volker2024bibsonomy,
  abstract = {The ever-growing corpus of scientific literature presents significant challenges for researchers with respect to discovery, management, and annotation of relevant publications. Traditional platforms like Semantic Scholar, BibSonomy, and Zotero offer tools for literature management, but largely require manual laborious and error-prone input of tags and metadata. Here, we introduce a novel retrieval augmented generation system that leverages chat-based large language models (LLMs) to streamline and enhance the process of publication management. It provides a unified chat-based interface, enabling intuitive interactions with various backends, including Semantic Scholar, BibSonomy, and the Zotero Webscraper. It supports two main use-cases: (1) Explorative Search & Retrieval - leveraging LLMs to search for and retrieve both specific and general scientific publications, while addressing the challenges of content hallucination and data obsolescence; and (2) Cataloguing & Management - aiding in the organization of personal publication libraries, in this case BibSonomy, by automating the addition of metadata and tags, while facilitating manual edits and updates. We compare our system to different LLM models in three different settings, including a user study, and we can show its advantages in different metrics.},
  author = {Völker, Tom and Pfister, Jan and Koopmann, Tobias and Hotho, Andreas},
  keywords = {from:janpf},
  note = {cite arxiv:2401.09092Comment: Accepted at 2024 ACM SIGIR CHIIR, For a demo see here http://professor-x.de/demos/bibsonomy-chatgpt/demo.mp4},
  title = {BibSonomy Meets ChatLLMs for Publication Management: From Chat to Publication Management: Organizing your related work using BibSonomy & LLMs},
  year = 2024
}

The {F}airy{N}et Corpus - Character Networks for {G}erman Fairy Tales. Schmidt, David; Zehe, Albin; Lorenzen, Janne; Sergel, Lisa; D{\"u}ker, Sebastian; Krug, Markus; Puppe, Frank. In Proceedings of the 5th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pp. 49–56. Association for Computational Linguistics, Punta Cana, Dominican Republic (online), 2021.

[ Abstract ]
[ BibTeX ]
[ URL ]
[ DOI ]
[ Download ]

This paper presents a data set of German fairy tales, manually annotated with character networks which were obtained with high inter rater agreement. The release of this corpus provides an opportunity of training and comparing different algorithms for the extraction of character networks, which so far was barely possible due to heterogeneous interests of previous researchers. We demonstrate the usefulness of our data set by providing baseline experiments for the automatic extraction of character networks, applying a rule-based pipeline as well as a neural approach, and find the neural approach outperforming the rule-approach in most evaluation settings.

@inproceedings{schmidt2021fairynet,
  abstract = {This paper presents a data set of German fairy tales, manually annotated with character networks which were obtained with high inter rater agreement. The release of this corpus provides an opportunity of training and comparing different algorithms for the extraction of character networks, which so far was barely possible due to heterogeneous interests of previous researchers. We demonstrate the usefulness of our data set by providing baseline experiments for the automatic extraction of character networks, applying a rule-based pipeline as well as a neural approach, and find the neural approach outperforming the rule-approach in most evaluation settings.},
  address = {Punta Cana, Dominican Republic (online)},
  author = {Schmidt, David and Zehe, Albin and Lorenzen, Janne and Sergel, Lisa and D{\"u}ker, Sebastian and Krug, Markus and Puppe, Frank},
  booktitle = {Proceedings of the 5th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature},
  keywords = {networks},
  month = 11,
  pages = {49–56},
  publisher = {Association for Computational Linguistics},
  title = {The {F}airy{N}et Corpus - Character Networks for {G}erman Fairy Tales},
  year = 2021
}

Detecting Scenes in Fiction: A new Segmentation Task. Zehe, Albin; Konle, Leonard; Dümpelmann, Lea; Gius, Evelyn; Hotho, Andreas; Jannidis, Fotis; Kaufmann, Lucas; Krug, Markus; Puppe, Frank; Reiter, Nils; Schreiber, Annekea; Wiedmer, Nathalie. In Proceedings of the 16th Conference of the {E}uropean Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. ACL, 2021.

[ BibTeX ]
[ Download ]

@inproceedings{zehe2021detecting,
  author = {Zehe, Albin and Konle, Leonard and Dümpelmann, Lea and Gius, Evelyn and Hotho, Andreas and Jannidis, Fotis and Kaufmann, Lucas and Krug, Markus and Puppe, Frank and Reiter, Nils and Schreiber, Annekea and Wiedmer, Nathalie},
  booktitle = {Proceedings of the 16th Conference of the {E}uropean Chapter of the Association for Computational Linguistics: Volume 1, Long Papers},
  keywords = {author:zehe},
  publisher = {ACL},
  title = {Detecting Scenes in Fiction: A new Segmentation Task},
  year = 2021
}

Shared Task on Scene Segmentation @ KONVENS 2021. Zehe, Albin; Konle, Leonard; Guhr, Svenja; Dümpelmann, Lea; Gius, Evelyn; Hotho, Andreas; Jannidis, Fotis; Kaufmann, Lucas; Krug, Markus; Puppe, Frank; Reiter, Nils; Schreiber, Annekea. In Shared Task on Scene Segmentation @ KONVENS 2021, pp. 1–21. 2021.

[ BibTeX ]
[ URL ]
[ Download ]

@inproceedings{zehe2021shared,
  author = {Zehe, Albin and Konle, Leonard and Guhr, Svenja and Dümpelmann, Lea and Gius, Evelyn and Hotho, Andreas and Jannidis, Fotis and Kaufmann, Lucas and Krug, Markus and Puppe, Frank and Reiter, Nils and Schreiber, Annekea},
  booktitle = {Shared Task on Scene Segmentation @ KONVENS 2021},
  keywords = {author:zehe},
  pages = {1-21},
  title = {Shared Task on Scene Segmentation @ KONVENS 2021},
  year = 2021
}

Emote-Controlled: Obtaining Implicit Viewer Feedback through Emote based Sentiment Analysis on Comments of Popular Twitch.tv Channels. Kobs, Konstantin; Zehe, Albin; Bernstetter, Armin; Chibane, Julian; Pfister, Jan; Tritscher, Julian; Hotho, Andreas. In ACM Transactions on Social Computing. 2020.

[ BibTeX ]
[ DOI ]
[ Download ]

@article{kobs2020emotecontrolled,
  author = {Kobs, Konstantin and Zehe, Albin and Bernstetter, Armin and Chibane, Julian and Pfister, Jan and Tritscher, Julian and Hotho, Andreas},
  journal = {ACM Transactions on Social Computing},
  keywords = {sentiment-analysis},
  title = {Emote-Controlled: Obtaining Implicit Viewer Feedback through Emote based Sentiment Analysis on Comments of Popular Twitch.tv Channels},
  year = 2020
}

HarryMotions – Classifying Relationships in Harry Potter based on Emotion Analysis. Zehe, Albin; Arns, Julia; Hettinger, Lena; Hotho, Andreas. In 5th SwissText & 16th KONVENS Joint Conference. 2020.

[ BibTeX ]
[ Download ]

@inproceedings{zehe2020harrymotions,
  author = {Zehe, Albin and Arns, Julia and Hettinger, Lena and Hotho, Andreas},
  booktitle = {5th SwissText & 16th KONVENS Joint Conference},
  keywords = {author:zehe},
  title = {HarryMotions – Classifying Relationships in Harry Potter based on Emotion Analysis},
  year = 2020
}

LM4KG: Improving Common Sense Knowledge Graphs with Language Models. Omeliyanenko, Janna; Zehe, Albin; Hettinger, Lena; Hotho, Andreas. In International Semantic Web Conference. Springer, 2020.

[ BibTeX ]
[ Download ]

@inproceedings{omeliyanenko2020reweight,
  author = {Omeliyanenko, Janna and Zehe, Albin and Hettinger, Lena and Hotho, Andreas},
  booktitle = {International Semantic Web Conference},
  keywords = {graph},
  organization = {Springer},
  title = {LM4KG: Improving Common Sense Knowledge Graphs with Language Models},
  year = 2020
}

Detection of Scenes in Fiction. Gius, Evelyn; Jannidis, Fotis; Krug, Markus; Zehe, Albin; Hotho, Andreas; Puppe, Frank; Krebs, Jonathan; Reiter, Nils; Wiedmer, Nathalie; Konle, Leonard. In Proceedings of Digital Humanities 2019. 2019.

[ BibTeX ]
[ Download ]

@inproceedings{gius2019detection,
  author = {Gius, Evelyn and Jannidis, Fotis and Krug, Markus and Zehe, Albin and Hotho, Andreas and Puppe, Frank and Krebs, Jonathan and Reiter, Nils and Wiedmer, Nathalie and Konle, Leonard},
  booktitle = {Proceedings of Digital Humanities 2019},
  keywords = {author:zehe},
  title = {Detection of Scenes in Fiction},
  year = 2019
}

Analysing Direct Speech in German Novels. Jannidis, Fotis; Konle, Leonard; Zehe, Albin; Hotho, Andreas; Krug, Markus. In DHd 2018. 2018.

[ BibTeX ]
[ Download ]

@inproceedings{jannidis2018analysing,
  author = {Jannidis, Fotis and Konle, Leonard and Zehe, Albin and Hotho, Andreas and Krug, Markus},
  booktitle = {DHd 2018},
  keywords = {directspeech},
  title = {Analysing Direct Speech in German Novels},
  year = 2018
}

Burrows’ Zeta: Exploring and Evaluating Variants and Parameters. Schöch, Christof; Schlör, Daniel; Zehe, Albin; Gebhard, Henning; Becker, Martin; Hotho, Andreas. In DH, pp. 274–277. 2018.

[ BibTeX ]
[ URL ]

@inproceedings{schoech2018zeta,
  author = {Schöch, Christof and Schlör, Daniel and Zehe, Albin and Gebhard, Henning and Becker, Martin and Hotho, Andreas},
  booktitle = {DH},
  keywords = {zeta},
  pages = {274-277},
  title = {Burrows’ Zeta: Exploring and Evaluating Variants and Parameters},
  year = 2018
}

A White-Box Model for Detecting Author Nationality by Linguistic Differences in Spanish Novels. Zehe, Albin; Schlör, Daniel; Henny-Krahmer, Ulrike; Becker, Martin; Hotho, Andreas. In DH. ADHO, 2018.

[ BibTeX ]
[ Download ]

@inproceedings{zehe2018whitebox,
  author = {Zehe, Albin and Schlör, Daniel and Henny-Krahmer, Ulrike and Becker, Martin and Hotho, Andreas},
  booktitle = {DH},
  keywords = {classification},
  organization = {ADHO},
  title = {A White-Box Model for Detecting Author Nationality by Linguistic Differences in Spanish Novels},
  year = 2018
}

Burrows Zeta: Varianten und Evaluation. Schöch, Christof; Calvo, José; Zehe, Albin; Hotho, Andreas. In DHd 2018. 2018.

[ BibTeX ]
[ Download ]

@inproceedings{schoch2018burrows,
  author = {Schöch, Christof and Calvo, José and Zehe, Albin and Hotho, Andreas},
  booktitle = {DHd 2018},
  keywords = {zeta},
  title = {Burrows Zeta: Varianten und Evaluation},
  year = 2018
}

Towards Sentiment Analysis on German Literature. Zehe, Albin; Becker, Martin; Jannidis, Fotis; Hotho, Andreas. 2017.

[ BibTeX ]
[ Download ]

@inproceedings{zehe2017sentiment,
  author = {Zehe, Albin and Becker, Martin and Jannidis, Fotis and Hotho, Andreas},
  keywords = {author:zehe},
  title = {Towards Sentiment Analysis on German Literature},
  year = 2017
}

Prediction of Happy Endings in German Novels. Zehe, Albin; Becker, Martin; Hettinger, Lena; Hotho, Andreas; Reger, Isabella; Jannidis, Fotis. In Proceedings of the Workshop on Interactions between Data Mining and Natural Language Processing 2016, P. Cellier, T. Charnois, A. Hotho, S. Matwin, M.-F. Moens, Y. Toussaint (eds.), pp. 9–16. 2016.

[ Abstract ]
[ BibTeX ]
[ URL ]
[ Download ]

Identifying plot structure in novels is a valuable step towards automatic processing of literary corpora. We present an approach to classify novels as either having a happy ending or not. To achieve this, we use features based on different sentiment lexica as input for an SVM- classifier, which yields an average F1-score of about 73%.

@inproceedings{zehe2016prediction,
  abstract = {Identifying plot structure in novels is a valuable step towards automatic processing of literary corpora. We present an approach to classify novels as either having a happy ending or not. To achieve this, we use features based on different sentiment lexica as input for an SVM- classifier, which yields an average F1-score of about 73%.},
  author = {Zehe, Albin and Becker, Martin and Hettinger, Lena and Hotho, Andreas and Reger, Isabella and Jannidis, Fotis},
  booktitle = {Proceedings of the Workshop on Interactions between Data Mining and Natural Language Processing 2016},
  editor = {Cellier, Peggy and Charnois, Thierry and Hotho, Andreas and Matwin, Stan and Moens, Marie-Francine and Toussaint, Yannick},
  keywords = {novels},
  month = {07},
  pages = {9-16},
  title = {Prediction of Happy Endings in German Novels},
  year = 2016
}

Classification of Literary Subgenres. Hettinger, Lena; Jannidis, Fotis; Reger, Isabella; Hotho, Andreas. In DHd 2016. 2016.

[ BibTeX ]
[ Download ]

@inproceedings{hettinger2016classification,
  author = {Hettinger, Lena and Jannidis, Fotis and Reger, Isabella and Hotho, Andreas},
  booktitle = {DHd 2016},
  keywords = {classification},
  title = {Classification of Literary Subgenres},
  year = 2016
}

Straight Talk! Automatic Recognition of Direct Speech in Nineteenth-Century French Novels. Schöch, Christof; Schlör, Daniel; Popp, Stefanie; Brunner, Annelen; Henny, Ulrike; Tello, Jos{\’e} Calvo. In DH, pp. 346–353. 2016.

[ BibTeX ]
[ Download ]

@inproceedings{schoch2016straight,
  author = {Schöch, Christof and Schlör, Daniel and Popp, Stefanie and Brunner, Annelen and Henny, Ulrike and Tello, Jos{\'e} Calvo},
  booktitle = {DH},
  keywords = {directspeech},
  pages = {346–353},
  title = {Straight Talk! Automatic Recognition of Direct Speech in Nineteenth-Century French Novels.},
  year = 2016
}

Analyzing Features for the Detection of Happy Endings in German Novels. Jannidis, Fotis; Reger, Isabella; Zehe, Albin; Becker, Martin; Hettinger, Lena; Hotho, Andreas. 2016.

[ Abstract ]
[ BibTeX ]
[ URL ]
[ Download ]

With regard to a computational representation of literary plot, this paper looks at the use of sentiment analysis for happy ending detection in German novels. Its focus lies on the investigation of previously proposed sentiment features in order to gain insight about the relevance of specific features on the one hand and the implications of their performance on the other hand. Therefore, we study various partitionings of novels, considering the highly variable concept of "ending". We also show that our approach, even though still rather simple, can potentially lead to substantial findings relevant to literary studies.

@misc{jannidis2016analyzing,
  abstract = {With regard to a computational representation of literary plot, this paper looks at the use of sentiment analysis for happy ending detection in German novels. Its focus lies on the investigation of previously proposed sentiment features in order to gain insight about the relevance of specific features on the one hand and the implications of their performance on the other hand. Therefore, we study various partitionings of novels, considering the highly variable concept of "ending". We also show that our approach, even though still rather simple, can potentially lead to substantial findings relevant to literary studies.},
  author = {Jannidis, Fotis and Reger, Isabella and Zehe, Albin and Becker, Martin and Hettinger, Lena and Hotho, Andreas},
  keywords = {author:zehe},
  note = {cite arxiv:1611.09028},
  title = {Analyzing Features for the Detection of Happy Endings in German Novels},
  year = 2016
}

Straight Talk! Automatic Recognition of Direct Speech in Nineteenth-Century French Novels. Sch{\"o}ch, Christof; Schl{\"o}r, Daniel; Popp, Stefanie; Brunner, Annelen; Henny, Ulrike; Tello, Jos{\’e} Calvo. In DH, pp. 346–353. 2016.

[ BibTeX ]
[ Download ]

@inproceedings{schoch2016straight,
  author = {Sch{\"o}ch, Christof and Schl{\"o}r, Daniel and Popp, Stefanie and Brunner, Annelen and Henny, Ulrike and Tello, Jos{\'e} Calvo},
  booktitle = {DH},
  keywords = {directspeech},
  pages = {346–353},
  title = {Straight Talk! Automatic Recognition of Direct Speech in Nineteenth-Century French Novels.},
  year = 2016
}

Significance Testing for the Classification of Literary Subgenres. Hettinger, Lena; Jannidis, Fotis; Reger, Isabella; Hotho, Andreas. In DH 2016. 2016.

[ BibTeX ]
[ Download ]

@inproceedings{hettinger2016significance,
  author = {Hettinger, Lena and Jannidis, Fotis and Reger, Isabella and Hotho, Andreas},
  booktitle = {DH 2016},
  keywords = {classification},
  title = {Significance Testing for the Classification of Literary Subgenres},
  year = 2016
}

Genre classification on German novels. Hettinger, Lena; Becker, Martin; Reger, Isabella; Jannidis, Fotis; Hotho, Andreas. In Proceedings of the 12th International Workshop on Text-based Information Retrieval. 2015.

[ BibTeX ]
[ URL ]
[ Download ]

@inproceedings{schwemmlein2015genre,
  author = {Hettinger, Lena and Becker, Martin and Reger, Isabella and Jannidis, Fotis and Hotho, Andreas},
  booktitle = {Proceedings of the 12th International Workshop on Text-based Information Retrieval},
  keywords = {classification},
  title = {Genre classification on German novels},
  year = 2015
}

Proceedings of the 1st International Workshop on Interactions between Data Mining and Natural Language Processing co-located with The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, DMNLP@PKDD/ECML 2014, Nancy, France, September 15, 2014. Cellier, Peggy; Charnois, Thierry; Hotho, Andreas; Matwin, Stan; Moens, Marie{-}Francine; Toussaint, Yannick. In Vol. 1202 of {CEUR} Workshop Proceedings. CEUR-WS.org, 2014.

[ BibTeX ]
[ URL ]
[ Download ]

@proceedings{cellier2014proceedings,
  editor = {Cellier, Peggy and Charnois, Thierry and Hotho, Andreas and Matwin, Stan and Moens, Marie{-}Francine and Toussaint, Yannick},
  keywords = {from:hotho},
  publisher = {CEUR-WS.org},
  series = {{CEUR} Workshop Proceedings},
  title = {Proceedings of the 1st International Workshop on Interactions between Data Mining and Natural Language Processing co-located with The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, DMNLP@PKDD/ECML 2014, Nancy, France, September 15, 2014},
  volume = 1202,
  year = 2014
}

Deep Representation & Metric Learning

Publications

Bildnachweise