Julia Wunderle, M.Sc. - Data Science Chair

Julia Wunderle

Chair of Data Science (Informatik X)
University of Würzburg
Campus Hubland Nord
Emil-Fischer-Straße 50
97074 Würzburg
Germany

Email: julia.wunderle[at]uni-wuerzburg.de

Phone: (+49 931) 31-80351

Office: Room 50.03.020 (Zentrum für Künstliche Intelligenz und Data Science (CAIDAS))

About Me

I joined the Data Science Chair in December 2024 for my PhD studies after receiving my master's degree in Computer Science at the University of Würzburg.

My current research focuses on advancing the German NLP landscape. As part of this work, we trained both decoder and encoder model families from scratch (LLäMmlein and ModernGBERT) and maintain the first, comprehensive German NLU benchmark, SuperGLEBer.

Teaching

Seminar: Ausgewählte Themen des Machine Learning (SS'25 )

Other

First Place at SemEval 24 Task 4 & Overall Best Paper Honorable Mention Award
First Place at GermEval 25 Shared Task "Harmful Content Detection" in two subtasks, and second in the third subtask

Publications

2025[ to top ]

Die SuperGLEBer at GermEval 2025 Shared Tasks: Growing Pains - When More Isn’t Always Better. Wunderle, Julia; Pfister, Jan; Hotho, Andreas. In Proceedings of the 21st Conference on Natural Language Processing (KONVENS 2025): Workshops, C. Wartena, U. Heid (eds.), pp. 479–493. HsH Applied Academics, Hannover, Germany, 2025.
- [ BibTeX ]
- [ URL ]
- [ Download ]
@inproceedings{wunderle-etal-2025-die, address = {Hannover, Germany}, author = {Wunderle, Julia and Pfister, Jan and Hotho, Andreas}, booktitle = {Proceedings of the 21st Conference on Natural Language Processing (KONVENS 2025): Workshops}, editor = {Wartena, Christian and Heid, Ulrich}, keywords = {author:pfister}, month = {09}, pages = {479–493}, publisher = {HsH Applied Academics}, title = {Die {S}uper{GLEB}er at {G}erm{E}val 2025 Shared Tasks: Growing Pains - When More Isn{'}t Always Better}, year = 2025 }
LLäMmlein: Transparent, Compact and Competitive German-Only Language Models from Scratch. Pfister, Jan; Wunderle, Julia; Hotho, Andreas. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), W. Che, J. Nabende, E. Shutova, M. T. Pilehvar (eds.), pp. 2227–2246. Association for Computational Linguistics, Vienna, Austria, 2025.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ Download ]
We transparently create two German-only decoder models, LLäMmlein 120M and 1B, from scratch and publish them, along with the training data, for the (German) NLP research community to use. The model training involved several key steps, including data preprocessing/filtering, the creation of a German tokenizer, the training itself, as well as the evaluation of the final models on various benchmarks, also against existing models. Throughout the training process, multiple checkpoints were saved in equal intervals and analyzed using the German SuperGLEBer benchmark to gain insights into the models' learning process.Compared to state-of-the-art models on the SuperGLEBer benchmark, both LLäMmlein models performed competitively, consistently matching or surpassing models with similar parameter sizes. The results show that the models' quality scales with size as expected, but performance improvements on some tasks plateaued early during training, offering valuable insights into resource allocation for future models.

@inproceedings{pfister-etal-2025-llammlein, abstract = {We transparently create two German-only decoder models, LL{\"a}Mmlein 120M and 1B, from scratch and publish them, along with the training data, for the (German) NLP research community to use. The model training involved several key steps, including data preprocessing/filtering, the creation of a German tokenizer, the training itself, as well as the evaluation of the final models on various benchmarks, also against existing models. Throughout the training process, multiple checkpoints were saved in equal intervals and analyzed using the German SuperGLEBer benchmark to gain insights into the models' learning process.Compared to state-of-the-art models on the SuperGLEBer benchmark, both LL{\"a}Mmlein models performed competitively, consistently matching or surpassing models with similar parameter sizes. The results show that the models' quality scales with size as expected, but performance improvements on some tasks plateaued early during training, offering valuable insights into resource allocation for future models.}, address = {Vienna, Austria}, author = {Pfister, Jan and Wunderle, Julia and Hotho, Andreas}, booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)}, editor = {Che, Wanxiang and Nabende, Joyce and Shutova, Ekaterina and Pilehvar, Mohammad Taher}, keywords = {author:pfister}, month = {07}, pages = {2227–2246}, publisher = {Association for Computational Linguistics}, title = {LLäMmlein: Transparent, Compact and Competitive {G}erman-Only Language Models from Scratch}, year = 2025 }
ModernGBERT: German-only 1B Encoder Model Trained from Scratch. Ehrmanntraut, Anton; Wunderle, Julia; Pfister, Jan; Jannidis, Fotis; Hotho, Andreas. 2025.
- [ BibTeX ]
- [ URL ]
- [ Download ]
@misc{ehrmanntraut2025moderngbertgermanonly1bencoder, author = {Ehrmanntraut, Anton and Wunderle, Julia and Pfister, Jan and Jannidis, Fotis and Hotho, Andreas}, keywords = {author:pfister}, title = {ModernGBERT: German-only 1B Encoder Model Trained from Scratch}, year = 2025 }

2024[ to top ]

OtterlyObsessedWithSemantics at SemEval-2024 Task 4: Developing a Hierarchical Multi-Label Classification Head for Large Language Models. Wunderle, Julia; Schubert, Julian; Cacciatore, Antonella; Zehe, Albin; Pfister, Jan; Hotho, Andreas. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), A. K. Ojha, A. S. Dougruöz, H. Tayyar Madabushi, G. Da San Martino, S. Rosenthal, A. Rosá (eds.), pp. 602–612. Association for Computational Linguistics, Mexico City, Mexico, 2024.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ Download ]
For our submission for Subtask 1, we developed a custom classification head that is designed to be applied atop of a Large Language Model. We reconstructed the hierarchy across multiple fully connected layers, allowing us to incorporate previous foundational decisions in subsequent, more fine-grained layers. To find the best hyperparameters, we conducted a grid-search and to compete in the multilingual setting, we translated all documents to English.

@inproceedings{wunderle-etal-2024-otterlyobsessedwithsemantics, abstract = {For our submission for Subtask 1, we developed a custom classification head that is designed to be applied atop of a Large Language Model. We reconstructed the hierarchy across multiple fully connected layers, allowing us to incorporate previous foundational decisions in subsequent, more fine-grained layers. To find the best hyperparameters, we conducted a grid-search and to compete in the multilingual setting, we translated all documents to English.}, address = {Mexico City, Mexico}, author = {Wunderle, Julia and Schubert, Julian and Cacciatore, Antonella and Zehe, Albin and Pfister, Jan and Hotho, Andreas}, booktitle = {Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)}, editor = {Ojha, Atul Kr. and Do{\u{g}}ru{\"o}z, A. Seza and Tayyar Madabushi, Harish and Da San Martino, Giovanni and Rosenthal, Sara and Ros{\'a}, Aiala}, keywords = {author:pfister}, month = {06}, pages = {602–612}, publisher = {Association for Computational Linguistics}, title = {{O}tterly{O}bsessed{W}ith{S}emantics at {S}em{E}val-2024 Task 4: Developing a Hierarchical Multi-Label Classification Head for Large Language Models}, year = 2024 }

2023[ to top ]

Pointer Networks: A Unified Approach to Extracting German Opinions. Wunderle, Julia; Pfister, Jan; Hotho, Andreas. In Proceedings of the 19th Conference on Natural Language Processing (KONVENS 2023), M. Georges, A. Herygers, A. Friedrich, B. Roth (eds.), pp. 127–138. Association for Computational Lingustics, Ingolstadt, Germany, 2023.
- [ BibTeX ]
- [ URL ]
- [ Download ]
@inproceedings{wunderle-etal-2023-pointer, address = {Ingolstadt, Germany}, author = {Wunderle, Julia and Pfister, Jan and Hotho, Andreas}, booktitle = {Proceedings of the 19th Conference on Natural Language Processing (KONVENS 2023)}, editor = {Georges, Munir and Herygers, Aaricia and Friedrich, Annemarie and Roth, Benjamin}, keywords = {from:janpf}, month = {09}, pages = {127–138}, publisher = {Association for Computational Lingustics}, title = {Pointer Networks: A Unified Approach to Extracting {G}erman Opinions}, year = 2023 }

Bildnachweise