Jan Pfister, M.Sc.

Chair of Data Science (Informatik X)
University of Würzburg
Am Hubland
97074 Würzburg
Germany

Email: pfister@informatik.uni-wuerzburg.de

Phone: (+49 931) 31 - 81934

Office: Room B106 (Computer Science Building M2)

PGP Fingerprint: 9131 6BAF 85DE CFB3 AF89 86C9 568C 615A F18B C9C0

Research Interests & Project

I work in the field of Natural Language Processing (NLP) and in particular, I am interested in developing novel methods for understanding and extracting meaning from text. My work focuses on using large language models also in combination with pointer networks to capture the complexities of human language. I am currently working on applying these techniques to the task of aspect-based sentiment analysis, in order to extract fine-grained sentiment information from text.

I joined the DMIR group for my PhD studies after receiving my master's degree in Computer Science at the University of Würzburg in 2021. In the MOTIV project, we are working with voice-based digital assistants like Alexa, with the goal of educating users about the potential negative effects of mindless interactions and misconceptions.

Teaching

- Seminar: Ausgewählte Themen des Machine Learning (SS + WS '21)

- Lecture: Information Retrieval (SS '22)

- Project: Machine Learning in Natural Language Processing (since '22)

Publications

2023[ to top ]

Long-Term Effects of Perceived Friendship with Intelligent Voice Assistants on Usage Behavior, User Experience, and Social Perceptions Wienrich, Carolin; Carolus, Astrid; Markus, André; Augustin, Yannik; Pfister, Jan; Hotho, Andreas in Computers (2023). 12(4)
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ BibSonomy-Post ]
Social patterns and roles can develop when users talk to intelligent voice assistants (IVAs) daily. The current study investigates whether users assign different roles to devices and how this affects their usage behavior, user experience, and social perceptions. Since social roles take time to establish, we equipped 106 participants with Alexa or Google assistants and some smart home devices and observed their interactions for nine months. We analyzed diverse subjective (questionnaire) and objective data (interaction data). By combining social science and data science analyses, we identified two distinct clusters—users who assigned a friendship role to IVAs over time and users who did not. Interestingly, these clusters exhibited significant differences in their usage behavior, user experience, and social perceptions of the devices. For example, participants who assigned a role to IVAs attributed more friendship to them used them more frequently, reported more enjoyment during interactions, and perceived more empathy for IVAs. In addition, these users had distinct personal requirements, for example, they reported more loneliness. This study provides valuable insights into the role-specific effects and consequences of voice assistants. Recent developments in conversational language models such as ChatGPT suggest that the findings of this study could make an important contribution to the design of dialogic human–AI interactions.

@article{wienrich2023longterm, abstract = {Social patterns and roles can develop when users talk to intelligent voice assistants (IVAs) daily. The current study investigates whether users assign different roles to devices and how this affects their usage behavior, user experience, and social perceptions. Since social roles take time to establish, we equipped 106 participants with Alexa or Google assistants and some smart home devices and observed their interactions for nine months. We analyzed diverse subjective (questionnaire) and objective data (interaction data). By combining social science and data science analyses, we identified two distinct clusters—users who assigned a friendship role to IVAs over time and users who did not. Interestingly, these clusters exhibited significant differences in their usage behavior, user experience, and social perceptions of the devices. For example, participants who assigned a role to IVAs attributed more friendship to them used them more frequently, reported more enjoyment during interactions, and perceived more empathy for IVAs. In addition, these users had distinct personal requirements, for example, they reported more loneliness. This study provides valuable insights into the role-specific effects and consequences of voice assistants. Recent developments in conversational language models such as ChatGPT suggest that the findings of this study could make an important contribution to the design of dialogic human–AI interactions.}, author = {Wienrich, Carolin and Carolus, Astrid and Markus, André and Augustin, Yannik and Pfister, Jan and Hotho, Andreas}, journal = {Computers}, keywords = {myown}, number = 4, title = {Long-Term Effects of Perceived Friendship with Intelligent Voice Assistants on Usage Behavior, User Experience, and Social Perceptions}, volume = 12, year = 2023 }

2022[ to top ]

Point me to your Opinion, SenPoi Pfister, Jan; Wankerl, Sebastian; Hotho, Andreas in Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022) (2022). 1313–1323.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ Download ]
- [ BibSonomy-Post ]
Structured Sentiment Analysis is the task of extracting sentiment tuples in a graph structure commonly from review texts. We adapt the Aspect-Based Sentiment Analysis pointer network BARTABSA to model this tuple extraction as a sequence prediction task and extend their output grammar to account for the increased complexity of Structured Sentiment Analysis. To predict structured sentiment tuples in languages other than English we swap BART for a multilingual mT5 and introduce a novel Output Length Regularization to mitigate overfitting to common target sequence lengths, thereby improving the performance of the model by up to 70%. We evaluate our approach on seven datasets in five languages including a zero shot crosslingual setting.

@inproceedings{pfister-etal-2022-senpoi, abstract = {Structured Sentiment Analysis is the task of extracting sentiment tuples in a graph structure commonly from review texts. We adapt the Aspect-Based Sentiment Analysis pointer network BARTABSA to model this tuple extraction as a sequence prediction task and extend their output grammar to account for the increased complexity of Structured Sentiment Analysis. To predict structured sentiment tuples in languages other than English we swap BART for a multilingual mT5 and introduce a novel Output Length Regularization to mitigate overfitting to common target sequence lengths, thereby improving the performance of the model by up to 70{\%}. We evaluate our approach on seven datasets in five languages including a zero shot crosslingual setting.}, address = {Seattle, United States}, author = {Pfister, Jan and Wankerl, Sebastian and Hotho, Andreas}, booktitle = {Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)}, keywords = {sentiment}, month = {jul}, pages = {1313–1323}, publisher = {Association for Computational Linguistics}, title = {Point me to your Opinion, SenPoi}, year = 2022 }

2021[ to top ]

Self-Supervised Multi-Task Pretraining Improves Image Aesthetic Assessment Pfister, Jan; Kobs, Konstantin; Hotho, Andreas in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2021). 816–825.
- [ BibTeX ]
- [ URL ]
- [ BibSonomy-Post ]
@inproceedings{Pfister_2021_CVPR, author = {Pfister, Jan and Kobs, Konstantin and Hotho, Andreas}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops}, keywords = {self-supervised}, month = {June}, pages = {816-825}, title = {Self-Supervised Multi-Task Pretraining Improves Image Aesthetic Assessment}, year = 2021 }

2020[ to top ]

Emote-Controlled: Obtaining Implicit Viewer Feedback through Emote based Sentiment Analysis on Comments of Popular Twitch.tv Channels Kobs, Konstantin; Zehe, Albin; Bernstetter, Armin; Chibane, Julian; Pfister, Jan; Tritscher, Julian; Hotho, Andreas in ACM Transactions on Social Computing (2020). 3(2) 1–34.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ Download ]
- [ BibSonomy-Post ]
In recent years, streaming platforms for video games have seen increasingly large interest, as so-called esports have developed into a lucrative branch of business. Like for other sports, watching esports has become a new kind of entertainment medium, which is possible due to platforms that allow gamers to live stream their gameplay, the most popular platform being Twitch.tv. On these platforms, users can comment on streams in real time and thereby express their opinion about the events in the stream. Due to the popularity of Twitch.tv, this can be a valuable source of feedback for streamers aiming to improve their reception in a gaming-oriented audience. In this work, we explore the possibility of deriving feedback for video streams on Twitch.tv by analyzing the sentiment of live text comments made by stream viewers in highly active channels. Automatic sentiment analysis on these comments is a challenging task, as one can compare the language used in Twitch.tv with that used by an audience in a stadium, shouting as loud as possible in sometimes nonorganized ways. This language is very different from common English, mixing Internet slang and gaming-related language with abbreviations, intentional and unintentional grammatical and orthographic mistakes, and emoji-like images called emotes. Classic lexicon-based sentiment analysis techniques therefore fail when applied to Twitch comments. To overcome the challenge posed by the nonstandard language, we propose two unsupervised lexicon-based approaches that make heavy use of the information encoded in emotes, as well as a weakly supervised neural network–based classifier trained on the lexicon-based outputs, which is supposed to help generalization to unknown words by use of domain-specific word embeddings. To enable better understanding of Twitch.tv comments, we analyze a large dataset of comments, uncovering specific properties of their language, and provide a smaller set of comments labeled with sentiment information by crowdsourcing. We present two case studies showing the effectiveness of our methods in generating sentiment trajectories for events live streamed on Twitch.tv that correlate well with specific topics in the given stream. This allows for a new kind of implicit real-time feedback gathering for Twitch streamers and companies producing games or streaming content on Twitch. We make our datasets and code publicly available for further research.

@article{emote-controlled, abstract = {In recent years, streaming platforms for video games have seen increasingly large interest, as so-called esports have developed into a lucrative branch of business. Like for other sports, watching esports has become a new kind of entertainment medium, which is possible due to platforms that allow gamers to live stream their gameplay, the most popular platform being Twitch.tv. On these platforms, users can comment on streams in real time and thereby express their opinion about the events in the stream. Due to the popularity of Twitch.tv, this can be a valuable source of feedback for streamers aiming to improve their reception in a gaming-oriented audience. In this work, we explore the possibility of deriving feedback for video streams on Twitch.tv by analyzing the sentiment of live text comments made by stream viewers in highly active channels. Automatic sentiment analysis on these comments is a challenging task, as one can compare the language used in Twitch.tv with that used by an audience in a stadium, shouting as loud as possible in sometimes nonorganized ways. This language is very different from common English, mixing Internet slang and gaming-related language with abbreviations, intentional and unintentional grammatical and orthographic mistakes, and emoji-like images called emotes. Classic lexicon-based sentiment analysis techniques therefore fail when applied to Twitch comments. To overcome the challenge posed by the nonstandard language, we propose two unsupervised lexicon-based approaches that make heavy use of the information encoded in emotes, as well as a weakly supervised neural network–based classifier trained on the lexicon-based outputs, which is supposed to help generalization to unknown words by use of domain-specific word embeddings. To enable better understanding of Twitch.tv comments, we analyze a large dataset of comments, uncovering specific properties of their language, and provide a smaller set of comments labeled with sentiment information by crowdsourcing. We present two case studies showing the effectiveness of our methods in generating sentiment trajectories for events live streamed on Twitch.tv that correlate well with specific topics in the given stream. This allows for a new kind of implicit real-time feedback gathering for Twitch streamers and companies producing games or streaming content on Twitch. We make our datasets and code publicly available for further research.}, author = {Kobs, Konstantin and Zehe, Albin and Bernstetter, Armin and Chibane, Julian and Pfister, Jan and Tritscher, Julian and Hotho, Andreas}, journal = {{ACM} Transactions on Social Computing}, keywords = {weakly-supervised}, month = {may}, number = 2, pages = {1–34}, publisher = {Association for Computing Machinery ({ACM})}, title = {Emote-Controlled: Obtaining Implicit Viewer Feedback through Emote based Sentiment Analysis on Comments of Popular Twitch.tv Channels}, volume = 3, year = 2020 }

Research Interests & Project

Teaching

Publications

Hinweis zum Datenschutz

Hinweis zum Datenschutz

Bildnachweise