Publications

• Do you Agree? Contrasting Google’s Core Web Vitals and the Impact of Cookie Consent Banners with Actual Web QoE Wehner, Nikolas; Seufert, Michael; Schatz, Raimund; Hoßfeld, Tobias in Quality and User Experience (2023).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  Providing sophisticated web Quality of Experience (QoE) has become paramount for web service providers and network operators alike. Due to advances in web technologies (HTML5, respon- sive design, etc.), traditional web QoE models focusing mainly on loading times have to be refined and improved. In this work, we relate Google’s Core Web Vitals, a set of metrics for improving user experience, to the loading time aspects of web QoE, and investigate whether the Core Web Vitals and web QoE agree on the perceived experience. To this end, we first perform objective measurements in the web using Google’s Lighthouse. To close the gap be- tween metrics and experience, we complement these objective measurements with subjective assessment by performing multiple crowdsourcing QoE studies. For this purpose, we developed CWeQS, a customized framework to emulate the entire web page loading process, and ask users for their experience while controlling the Core Web Vitals, which is available to the public. To properly configure CWeQS for the planned QoE study and the crowdsourcing setup, we conduct pre-studies, in which we evaluate the importance of the loading strategy of a web page and the impor- tance of the user task. The obtained insights allow us to conduct the desired QoE studies for each of the Core Web Vitals. Furthermore, we assess the impact of cookie consent banners, which have become ubiquitous due to regulatory demands, on the Core Web Vitals and investigate their influence on web QoE. Our results suggest that the Core Web Vitals are much less predictive for web QoE than expected and that page loading times remain the main metric and influence factor in this context. We further observe that unobtrusive and acentric cookie consent banners are preferred by end-users and that addi- tional delays caused by interacting with consent banners in order to agree to or reject cookies should be accounted along with the actual page load time to reduce waiting times and thus to improve web QoE.

  @article{info3-article-2023-3, abstract = {Providing sophisticated web Quality of Experience (QoE) has become paramount for web service providers and network operators alike. Due to advances in web technologies (HTML5, respon- sive design, etc.), traditional web QoE models focusing mainly on loading times have to be refined and improved. In this work, we relate Google’s Core Web Vitals, a set of metrics for improving user experience, to the loading time aspects of web QoE, and investigate whether the Core Web Vitals and web QoE agree on the perceived experience. To this end, we first perform objective measurements in the web using Google’s Lighthouse. To close the gap be- tween metrics and experience, we complement these objective measurements with subjective assessment by performing multiple crowdsourcing QoE studies. For this purpose, we developed CWeQS, a customized framework to emulate the entire web page loading process, and ask users for their experience while controlling the Core Web Vitals, which is available to the public. To properly configure CWeQS for the planned QoE study and the crowdsourcing setup, we conduct pre-studies, in which we evaluate the importance of the loading strategy of a web page and the impor- tance of the user task. The obtained insights allow us to conduct the desired QoE studies for each of the Core Web Vitals. Furthermore, we assess the impact of cookie consent banners, which have become ubiquitous due to regulatory demands, on the Core Web Vitals and investigate their influence on web QoE. Our results suggest that the Core Web Vitals are much less predictive for web QoE than expected and that page loading times remain the main metric and influence factor in this context. We further observe that unobtrusive and acentric cookie consent banners are preferred by end-users and that addi- tional delays caused by interacting with consent banners in order to agree to or reject cookies should be accounted along with the actual page load time to reduce waiting times and thus to improve web QoE.}, author = {Wehner, Nikolas and Seufert, Michael and Schatz, Raimund and Hoßfeld, Tobias}, journal = {Quality and User Experience}, keywords = {ucn}, title = {Do you Agree? Contrasting Google’s Core Web Vitals and the Impact of Cookie Consent Banners with Actual Web QoE}, year = 2023 }

  %0 Journal Article %1 info3-article-2023-3 %A Wehner, Nikolas %A Seufert, Michael %A Schatz, Raimund %A Hoßfeld, Tobias %D 2023 %J Quality and User Experience %T Do you Agree? Contrasting Google’s Core Web Vitals and the Impact of Cookie Consent Banners with Actual Web QoE %X Providing sophisticated web Quality of Experience (QoE) has become paramount for web service providers and network operators alike. Due to advances in web technologies (HTML5, respon- sive design, etc.), traditional web QoE models focusing mainly on loading times have to be refined and improved. In this work, we relate Google’s Core Web Vitals, a set of metrics for improving user experience, to the loading time aspects of web QoE, and investigate whether the Core Web Vitals and web QoE agree on the perceived experience. To this end, we first perform objective measurements in the web using Google’s Lighthouse. To close the gap be- tween metrics and experience, we complement these objective measurements with subjective assessment by performing multiple crowdsourcing QoE studies. For this purpose, we developed CWeQS, a customized framework to emulate the entire web page loading process, and ask users for their experience while controlling the Core Web Vitals, which is available to the public. To properly configure CWeQS for the planned QoE study and the crowdsourcing setup, we conduct pre-studies, in which we evaluate the importance of the loading strategy of a web page and the impor- tance of the user task. The obtained insights allow us to conduct the desired QoE studies for each of the Core Web Vitals. Furthermore, we assess the impact of cookie consent banners, which have become ubiquitous due to regulatory demands, on the Core Web Vitals and investigate their influence on web QoE. Our results suggest that the Core Web Vitals are much less predictive for web QoE than expected and that page loading times remain the main metric and influence factor in this context. We further observe that unobtrusive and acentric cookie consent banners are preferred by end-users and that addi- tional delays caused by interacting with consent banners in order to agree to or reject cookies should be accounted along with the actual page load time to reduce waiting times and thus to improve web QoE.
• Explainable Data-Driven QoE Modelling using XAI Wehner, Nikolas; Seufert, Anika; Hoßfeld, Tobias; Seufert, Michael in 15th International Conference on Quality of Multimedia Experience (QoMEX) (2023).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  Data-driven QoE modelling using Machine Learning (ML) allows to reduce the modelling bias and to continuously integrate new QoE results into the QoE model, which can improve its generalizability. The downside is that the majority of ML models are black-box models, which prevent to obtain insights about QoE influence factors and their fundamental relationships that are highly relevant for researchers and providers of services and networks. However, recent advances in the field of eXplainable Artificial Intelligence (XAI) resolve these issues. Thus, XAI allows to benefit from data-driven QoE modelling to obtain generalizable QoE models, and at the same time to understand what QoE factors are relevant and how they affect the QoE score. In this work, we showcase the feasibility of explainable data-driven QoE modelling for video streaming, since video streaming QoE has been well researched, and thus, allows us to validate our results. Finally, we discuss opportunities and challenges of deploying XAI for QoE modelling.

  @inproceedings{info3-inproceedings-2023-4, abstract = {Data-driven QoE modelling using Machine Learning (ML) allows to reduce the modelling bias and to continuously integrate new QoE results into the QoE model, which can improve its generalizability. The downside is that the majority of ML models are black-box models, which prevent to obtain insights about QoE influence factors and their fundamental relationships that are highly relevant for researchers and providers of services and networks. However, recent advances in the field of eXplainable Artificial Intelligence (XAI) resolve these issues. Thus, XAI allows to benefit from data-driven QoE modelling to obtain generalizable QoE models, and at the same time to understand what QoE factors are relevant and how they affect the QoE score. In this work, we showcase the feasibility of explainable data-driven QoE modelling for video streaming, since video streaming QoE has been well researched, and thus, allows us to validate our results. Finally, we discuss opportunities and challenges of deploying XAI for QoE modelling.}, address = {Ghent, Belgium}, author = {Wehner, Nikolas and Seufert, Anika and Hoßfeld, Tobias and Seufert, Michael}, booktitle = {15th International Conference on Quality of Multimedia Experience (QoMEX)}, keywords = {ucn}, month = 6, title = {Explainable Data-Driven QoE Modelling using XAI}, year = 2023 }

  %0 Conference Paper %1 info3-inproceedings-2023-4 %A Wehner, Nikolas %A Seufert, Anika %A Hoßfeld, Tobias %A Seufert, Michael %B 15th International Conference on Quality of Multimedia Experience (QoMEX) %C Ghent, Belgium %D 2023 %T Explainable Data-Driven QoE Modelling using XAI %X Data-driven QoE modelling using Machine Learning (ML) allows to reduce the modelling bias and to continuously integrate new QoE results into the QoE model, which can improve its generalizability. The downside is that the majority of ML models are black-box models, which prevent to obtain insights about QoE influence factors and their fundamental relationships that are highly relevant for researchers and providers of services and networks. However, recent advances in the field of eXplainable Artificial Intelligence (XAI) resolve these issues. Thus, XAI allows to benefit from data-driven QoE modelling to obtain generalizable QoE models, and at the same time to understand what QoE factors are relevant and how they affect the QoE score. In this work, we showcase the feasibility of explainable data-driven QoE modelling for video streaming, since video streaming QoE has been well researched, and thus, allows us to validate our results. Finally, we discuss opportunities and challenges of deploying XAI for QoE modelling.

• Fingerprinting Webpages and Smartphone Apps from Encrypted Network Traffic with WebScanner Casas, Pedro; Wehner, Nikolas; Wassermann, Sarah; Seufert, Michael in Global Internet (GI) Symposium (2022).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  The wide adoption of end-to-end encryption drastically limits the visibility Internet Service Providers (ISPs) have on the services consumed by their customers. This is particularly challenging for monitoring and analysis of web and apps traffic, which is highly complex and heterogeneous. Loading a web page or app today requires tens of flows to download the various resources located in distributed cloud servers from different content providers. We introduce WebScanner, a webpage and app fingerprinting approach capable to identify all the traffic flows corresponding to individual webpage and app loading sessions within concurrent web pages traffic, enabling highly detailed, per webpage analysis in practical deployments. Different from the state of the art in web and app traffic fingerprinting, and as our most relevant contribution, WebScanner automatically performs the parsing of all the traffic generated by a web visit and its isolation from concurrent traffic, instead of assuming that an external oracle system does so. WebScanner also implements a deep fingerprinting approach to detect user action-dependent traffic from apps, relying on simple machine learning models and strong input features as fingerprints. Extensive evaluation across a large measurement dataset of popular webpages and mobile apps confirms the outstanding performance of WebScanner, identifying the top-500 Alexa websites with precision and recall (P/R) above 95%, isolating their full contents with P/R above 80% for up to 15 concurrent webpages, and detecting specific action-dependent apps traffic with average P/R above 92%.

  @inproceedings{info3-inproceedings-2022-18, abstract = {The wide adoption of end-to-end encryption drastically limits the visibility Internet Service Providers (ISPs) have on the services consumed by their customers. This is particularly challenging for monitoring and analysis of web and apps traffic, which is highly complex and heterogeneous. Loading a web page or app today requires tens of flows to download the various resources located in distributed cloud servers from different content providers. We introduce WebScanner, a webpage and app fingerprinting approach capable to identify all the traffic flows corresponding to individual webpage and app loading sessions within concurrent web pages traffic, enabling highly detailed, per webpage analysis in practical deployments. Different from the state of the art in web and app traffic fingerprinting, and as our most relevant contribution, WebScanner automatically performs the parsing of all the traffic generated by a web visit and its isolation from concurrent traffic, instead of assuming that an external oracle system does so. WebScanner also implements a deep fingerprinting approach to detect user action-dependent traffic from apps, relying on simple machine learning models and strong input features as fingerprints. Extensive evaluation across a large measurement dataset of popular webpages and mobile apps confirms the outstanding performance of WebScanner, identifying the top-500 Alexa websites with precision and recall (P/R) above 95%, isolating their full contents with P/R above 80% for up to 15 concurrent webpages, and detecting specific action-dependent apps traffic with average P/R above 92%.}, address = {Paris, France}, author = {Casas, Pedro and Wehner, Nikolas and Wassermann, Sarah and Seufert, Michael}, booktitle = {Global Internet (GI) Symposium}, keywords = {ucn}, month = 11, title = {Fingerprinting Webpages and Smartphone Apps from Encrypted Network Traffic with WebScanner}, year = 2022 }

  %0 Conference Paper %1 info3-inproceedings-2022-18 %A Casas, Pedro %A Wehner, Nikolas %A Wassermann, Sarah %A Seufert, Michael %B Global Internet (GI) Symposium %C Paris, France %D 2022 %T Fingerprinting Webpages and Smartphone Apps from Encrypted Network Traffic with WebScanner %X The wide adoption of end-to-end encryption drastically limits the visibility Internet Service Providers (ISPs) have on the services consumed by their customers. This is particularly challenging for monitoring and analysis of web and apps traffic, which is highly complex and heterogeneous. Loading a web page or app today requires tens of flows to download the various resources located in distributed cloud servers from different content providers. We introduce WebScanner, a webpage and app fingerprinting approach capable to identify all the traffic flows corresponding to individual webpage and app loading sessions within concurrent web pages traffic, enabling highly detailed, per webpage analysis in practical deployments. Different from the state of the art in web and app traffic fingerprinting, and as our most relevant contribution, WebScanner automatically performs the parsing of all the traffic generated by a web visit and its isolation from concurrent traffic, instead of assuming that an external oracle system does so. WebScanner also implements a deep fingerprinting approach to detect user action-dependent traffic from apps, relying on simple machine learning models and strong input features as fingerprints. Extensive evaluation across a large measurement dataset of popular webpages and mobile apps confirms the outstanding performance of WebScanner, identifying the top-500 Alexa websites with precision and recall (P/R) above 95%, isolating their full contents with P/R above 80% for up to 15 concurrent webpages, and detecting specific action-dependent apps traffic with average P/R above 92%.
• A Vital Improvement? Relating Google’s Core Web Vitals to Actual Web QoE Wehner, Nikolas; Amir, Monisha; Seufert, Michael; Schatz, Raimund; Hoßfeld, Tobias in 14th International Conference on Quality of Multimedia Experience (QoMEX), Best Paper Award (2022).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  Providing sophisticated Web Quality of Experience (QoE) has become paramount for web service providers and network operators alike. Due to advances in Web technologies (HTML5, responsive design, etc.), traditional Web QoE models focusing mainly on loading times have to be refined and improved. In this work, we relate Google's Core Web Vitals, a set of metrics for improving user experience, to the loading time aspects of Web QoE. To this end, we first perform objective measurements in the Web using Google's Lighthouse. To close the gap between metrics and experience, we complement these objective measurements with subjective assessment by performing multiple crowdsourcing QoE studies. In these studies, we use CWeQS, a customized framework to emulate the entire web page loading process, and ask users for their experience while controlling the Core Web Vitals. Our results suggest that the Core Web Vitals have less predictive value for Web QoE than expected and that page loading times remain the main influence factor in this context.

  @inproceedings{info3-inproceedings-2022-12, abstract = {Providing sophisticated Web Quality of Experience (QoE) has become paramount for web service providers and network operators alike. Due to advances in Web technologies (HTML5, responsive design, etc.), traditional Web QoE models focusing mainly on loading times have to be refined and improved. In this work, we relate Google's Core Web Vitals, a set of metrics for improving user experience, to the loading time aspects of Web QoE. To this end, we first perform objective measurements in the Web using Google's Lighthouse. To close the gap between metrics and experience, we complement these objective measurements with subjective assessment by performing multiple crowdsourcing QoE studies. In these studies, we use CWeQS, a customized framework to emulate the entire web page loading process, and ask users for their experience while controlling the Core Web Vitals. Our results suggest that the Core Web Vitals have less predictive value for Web QoE than expected and that page loading times remain the main influence factor in this context.}, address = {Lippstadt, Germany}, author = {Wehner, Nikolas and Amir, Monisha and Seufert, Michael and Schatz, Raimund and Hoßfeld, Tobias}, booktitle = {14th International Conference on Quality of Multimedia Experience (QoMEX), Best Paper Award}, keywords = {ucn}, month = 9, title = {A Vital Improvement? Relating Google's Core Web Vitals to Actual Web QoE}, year = 2022 }

  %0 Conference Paper %1 info3-inproceedings-2022-12 %A Wehner, Nikolas %A Amir, Monisha %A Seufert, Michael %A Schatz, Raimund %A Hoßfeld, Tobias %B 14th International Conference on Quality of Multimedia Experience (QoMEX), Best Paper Award %C Lippstadt, Germany %D 2022 %T A Vital Improvement? Relating Google's Core Web Vitals to Actual Web QoE %X Providing sophisticated Web Quality of Experience (QoE) has become paramount for web service providers and network operators alike. Due to advances in Web technologies (HTML5, responsive design, etc.), traditional Web QoE models focusing mainly on loading times have to be refined and improved. In this work, we relate Google's Core Web Vitals, a set of metrics for improving user experience, to the loading time aspects of Web QoE. To this end, we first perform objective measurements in the Web using Google's Lighthouse. To close the gap between metrics and experience, we complement these objective measurements with subjective assessment by performing multiple crowdsourcing QoE studies. In these studies, we use CWeQS, a customized framework to emulate the entire web page loading process, and ask users for their experience while controlling the Core Web Vitals. Our results suggest that the Core Web Vitals have less predictive value for Web QoE than expected and that page loading times remain the main influence factor in this context.
• Not all Web Pages are Born the Same. Content Tailored Learning for Web QoE Inference Casas, Pedro; Wassermann, Sarah; Wehner, Nikolas; Seufert, Michael; Hoßfeld, Tobias in IEEE International Symposium on Measurements & Networking (M&N 2022) (2022).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  Web Quality of Experience (QoE) monitoring is a critical task for Internet Service Providers (ISPs), especially due to the key role played by customer experience in churn management. Previously, we have tackled the problem of Web QoE inference from the ISP perspective, relying on passive measurement of encrypted network traffic and machine learning models. In this paper, we exploit the broad heterogeneity of contents embedded in web pages to improve the state of the art performance in Web QoE inference, relying on web-content learning model tailoring. By analyzing the top-500 most popular web pages of the Internet through unsupervised learning, we discover different web page content classes which realize significantly different Web QoE inference performance. We train supervised learning inference models separately for each of these classes, using the well-known Speed Index (SI) metric as proxy to Web QoE. Empirical evaluations on a large corpus of Web QoE measurements for top popular websites demonstrate that our combined content-tailored approach improves the inference performance of the SI by almost 30% with respect to previous single-model approaches, reducing the QoE inference error in terms of mean opinion scores by more than 40%.

  @inproceedings{info3-inproceedings-2022-10, abstract = {Web Quality of Experience (QoE) monitoring is a critical task for Internet Service Providers (ISPs), especially due to the key role played by customer experience in churn management. Previously, we have tackled the problem of Web QoE inference from the ISP perspective, relying on passive measurement of encrypted network traffic and machine learning models. In this paper, we exploit the broad heterogeneity of contents embedded in web pages to improve the state of the art performance in Web QoE inference, relying on web-content learning model tailoring. By analyzing the top-500 most popular web pages of the Internet through unsupervised learning, we discover different web page content classes which realize significantly different Web QoE inference performance. We train supervised learning inference models separately for each of these classes, using the well-known Speed Index (SI) metric as proxy to Web QoE. Empirical evaluations on a large corpus of Web QoE measurements for top popular websites demonstrate that our combined content-tailored approach improves the inference performance of the SI by almost 30% with respect to previous single-model approaches, reducing the QoE inference error in terms of mean opinion scores by more than 40%.}, author = {Casas, Pedro and Wassermann, Sarah and Wehner, Nikolas and Seufert, Michael and Hoßfeld, Tobias}, booktitle = {IEEE International Symposium on Measurements & Networking (M&N 2022)}, keywords = {sys:hidden:inproceedings}, month = 7, title = {Not all Web Pages are Born the Same. Content Tailored Learning for Web QoE Inference}, year = 2022 }

  %0 Conference Paper %1 info3-inproceedings-2022-10 %A Casas, Pedro %A Wassermann, Sarah %A Wehner, Nikolas %A Seufert, Michael %A Hoßfeld, Tobias %B IEEE International Symposium on Measurements & Networking (M&N 2022) %D 2022 %T Not all Web Pages are Born the Same. Content Tailored Learning for Web QoE Inference %X Web Quality of Experience (QoE) monitoring is a critical task for Internet Service Providers (ISPs), especially due to the key role played by customer experience in churn management. Previously, we have tackled the problem of Web QoE inference from the ISP perspective, relying on passive measurement of encrypted network traffic and machine learning models. In this paper, we exploit the broad heterogeneity of contents embedded in web pages to improve the state of the art performance in Web QoE inference, relying on web-content learning model tailoring. By analyzing the top-500 most popular web pages of the Internet through unsupervised learning, we discover different web page content classes which realize significantly different Web QoE inference performance. We train supervised learning inference models separately for each of these classes, using the well-known Speed Index (SI) metric as proxy to Web QoE. Empirical evaluations on a large corpus of Web QoE measurements for top popular websites demonstrate that our combined content-tailored approach improves the inference performance of the SI by almost 30% with respect to previous single-model approaches, reducing the QoE inference error in terms of mean opinion scores by more than 40%.
• X-Ray Goggles for the ISP: Improving in-Network Web and App QoE Monitoring with Deep Learning Casas, Pedro; Wassermann, Sarah; Seufert, Michael; Wehner, Nikolas; Dinica, Olivia; Hoßfeld, Tobias in Network Traffic Measurement and Analysis Conference (TMA) (2022).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  The wide adoption of end-to-end encryption is drastically limiting the visibility Internet Service Providers (ISPs) have on the performance of the services consumed by their customers. In times of strong competition, where customer experience plays a key role in churn management, ISPs require novel solutions enabling network-wide Quality of Experience (QoE) monitoring. To this end, we present DeepQoE, a deep-learning based approach to infer the QoE of web services and mobile applications from the ISP perspective, relying exclusively on the analysis of encrypted network traffic. Using raw features derived from the encrypted stream of bytes as input to deep Convolutional Neural Networks (CNNs), DeepQoE infers the Speed Index of web browsing sessions and general mobile apps with unprecedented accuracy, improving the state of the art by more than 25%, and reducing the QoE inference error in terms of mean opinion scores by nearly 40%. DeepQoE implements a web fingerprinting solution to identify individual web browsing sessions within concurrent web pages traffic, enabling highly detailed, per web page QoE inference in practical deployments. Extensive evaluations over a large and heterogeneous dataset composed of web and app measurements, using different device types and for top-popular websites and apps, confirm the out-performance of DeepQoE over previously used shallow-learning models, as well as the deep-model generalization to different devices, web pages, apps, and network setups. DeepQoE is the first deployable system providing such a deep, highly-detailed QoE for individual web browsing and mobile apps over encrypted traffic, using deep learning models on heterogeneous measurements.

  @inproceedings{info3-inproceedings-2022-11, abstract = {The wide adoption of end-to-end encryption is drastically limiting the visibility Internet Service Providers (ISPs) have on the performance of the services consumed by their customers. In times of strong competition, where customer experience plays a key role in churn management, ISPs require novel solutions enabling network-wide Quality of Experience (QoE) monitoring. To this end, we present DeepQoE, a deep-learning based approach to infer the QoE of web services and mobile applications from the ISP perspective, relying exclusively on the analysis of encrypted network traffic. Using raw features derived from the encrypted stream of bytes as input to deep Convolutional Neural Networks (CNNs), DeepQoE infers the Speed Index of web browsing sessions and general mobile apps with unprecedented accuracy, improving the state of the art by more than 25%, and reducing the QoE inference error in terms of mean opinion scores by nearly 40%. DeepQoE implements a web fingerprinting solution to identify individual web browsing sessions within concurrent web pages traffic, enabling highly detailed, per web page QoE inference in practical deployments. Extensive evaluations over a large and heterogeneous dataset composed of web and app measurements, using different device types and for top-popular websites and apps, confirm the out-performance of DeepQoE over previously used shallow-learning models, as well as the deep-model generalization to different devices, web pages, apps, and network setups. DeepQoE is the first deployable system providing such a deep, highly-detailed QoE for individual web browsing and mobile apps over encrypted traffic, using deep learning models on heterogeneous measurements.}, author = {Casas, Pedro and Wassermann, Sarah and Seufert, Michael and Wehner, Nikolas and Dinica, Olivia and Hoßfeld, Tobias}, booktitle = {Network Traffic Measurement and Analysis Conference (TMA)}, keywords = {sys:hidden:inproceedings}, month = 6, title = {X-Ray Goggles for the ISP: Improving in-Network Web and App QoE Monitoring with Deep Learning}, year = 2022 }

  %0 Conference Paper %1 info3-inproceedings-2022-11 %A Casas, Pedro %A Wassermann, Sarah %A Seufert, Michael %A Wehner, Nikolas %A Dinica, Olivia %A Hoßfeld, Tobias %B Network Traffic Measurement and Analysis Conference (TMA) %D 2022 %T X-Ray Goggles for the ISP: Improving in-Network Web and App QoE Monitoring with Deep Learning %X The wide adoption of end-to-end encryption is drastically limiting the visibility Internet Service Providers (ISPs) have on the performance of the services consumed by their customers. In times of strong competition, where customer experience plays a key role in churn management, ISPs require novel solutions enabling network-wide Quality of Experience (QoE) monitoring. To this end, we present DeepQoE, a deep-learning based approach to infer the QoE of web services and mobile applications from the ISP perspective, relying exclusively on the analysis of encrypted network traffic. Using raw features derived from the encrypted stream of bytes as input to deep Convolutional Neural Networks (CNNs), DeepQoE infers the Speed Index of web browsing sessions and general mobile apps with unprecedented accuracy, improving the state of the art by more than 25%, and reducing the QoE inference error in terms of mean opinion scores by nearly 40%. DeepQoE implements a web fingerprinting solution to identify individual web browsing sessions within concurrent web pages traffic, enabling highly detailed, per web page QoE inference in practical deployments. Extensive evaluations over a large and heterogeneous dataset composed of web and app measurements, using different device types and for top-popular websites and apps, confirm the out-performance of DeepQoE over previously used shallow-learning models, as well as the deep-model generalization to different devices, web pages, apps, and network setups. DeepQoE is the first deployable system providing such a deep, highly-detailed QoE for individual web browsing and mobile apps over encrypted traffic, using deep learning models on heterogeneous measurements.
• DeepCrypt - Deep Learning for QoE Monitoring and Fingerprinting of User Actions in Adaptive Video Streaming Casas, Pedro; Seufert, Michael; Wassermann, Sarah; Gardlo, Bruno; Wehner, Nikolas; Schatz, Raimund in IEEE Conference on Network Softwarization (IEEE NetSoft 2022) (2022).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  We introduce DeepCrypt, a deep-learning based approach to analyze YouTube video streaming Quality of Experience (QoE) from the Internet Service Provider (ISP) perspective, relying exclusively on the analysis of encrypted network traffic. Using raw features derived on-line from the encrypted stream of bytes, DeepCrypt can infer six different video QoE indicators capturing the user-perceived performance of the service, including the initial playback delay, the number and frequency of rebuffering events, the video playback quality, the video encoding bitrate, and the number of quality changes. DeepCrypt offers deep visibility into the behavior of the end-user, enabling the fingerprinting and detection of different user actions on the video player, such as video pauses and playback scrubbing (forward, backward, out-of-buffer), offering a complete visibility on the video streaming process from in-network traffic measurements. Extensive evaluations over a large and heterogeneous dataset composed of mobile and fixed-line measurements, using the YouTube HTML5 player, the native YouTube mobile app, as well as a generic HTML5 video player built on top of open source libraries, and considering measurements collected at different ISPs, confirm the out-performance of DeepCrypt over previously used shallow-learning models, as well as the deep-model generalization to different video players and network setups. To the best of our knowledge, this is the first paper tackling such a combined QoE and user-actions approach for video streaming applications, using deep learning models over heterogeneous measurements.

  @inproceedings{info3-inproceedings-2022-9, abstract = {We introduce DeepCrypt, a deep-learning based approach to analyze YouTube video streaming Quality of Experience (QoE) from the Internet Service Provider (ISP) perspective, relying exclusively on the analysis of encrypted network traffic. Using raw features derived on-line from the encrypted stream of bytes, DeepCrypt can infer six different video QoE indicators capturing the user-perceived performance of the service, including the initial playback delay, the number and frequency of rebuffering events, the video playback quality, the video encoding bitrate, and the number of quality changes. DeepCrypt offers deep visibility into the behavior of the end-user, enabling the fingerprinting and detection of different user actions on the video player, such as video pauses and playback scrubbing (forward, backward, out-of-buffer), offering a complete visibility on the video streaming process from in-network traffic measurements. Extensive evaluations over a large and heterogeneous dataset composed of mobile and fixed-line measurements, using the YouTube HTML5 player, the native YouTube mobile app, as well as a generic HTML5 video player built on top of open source libraries, and considering measurements collected at different ISPs, confirm the out-performance of DeepCrypt over previously used shallow-learning models, as well as the deep-model generalization to different video players and network setups. To the best of our knowledge, this is the first paper tackling such a combined QoE and user-actions approach for video streaming applications, using deep learning models over heterogeneous measurements.}, author = {Casas, Pedro and Seufert, Michael and Wassermann, Sarah and Gardlo, Bruno and Wehner, Nikolas and Schatz, Raimund}, booktitle = {IEEE Conference on Network Softwarization (IEEE NetSoft 2022)}, keywords = {sys:hidden:inproceedings}, month = 6, title = {DeepCrypt - Deep Learning for QoE Monitoring and Fingerprinting of User Actions in Adaptive Video Streaming}, year = 2022 }

  %0 Conference Paper %1 info3-inproceedings-2022-9 %A Casas, Pedro %A Seufert, Michael %A Wassermann, Sarah %A Gardlo, Bruno %A Wehner, Nikolas %A Schatz, Raimund %B IEEE Conference on Network Softwarization (IEEE NetSoft 2022) %D 2022 %T DeepCrypt - Deep Learning for QoE Monitoring and Fingerprinting of User Actions in Adaptive Video Streaming %X We introduce DeepCrypt, a deep-learning based approach to analyze YouTube video streaming Quality of Experience (QoE) from the Internet Service Provider (ISP) perspective, relying exclusively on the analysis of encrypted network traffic. Using raw features derived on-line from the encrypted stream of bytes, DeepCrypt can infer six different video QoE indicators capturing the user-perceived performance of the service, including the initial playback delay, the number and frequency of rebuffering events, the video playback quality, the video encoding bitrate, and the number of quality changes. DeepCrypt offers deep visibility into the behavior of the end-user, enabling the fingerprinting and detection of different user actions on the video player, such as video pauses and playback scrubbing (forward, backward, out-of-buffer), offering a complete visibility on the video streaming process from in-network traffic measurements. Extensive evaluations over a large and heterogeneous dataset composed of mobile and fixed-line measurements, using the YouTube HTML5 player, the native YouTube mobile app, as well as a generic HTML5 video player built on top of open source libraries, and considering measurements collected at different ISPs, confirm the out-performance of DeepCrypt over previously used shallow-learning models, as well as the deep-model generalization to different video players and network setups. To the best of our knowledge, this is the first paper tackling such a combined QoE and user-actions approach for video streaming applications, using deep learning models over heterogeneous measurements.
• On Learning Hierarchical Embeddings from Encrypted Network Traffic Wehner, Nikolas; Ring, Markus; Schüler, Joshua; Hotho, Andreas; Hoßfeld, Tobias; Seufert, Michael in 33th IEEE/IFIP Network Operations and Management Symposium (NOMS) (2022).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  This work presents a novel concept for learning embeddings from encrypted network traffic. In contrast to existing approaches, we evaluate the feasibility of hierarchical embeddings by iteratively aggregating packet embeddings to flow embeddings, and flow embeddings to trace embeddings. The hierarchical embedding concept was designed to especially consider complex dependencies of Internet traffic on different time scales. We describe this novel embedding concept for the domain of network traffic in full detail, and evaluate its performance for the downstream task of website fingerprinting, i.e., identifying websites from encrypted traffic, which is relevant for network management, e.g., as a prerequisite for QoE monitoring or for intrusion detection. Our evaluation reveals that embeddings are a promising solution for website fingerprinting as our model correctly labels up to 99.8% of traces from 500 target websites.

  @inproceedings{info3-inproceedings-2022-1, abstract = {This work presents a novel concept for learning embeddings from encrypted network traffic. In contrast to existing approaches, we evaluate the feasibility of hierarchical embeddings by iteratively aggregating packet embeddings to flow embeddings, and flow embeddings to trace embeddings. The hierarchical embedding concept was designed to especially consider complex dependencies of Internet traffic on different time scales. We describe this novel embedding concept for the domain of network traffic in full detail, and evaluate its performance for the downstream task of website fingerprinting, i.e., identifying websites from encrypted traffic, which is relevant for network management, e.g., as a prerequisite for QoE monitoring or for intrusion detection. Our evaluation reveals that embeddings are a promising solution for website fingerprinting as our model correctly labels up to 99.8\% of traces from 500 target websites.}, address = {Budapest, Hungary}, author = {Wehner, Nikolas and Ring, Markus and Schüler, Joshua and Hotho, Andreas and Hoßfeld, Tobias and Seufert, Michael}, booktitle = {33th IEEE/IFIP Network Operations and Management Symposium (NOMS)}, keywords = {sys:hidden:inproceedings}, month = 4, title = {On Learning Hierarchical Embeddings from Encrypted Network Traffic}, year = 2022 }

  %0 Conference Paper %1 info3-inproceedings-2022-1 %A Wehner, Nikolas %A Ring, Markus %A Schüler, Joshua %A Hotho, Andreas %A Hoßfeld, Tobias %A Seufert, Michael %B 33th IEEE/IFIP Network Operations and Management Symposium (NOMS) %C Budapest, Hungary %D 2022 %T On Learning Hierarchical Embeddings from Encrypted Network Traffic %X This work presents a novel concept for learning embeddings from encrypted network traffic. In contrast to existing approaches, we evaluate the feasibility of hierarchical embeddings by iteratively aggregating packet embeddings to flow embeddings, and flow embeddings to trace embeddings. The hierarchical embedding concept was designed to especially consider complex dependencies of Internet traffic on different time scales. We describe this novel embedding concept for the domain of network traffic in full detail, and evaluate its performance for the downstream task of website fingerprinting, i.e., identifying websites from encrypted traffic, which is relevant for network management, e.g., as a prerequisite for QoE monitoring or for intrusion detection. Our evaluation reveals that embeddings are a promising solution for website fingerprinting as our model correctly labels up to 99.8% of traces from 500 target websites.

• Mobile Web and App QoE Monitoring for ISPs - from Encrypted Traffic to Speed Index through Machine Learning Casas, Pedro; Wassermann, Sarah; Wehner, Nikolas; Seufert, Michael; Schüler, Joshua; Hoßfeld, Tobias in 13th Wireless and Mobile Networking Conference (WMNC), Best Paper Award (2021).
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{info3-inproceedings-2021-17, address = {Montreal, Canada (Virtual Conference)}, author = {Casas, Pedro and Wassermann, Sarah and Wehner, Nikolas and Seufert, Michael and Schüler, Joshua and Hoßfeld, Tobias}, booktitle = {13th Wireless and Mobile Networking Conference (WMNC), Best Paper Award}, keywords = {ucn}, month = 10, title = {Mobile Web and App QoE Monitoring for ISPs - from Encrypted Traffic to Speed Index through Machine Learning}, year = 2021 }

  %0 Conference Paper %1 info3-inproceedings-2021-17 %A Casas, Pedro %A Wassermann, Sarah %A Wehner, Nikolas %A Seufert, Michael %A Schüler, Joshua %A Hoßfeld, Tobias %B 13th Wireless and Mobile Networking Conference (WMNC), Best Paper Award %C Montreal, Canada (Virtual Conference) %D 2021 %T Mobile Web and App QoE Monitoring for ISPs - from Encrypted Traffic to Speed Index through Machine Learning
• How are your Apps Doing? QoE Inference and Analysis in Mobile Devices Wehner, Nikolas; Seufert, Michael; Schüler, Joshua; Casas, Pedro; Hoßfeld, Tobias in 17th International Conference on Network and Service Management (CNSM) (2021).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  Web browsing has become the most important application of the Internet for the end user. When it comes to mobile devices, web services are mainly accessed through apps. This paper tackles the problem of Web Quality of Experience (QoE) in mobile devices, with a specific focus on apps QoE monitoring and analysis, using in-network (encrypted) traffic measurements. Measuring apps QoE is complex, not only from an instrumentation point of view, but also from the heterogeneity of user interactions which might realize substantially different user experience. To this end, we conduct a feasibility study on four specific and popular Android apps and their corresponding web services. Our test automation framework emulates and measures different user interactions commonly executed during an app session, including the app startup, clicking, scrolling, and searching. The resulting traffic is characterized on different dimensions, and machine learning models are trained to identify web services, apps, and user interactions, and to infer their QoE. The proposed models can correctly identify the specific web service and app in 86% of the cases and accurately estimate the associated QoE with small errors. Our preliminary study represents a first step towards an in-network, web QoE monitoring solution for mobile-device apps.

  @inproceedings{info3-inproceedings-2021-15, abstract = {Web browsing has become the most important application of the Internet for the end user. When it comes to mobile devices, web services are mainly accessed through apps. This paper tackles the problem of Web Quality of Experience (QoE) in mobile devices, with a specific focus on apps QoE monitoring and analysis, using in-network (encrypted) traffic measurements. Measuring apps QoE is complex, not only from an instrumentation point of view, but also from the heterogeneity of user interactions which might realize substantially different user experience. To this end, we conduct a feasibility study on four specific and popular Android apps and their corresponding web services. Our test automation framework emulates and measures different user interactions commonly executed during an app session, including the app startup, clicking, scrolling, and searching. The resulting traffic is characterized on different dimensions, and machine learning models are trained to identify web services, apps, and user interactions, and to infer their QoE. The proposed models can correctly identify the specific web service and app in 86% of the cases and accurately estimate the associated QoE with small errors. Our preliminary study represents a first step towards an in-network, web QoE monitoring solution for mobile-device apps.}, address = {Izmir, Turkey (Virtual Conference)}, author = {Wehner, Nikolas and Seufert, Michael and Schüler, Joshua and Casas, Pedro and Hoßfeld, Tobias}, booktitle = {17th International Conference on Network and Service Management (CNSM)}, keywords = {ucn}, month = 10, title = {How are your Apps Doing? QoE Inference and Analysis in Mobile Devices}, year = 2021 }

  %0 Conference Paper %1 info3-inproceedings-2021-15 %A Wehner, Nikolas %A Seufert, Michael %A Schüler, Joshua %A Casas, Pedro %A Hoßfeld, Tobias %B 17th International Conference on Network and Service Management (CNSM) %C Izmir, Turkey (Virtual Conference) %D 2021 %T How are your Apps Doing? QoE Inference and Analysis in Mobile Devices %X Web browsing has become the most important application of the Internet for the end user. When it comes to mobile devices, web services are mainly accessed through apps. This paper tackles the problem of Web Quality of Experience (QoE) in mobile devices, with a specific focus on apps QoE monitoring and analysis, using in-network (encrypted) traffic measurements. Measuring apps QoE is complex, not only from an instrumentation point of view, but also from the heterogeneity of user interactions which might realize substantially different user experience. To this end, we conduct a feasibility study on four specific and popular Android apps and their corresponding web services. Our test automation framework emulates and measures different user interactions commonly executed during an app session, including the app startup, clicking, scrolling, and searching. The resulting traffic is characterized on different dimensions, and machine learning models are trained to identify web services, apps, and user interactions, and to infer their QoE. The proposed models can correctly identify the specific web service and app in 86% of the cases and accurately estimate the associated QoE with small errors. Our preliminary study represents a first step towards an in-network, web QoE monitoring solution for mobile-device apps.
• Quality that Matters: QoE Monitoring in Education Service Provider (ESP) Networks Wehner, Nikolas; Seufert, Michael; Wieser, Viktoria; Casas, Pedro; Capdehourat, Germán in IFIP/IEEE International Symposium on Integrated Network Management (IM 2021) (2021).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  Education Service Providers (ESPs) play a crucial role in the digitization of education as they equip students and teachers with reliable devices and provide high quality Internet access at schools. This paper investigates four months worth of continuous measurements conducted by an ESP using a largescale, passive, in-device Quality of Experience (QoE) monitoring system deployed into a nationwide network of education-purpose devices. These measurements cover more than 800 education centers and about 4000 devices, used both in schools and at home. Using this rich dataset, we present an exhaustive characterization of the browsing behavior, and a quantification of the web and video QoE in this educational context. Web QoE results showed a better performance for school Wi-Fi networks compared to home connections, suggesting that several issues may arise for ESPs due to the increasing relevance of home-schooling caused by the COVID-19 pandemic.

  @inproceedings{info3-inproceedings-2021-3, abstract = {Education Service Providers (ESPs) play a crucial role in the digitization of education as they equip students and teachers with reliable devices and provide high quality Internet access at schools. This paper investigates four months worth of continuous measurements conducted by an ESP using a largescale, passive, in-device Quality of Experience (QoE) monitoring system deployed into a nationwide network of education-purpose devices. These measurements cover more than 800 education centers and about 4000 devices, used both in schools and at home. Using this rich dataset, we present an exhaustive characterization of the browsing behavior, and a quantification of the web and video QoE in this educational context. Web QoE results showed a better performance for school Wi-Fi networks compared to home connections, suggesting that several issues may arise for ESPs due to the increasing relevance of home-schooling caused by the COVID-19 pandemic.}, address = {Bordeaux, France (Virtual Conference)}, author = {Wehner, Nikolas and Seufert, Michael and Wieser, Viktoria and Casas, Pedro and Capdehourat, Germán}, booktitle = {IFIP/IEEE International Symposium on Integrated Network Management (IM 2021)}, keywords = {ucn}, month = 5, title = {Quality that Matters: QoE Monitoring in Education Service Provider (ESP) Networks}, year = 2021 }

  %0 Conference Paper %1 info3-inproceedings-2021-3 %A Wehner, Nikolas %A Seufert, Michael %A Wieser, Viktoria %A Casas, Pedro %A Capdehourat, Germán %B IFIP/IEEE International Symposium on Integrated Network Management (IM 2021) %C Bordeaux, France (Virtual Conference) %D 2021 %T Quality that Matters: QoE Monitoring in Education Service Provider (ESP) Networks %X Education Service Providers (ESPs) play a crucial role in the digitization of education as they equip students and teachers with reliable devices and provide high quality Internet access at schools. This paper investigates four months worth of continuous measurements conducted by an ESP using a largescale, passive, in-device Quality of Experience (QoE) monitoring system deployed into a nationwide network of education-purpose devices. These measurements cover more than 800 education centers and about 4000 devices, used both in schools and at home. Using this rich dataset, we present an exhaustive characterization of the browsing behavior, and a quantification of the web and video QoE in this educational context. Web QoE results showed a better performance for school Wi-Fi networks compared to home connections, suggesting that several issues may arise for ESPs due to the increasing relevance of home-schooling caused by the COVID-19 pandemic.
• ML-Assisted Latency Assignments in Time-Sensitive Networking Grigorjew, Alexej; Seufert, Michael; Wehner, Nikolas; Hofmann, Jan; Hoßfeld, Tobias in IFIP/IEEE International Symposium on Integrated Network Management (IM 2021) (2021).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  Recent developments in industrial automation and invehicle communication have raised the requirements of real-time networking. Bus systems that were traditionally deployed in these fields cannot provide sufficient bandwidth and are now shifting towards Ethernet for their real-time communication needs. In this field, standardization efforts from the IEEE and the IETF have developed new data plane mechanisms such as shapers and schedulers, as well as control plane mechanisms such as reservation protocols to support their new requirements. However, their implementation and their optimal configuration remain an important factor for their efficiency. This work presents a machine learning framework that takes on the configuration task. Four different models are trained for the configuration of per-hop latency guarantees in a distributed resource reservation process and compared with respect to their real-time traffic capacity. The evaluation shows that all models provide good configurations for the provided scenarios, but more importantly, they represent a first step for a semi-automated configuration of parameters in Time-Sensitive Networking.

  @inproceedings{info3-inproceedings-2021-2, abstract = {Recent developments in industrial automation and invehicle communication have raised the requirements of real-time networking. Bus systems that were traditionally deployed in these fields cannot provide sufficient bandwidth and are now shifting towards Ethernet for their real-time communication needs. In this field, standardization efforts from the IEEE and the IETF have developed new data plane mechanisms such as shapers and schedulers, as well as control plane mechanisms such as reservation protocols to support their new requirements. However, their implementation and their optimal configuration remain an important factor for their efficiency. This work presents a machine learning framework that takes on the configuration task. Four different models are trained for the configuration of per-hop latency guarantees in a distributed resource reservation process and compared with respect to their real-time traffic capacity. The evaluation shows that all models provide good configurations for the provided scenarios, but more importantly, they represent a first step for a semi-automated configuration of parameters in Time-Sensitive Networking.}, address = {Bordeaux, France (Virtual Conference)}, author = {Grigorjew, Alexej and Seufert, Michael and Wehner, Nikolas and Hofmann, Jan and Hoßfeld, Tobias}, booktitle = {IFIP/IEEE International Symposium on Integrated Network Management (IM 2021)}, keywords = {kikdin}, month = 5, title = {ML-Assisted Latency Assignments in Time-Sensitive Networking}, year = 2021 }

  %0 Conference Paper %1 info3-inproceedings-2021-2 %A Grigorjew, Alexej %A Seufert, Michael %A Wehner, Nikolas %A Hofmann, Jan %A Hoßfeld, Tobias %B IFIP/IEEE International Symposium on Integrated Network Management (IM 2021) %C Bordeaux, France (Virtual Conference) %D 2021 %T ML-Assisted Latency Assignments in Time-Sensitive Networking %X Recent developments in industrial automation and invehicle communication have raised the requirements of real-time networking. Bus systems that were traditionally deployed in these fields cannot provide sufficient bandwidth and are now shifting towards Ethernet for their real-time communication needs. In this field, standardization efforts from the IEEE and the IETF have developed new data plane mechanisms such as shapers and schedulers, as well as control plane mechanisms such as reservation protocols to support their new requirements. However, their implementation and their optimal configuration remain an important factor for their efficiency. This work presents a machine learning framework that takes on the configuration task. Four different models are trained for the configuration of per-hop latency guarantees in a distributed resource reservation process and compared with respect to their real-time traffic capacity. The evaluation shows that all models provide good configurations for the provided scenarios, but more importantly, they represent a first step for a semi-automated configuration of parameters in Time-Sensitive Networking.

• Are you on Mobile or Desktop? On the Impact of End-User Device on Web QoE Inference from Encrypted Traffic Wassermann, Sarah; Casas, Pedro; Ben Houidi, Zied; Huet, Alexis; Seufert, Michael; Wehner, Nikolas; Schüler, Joshua; Cai, Sheng-Ming; Shi, Hao; Xu, Jinchun; Hoßfeld, Tobias; Rossi, Dario in 16th International Conference on Network and Service Management (CNSM) (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  Web browsing is one of the key applications of the Internet, if not the most important one. We address the problem of Web Quality of Experience (QoE) monitoring from the ISP perspective, relying on in-network, passive measurements. As a proxy to Web QoE, we focus on the analysis of the well-known SpeedIndex (SI) metric. Given the lack of app-level data visibility introduced by the wide adoption of end-to-end encryption, we resort to machine-learning models to infer the SI and the QoE level of individual web page loading sessions, using as input only packet- and flow-level data. Our study targets the impact of different end-user device types (e.g., smartphone, desktop, tablet) on the performance of such models. Empirical evaluations on a large, multi-device, heterogeneous corpus of Web QoE measurements for top popular websites demonstrate that the proposed solution can infer the SI as well as estimate QoE ranges with high accuracy, using either packet-level or flow-level measurements. In addition, we show that the device type adds a strong bias in the feasibility of these Web QoE models, putting into question the applicability of previously conceived approaches on single-device measurements. To improve the state of affairs, we conceive cross-device generalizable models operating at both packet and flow levels, offering a feasible solution for Web QoE monitoring in operational, multi-device networks. To the best of our knowledge, this is the first study tackling the analysis of Web QoE from encrypted network traffic in multi-device scenarios.

  @inproceedings{info3-inproceedings-2020-10, abstract = {Web browsing is one of the key applications of the Internet, if not the most important one. We address the problem of Web Quality of Experience (QoE) monitoring from the ISP perspective, relying on in-network, passive measurements. As a proxy to Web QoE, we focus on the analysis of the well-known SpeedIndex (SI) metric. Given the lack of app-level data visibility introduced by the wide adoption of end-to-end encryption, we resort to machine-learning models to infer the SI and the QoE level of individual web page loading sessions, using as input only packet- and flow-level data. Our study targets the impact of different end-user device types (e.g., smartphone, desktop, tablet) on the performance of such models. Empirical evaluations on a large, multi-device, heterogeneous corpus of Web QoE measurements for top popular websites demonstrate that the proposed solution can infer the SI as well as estimate QoE ranges with high accuracy, using either packet-level or flow-level measurements. In addition, we show that the device type adds a strong bias in the feasibility of these Web QoE models, putting into question the applicability of previously conceived approaches on single-device measurements. To improve the state of affairs, we conceive cross-device generalizable models operating at both packet and flow levels, offering a feasible solution for Web QoE monitoring in operational, multi-device networks. To the best of our knowledge, this is the first study tackling the analysis of Web QoE from encrypted network traffic in multi-device scenarios.}, address = {Izmir, Turkey}, author = {Wassermann, Sarah and Casas, Pedro and Ben Houidi, Zied and Huet, Alexis and Seufert, Michael and Wehner, Nikolas and Schüler, Joshua and Cai, Sheng-Ming and Shi, Hao and Xu, Jinchun and Hoßfeld, Tobias and Rossi, Dario}, booktitle = {16th International Conference on Network and Service Management (CNSM)}, keywords = {ucn}, month = 11, title = {Are you on Mobile or Desktop? On the Impact of End-User Device on Web QoE Inference from Encrypted Traffic}, year = 2020 }

  %0 Conference Paper %1 info3-inproceedings-2020-10 %A Wassermann, Sarah %A Casas, Pedro %A Ben Houidi, Zied %A Huet, Alexis %A Seufert, Michael %A Wehner, Nikolas %A Schüler, Joshua %A Cai, Sheng-Ming %A Shi, Hao %A Xu, Jinchun %A Hoßfeld, Tobias %A Rossi, Dario %B 16th International Conference on Network and Service Management (CNSM) %C Izmir, Turkey %D 2020 %T Are you on Mobile or Desktop? On the Impact of End-User Device on Web QoE Inference from Encrypted Traffic %X Web browsing is one of the key applications of the Internet, if not the most important one. We address the problem of Web Quality of Experience (QoE) monitoring from the ISP perspective, relying on in-network, passive measurements. As a proxy to Web QoE, we focus on the analysis of the well-known SpeedIndex (SI) metric. Given the lack of app-level data visibility introduced by the wide adoption of end-to-end encryption, we resort to machine-learning models to infer the SI and the QoE level of individual web page loading sessions, using as input only packet- and flow-level data. Our study targets the impact of different end-user device types (e.g., smartphone, desktop, tablet) on the performance of such models. Empirical evaluations on a large, multi-device, heterogeneous corpus of Web QoE measurements for top popular websites demonstrate that the proposed solution can infer the SI as well as estimate QoE ranges with high accuracy, using either packet-level or flow-level measurements. In addition, we show that the device type adds a strong bias in the feasibility of these Web QoE models, putting into question the applicability of previously conceived approaches on single-device measurements. To improve the state of affairs, we conceive cross-device generalizable models operating at both packet and flow levels, offering a feasible solution for Web QoE monitoring in operational, multi-device networks. To the best of our knowledge, this is the first study tackling the analysis of Web QoE from encrypted network traffic in multi-device scenarios.
• Mind the (QoE) Gap: On the Incompatibility of Web and Video QoE Models in the Wild Seufert, Michael; Wehner, Nikolas; Wieser, Viktoria; Casas, Pedro; Capdehourat, Germán in 16th International Conference on Network and Service Management (CNSM) (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  Education Service Providers (ESPs) have a paramount role in the digitization of education, providing reliable devices for students and teachers and high quality Internet access at schools. In this paper, a large-scale, passive, in-device Quality of Experience (QoE) monitoring system is presented, which was deployed into a nationwide network of education-purpose devices. Four months' worth of continuous measurements were conducted by an ESP, covering more than 800 education centers and about 4000 devices, used both in schools and at home. When analyzing the QoE of web sessions in school networks, we identify a fundamental issue with the compatibility of web browsing and video QoE models, which inhibits the successful application of QoE-aware network management for multiple services.

  @inproceedings{info3-inproceedings-2020-11, abstract = {Education Service Providers (ESPs) have a paramount role in the digitization of education, providing reliable devices for students and teachers and high quality Internet access at schools. In this paper, a large-scale, passive, in-device Quality of Experience (QoE) monitoring system is presented, which was deployed into a nationwide network of education-purpose devices. Four months' worth of continuous measurements were conducted by an ESP, covering more than 800 education centers and about 4000 devices, used both in schools and at home. When analyzing the QoE of web sessions in school networks, we identify a fundamental issue with the compatibility of web browsing and video QoE models, which inhibits the successful application of QoE-aware network management for multiple services.}, address = {Izmir, Turkey}, author = {Seufert, Michael and Wehner, Nikolas and Wieser, Viktoria and Casas, Pedro and Capdehourat, Germán}, booktitle = {16th International Conference on Network and Service Management (CNSM)}, keywords = {ucn}, month = 11, title = {Mind the (QoE) Gap: On the Incompatibility of Web and Video QoE Models in the Wild}, year = 2020 }

  %0 Conference Paper %1 info3-inproceedings-2020-11 %A Seufert, Michael %A Wehner, Nikolas %A Wieser, Viktoria %A Casas, Pedro %A Capdehourat, Germán %B 16th International Conference on Network and Service Management (CNSM) %C Izmir, Turkey %D 2020 %T Mind the (QoE) Gap: On the Incompatibility of Web and Video QoE Models in the Wild %X Education Service Providers (ESPs) have a paramount role in the digitization of education, providing reliable devices for students and teachers and high quality Internet access at schools. In this paper, a large-scale, passive, in-device Quality of Experience (QoE) monitoring system is presented, which was deployed into a nationwide network of education-purpose devices. Four months' worth of continuous measurements were conducted by an ESP, covering more than 800 education centers and about 4000 devices, used both in schools and at home. When analyzing the QoE of web sessions in school networks, we identify a fundamental issue with the compatibility of web browsing and video QoE models, which inhibits the successful application of QoE-aware network management for multiple services.
• Improving Web QoE Monitoring for Encrypted Network Traffic through Time Series Modeling Wehner, Nikolas; Seufert, Michael; Schüler, Joshua; Wassermann, Sarah; Casas, Pedro; Hoßfeld, Tobias in 2nd Workshop on AI in Networks and Distributed Systems (WAIN) (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  This paper addresses the problem of Quality of Experience (QoE) monitoring for web browsing. In particular, the inference of common Web QoE metrics such as Speed Index (SI) is investigated. Based on a large dataset collected with open web-measurement platforms on different device-types, a unique feature set is designed and used to estimate the RUMSI -- an efficient approximation to SI, with machine-learning based regression and classification approaches. Results indicate that it is possible to estimate the RUMSI accurately, and that in particular, recurrent neural networks are highly suitable for the task, as they capture the network dynamics more precisely.

  @inproceedings{info3-inproceedings-2020-12, abstract = {This paper addresses the problem of Quality of Experience (QoE) monitoring for web browsing. In particular, the inference of common Web QoE metrics such as Speed Index (SI) is investigated. Based on a large dataset collected with open web-measurement platforms on different device-types, a unique feature set is designed and used to estimate the RUMSI – an efficient approximation to SI, with machine-learning based regression and classification approaches. Results indicate that it is possible to estimate the RUMSI accurately, and that in particular, recurrent neural networks are highly suitable for the task, as they capture the network dynamics more precisely.}, address = {Milan, Italy}, author = {Wehner, Nikolas and Seufert, Michael and Schüler, Joshua and Wassermann, Sarah and Casas, Pedro and Hoßfeld, Tobias}, booktitle = {2nd Workshop on AI in Networks and Distributed Systems (WAIN)}, keywords = {ucn}, month = 11, title = {Improving Web QoE Monitoring for Encrypted Network Traffic through Time Series Modeling}, year = 2020 }

  %0 Conference Paper %1 info3-inproceedings-2020-12 %A Wehner, Nikolas %A Seufert, Michael %A Schüler, Joshua %A Wassermann, Sarah %A Casas, Pedro %A Hoßfeld, Tobias %B 2nd Workshop on AI in Networks and Distributed Systems (WAIN) %C Milan, Italy %D 2020 %T Improving Web QoE Monitoring for Encrypted Network Traffic through Time Series Modeling %X This paper addresses the problem of Quality of Experience (QoE) monitoring for web browsing. In particular, the inference of common Web QoE metrics such as Speed Index (SI) is investigated. Based on a large dataset collected with open web-measurement platforms on different device-types, a unique feature set is designed and used to estimate the RUMSI -- an efficient approximation to SI, with machine-learning based regression and classification approaches. Results indicate that it is possible to estimate the RUMSI accurately, and that in particular, recurrent neural networks are highly suitable for the task, as they capture the network dynamics more precisely.
• Scoring High: Analysis and Prediction of Viewer Behavior and Engagement in the Context of 2018 FIFA WC Live Streaming Wehner, Nikolas; Seufert, Michael; Egger-Lampl, Sebastian; Gardlo, Bruno; Casas, Pedro; Schatz, Raimund in Proceedings of the 28th ACM International Conference on Multimedia (MM) (2020).
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{info3-inproceedings-2020-9, address = {Seattle, WA, USA}, author = {Wehner, Nikolas and Seufert, Michael and Egger-Lampl, Sebastian and Gardlo, Bruno and Casas, Pedro and Schatz, Raimund}, booktitle = {Proceedings of the 28th ACM International Conference on Multimedia (MM)}, keywords = {ucn}, month = 10, title = {Scoring High: Analysis and Prediction of Viewer Behavior and Engagement in the Context of 2018 FIFA WC Live Streaming}, year = 2020 }

  %0 Conference Paper %1 info3-inproceedings-2020-9 %A Wehner, Nikolas %A Seufert, Michael %A Egger-Lampl, Sebastian %A Gardlo, Bruno %A Casas, Pedro %A Schatz, Raimund %B Proceedings of the 28th ACM International Conference on Multimedia (MM) %C Seattle, WA, USA %D 2020 %T Scoring High: Analysis and Prediction of Viewer Behavior and Engagement in the Context of 2018 FIFA WC Live Streaming
• Studying the Impact of the Content Selection Method on the Video QoE on Mobile Devices Wehner, Nikolas; Mertinat, Nils; Seufert, Michael; Hoßfeld, Tobias in 12th International Conference on Quality of Multimedia Experience (QoMEX) (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  When conducting video QoE studies, participants are usually asked to rate the QoE of prepared test videos. However, participants are given no choice to select content, which they like or in which they are interested. This may cause annoyance or frustration when conducting the QoE study, which eventually might affect the QoE results of the study. The consequent question is whether the content liking has a direct impact on the submitted ratings by the participants and whether the freedom of choosing the video content in QoE studies results in better ratings. To investigate this research question, MCA, an existing framework for crowdsourced video testing, is extended and used in a pilot field study. MCA runs on mobile devices and allows users to watch the tested conditions of a QoE study within video content of their interest. The results of a QoE study with individual and dynamic content selection are compared to a QoE study with pre-selected contents. Moreover, this work includes a comparison to a previous QoE study for validation. As the previous study was conducted on desktop PCs, the MCA study further allows to identify differences in the stalling perception between studies on desktop PCs and mobile devices.

  @inproceedings{info3-inproceedings-2020-8, abstract = {When conducting video QoE studies, participants are usually asked to rate the QoE of prepared test videos. However, participants are given no choice to select content, which they like or in which they are interested. This may cause annoyance or frustration when conducting the QoE study, which eventually might affect the QoE results of the study. The consequent question is whether the content liking has a direct impact on the submitted ratings by the participants and whether the freedom of choosing the video content in QoE studies results in better ratings. To investigate this research question, MCA, an existing framework for crowdsourced video testing, is extended and used in a pilot field study. MCA runs on mobile devices and allows users to watch the tested conditions of a QoE study within video content of their interest. The results of a QoE study with individual and dynamic content selection are compared to a QoE study with pre-selected contents. Moreover, this work includes a comparison to a previous QoE study for validation. As the previous study was conducted on desktop PCs, the MCA study further allows to identify differences in the stalling perception between studies on desktop PCs and mobile devices.}, address = {Athlone, Ireland}, author = {Wehner, Nikolas and Mertinat, Nils and Seufert, Michael and Hoßfeld, Tobias}, booktitle = {12th International Conference on Quality of Multimedia Experience (QoMEX)}, keywords = {ucn}, month = 5, title = {Studying the Impact of the Content Selection Method on the Video QoE on Mobile Devices}, year = 2020 }

  %0 Conference Paper %1 info3-inproceedings-2020-8 %A Wehner, Nikolas %A Mertinat, Nils %A Seufert, Michael %A Hoßfeld, Tobias %B 12th International Conference on Quality of Multimedia Experience (QoMEX) %C Athlone, Ireland %D 2020 %T Studying the Impact of the Content Selection Method on the Video QoE on Mobile Devices %X When conducting video QoE studies, participants are usually asked to rate the QoE of prepared test videos. However, participants are given no choice to select content, which they like or in which they are interested. This may cause annoyance or frustration when conducting the QoE study, which eventually might affect the QoE results of the study. The consequent question is whether the content liking has a direct impact on the submitted ratings by the participants and whether the freedom of choosing the video content in QoE studies results in better ratings. To investigate this research question, MCA, an existing framework for crowdsourced video testing, is extended and used in a pilot field study. MCA runs on mobile devices and allows users to watch the tested conditions of a QoE study within video content of their interest. The results of a QoE study with individual and dynamic content selection are compared to a QoE study with pre-selected contents. Moreover, this work includes a comparison to a previous QoE study for validation. As the previous study was conducted on desktop PCs, the MCA study further allows to identify differences in the stalling perception between studies on desktop PCs and mobile devices.

• A Fair Share for All: TCP-inspired Adaptation Logic for QoE Fairness among Heterogeneous HTTP Adaptive Video Streaming Clients Seufert, Michael; Wehner, Nikolas; Casas, Pedro in IEEE Transactions on Network and Service Management (2019). 16(2) 475–488.
  • [ BibTeX ]
  • [ EndNote ]
  @article{info3-article-2019-4, author = {Seufert, Michael and Wehner, Nikolas and Casas, Pedro}, journal = {IEEE Transactions on Network and Service Management}, keywords = {sys:hidden:article}, month = 6, number = 2, pages = {475-488}, title = {A Fair Share for All: TCP-inspired Adaptation Logic for QoE Fairness among Heterogeneous HTTP Adaptive Video Streaming Clients}, volume = 16, year = 2019 }

  %0 Journal Article %1 info3-article-2019-4 %A Seufert, Michael %A Wehner, Nikolas %A Casas, Pedro %D 2019 %J IEEE Transactions on Network and Service Management %N 2 %P 475-488 %T A Fair Share for All: TCP-inspired Adaptation Logic for QoE Fairness among Heterogeneous HTTP Adaptive Video Streaming Clients %V 16
• Is QUIC becoming the New TCP? On the Potential Impact of a New Protocol on Networked Multimedia QoE Seufert, Michael; Schatz, Raimund; Wehner, Nikolas; Gardlo, Bruno; Casas, Pedro in 11th International Conference on Quality of Multimedia Experience (QoMEX) (2019).
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{info3-inproceedings-2019-13, address = {Berlin, Germany}, author = {Seufert, Michael and Schatz, Raimund and Wehner, Nikolas and Gardlo, Bruno and Casas, Pedro}, booktitle = {11th International Conference on Quality of Multimedia Experience (QoMEX)}, keywords = {ua-net}, month = 6, title = {Is QUIC becoming the New TCP? On the Potential Impact of a New Protocol on Networked Multimedia QoE}, year = 2019 }

  %0 Conference Paper %1 info3-inproceedings-2019-13 %A Seufert, Michael %A Schatz, Raimund %A Wehner, Nikolas %A Gardlo, Bruno %A Casas, Pedro %B 11th International Conference on Quality of Multimedia Experience (QoMEX) %C Berlin, Germany %D 2019 %T Is QUIC becoming the New TCP? On the Potential Impact of a New Protocol on Networked Multimedia QoE
• Features that Matter: Feature Selection for On-line Stalling Prediction in Encrypted Video Streaming Seufert, Michael; Casas, Pedro; Wehner, Nikolas; Li, Gang; Kuang, Li in 2nd International Workshop on Network Intelligence (NI) (2019).
  • [ BibTeX ]
  • [ EndNote ]
  @inproceedings{info3-inproceedings-2019-11, address = {Paris, France}, author = {Seufert, Michael and Casas, Pedro and Wehner, Nikolas and Li, Gang and Kuang, Li}, booktitle = {2nd International Workshop on Network Intelligence (NI)}, keywords = {net-man}, month = 4, title = {Features that Matter: Feature Selection for On-line Stalling Prediction in Encrypted Video Streaming}, year = 2019 }

  %0 Conference Paper %1 info3-inproceedings-2019-11 %A Seufert, Michael %A Casas, Pedro %A Wehner, Nikolas %A Li, Gang %A Kuang, Li %B 2nd International Workshop on Network Intelligence (NI) %C Paris, France %D 2019 %T Features that Matter: Feature Selection for On-line Stalling Prediction in Encrypted Video Streaming
• Stream-based Machine Learning for Real-time QoE Analysis of Encrypted Video Streaming Traffic Seufert, Michael; Casas, Pedro; Wehner, Nikolas; Li, Gang; Kuang, Li in 3rd International Workshop on Quality of Experience Management (QoE-Management) (2019).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  As stalling is the worst Quality of Experience (QoE) degradation of HTTP adaptive video streaming (HAS), this work presents a stream-based machine learning approach, ViCrypt, which analyzes stalling of YouTube streaming sessions in real-time from encrypted network traffic. The video streaming session is subdivided into a stream of short time slots of 1 s length, while considering two additional macro windows each for the current streaming trend and the whole ongoing streaming session. Constant memory features are extracted from the encrypted network traffic in these three windows in a stream-based fashion, and fed into a random forest model, which predicts whether the current time slot contains stalling or not. The presented system can predict stalling with a very high accuracy and the finest granularity to date (1 s), and thus, can be used in networks for real-time QoE analysis from encrypted YouTube video streaming traffic. The independent predictions for each consecutive slot of a streaming session can further be aggregated to obtain stalling estimations for the whole session. Thereby, the proposed method allows to quantify the initial delay, as well as the overall number of stalling events and the stalling ratio, i.e., the ratio of total stalling time and total playback time.

  @inproceedings{info3-inproceedings-2019-8, abstract = {As stalling is the worst Quality of Experience (QoE) degradation of HTTP adaptive video streaming (HAS), this work presents a stream-based machine learning approach, ViCrypt, which analyzes stalling of YouTube streaming sessions in real-time from encrypted network traffic. The video streaming session is subdivided into a stream of short time slots of 1 s length, while considering two additional macro windows each for the current streaming trend and the whole ongoing streaming session. Constant memory features are extracted from the encrypted network traffic in these three windows in a stream-based fashion, and fed into a random forest model, which predicts whether the current time slot contains stalling or not. The presented system can predict stalling with a very high accuracy and the finest granularity to date (1 s), and thus, can be used in networks for real-time QoE analysis from encrypted YouTube video streaming traffic. The independent predictions for each consecutive slot of a streaming session can further be aggregated to obtain stalling estimations for the whole session. Thereby, the proposed method allows to quantify the initial delay, as well as the overall number of stalling events and the stalling ratio, i.e., the ratio of total stalling time and total playback time.}, address = {Paris, France}, author = {Seufert, Michael and Casas, Pedro and Wehner, Nikolas and Li, Gang and Kuang, Li}, booktitle = {3rd International Workshop on Quality of Experience Management (QoE-Management)}, keywords = {net-man}, month = 2, title = {Stream-based Machine Learning for Real-time QoE Analysis of Encrypted Video Streaming Traffic}, year = 2019 }

  %0 Conference Paper %1 info3-inproceedings-2019-8 %A Seufert, Michael %A Casas, Pedro %A Wehner, Nikolas %A Li, Gang %A Kuang, Li %B 3rd International Workshop on Quality of Experience Management (QoE-Management) %C Paris, France %D 2019 %T Stream-based Machine Learning for Real-time QoE Analysis of Encrypted Video Streaming Traffic %X As stalling is the worst Quality of Experience (QoE) degradation of HTTP adaptive video streaming (HAS), this work presents a stream-based machine learning approach, ViCrypt, which analyzes stalling of YouTube streaming sessions in real-time from encrypted network traffic. The video streaming session is subdivided into a stream of short time slots of 1 s length, while considering two additional macro windows each for the current streaming trend and the whole ongoing streaming session. Constant memory features are extracted from the encrypted network traffic in these three windows in a stream-based fashion, and fed into a random forest model, which predicts whether the current time slot contains stalling or not. The presented system can predict stalling with a very high accuracy and the finest granularity to date (1 s), and thus, can be used in networks for real-time QoE analysis from encrypted YouTube video streaming traffic. The independent predictions for each consecutive slot of a streaming session can further be aggregated to obtain stalling estimations for the whole session. Thereby, the proposed method allows to quantify the initial delay, as well as the overall number of stalling events and the stalling ratio, i.e., the ratio of total stalling time and total playback time.
• QUICker or not? - an Empirical Analysis of QUIC vs TCP for Video Streaming QoE Provisioning Seufert, Michael; Schatz, Raimund; Wehner, Nikolas; Casas, Pedro in 3rd International Workshop on Quality of Experience Management (QoE-Management) (2019).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  The introduction of the QUIC (Quick UDP Internet Connections) transport protocol by Google aimed to improve the Quality of Experience (QoE) with web services compared to the prevailing Transport Control Protocol (TCP). Nowadays, QUIC has become the default protocol to communicate between the Google Chrome browser and Google servers and accounts for an increasing share of the Internet traffic. This work investigates whether the promised QoE benefits of QUIC are indeed noticeable for end users or not. A measurement study was conducted for YouTube video streaming in two mobile and two fixed access networks, in which a defined set of videos was streamed back-to-back with QUIC and TCP in randomized order. QoE factors of video streaming (such as initial delay, the visual quality of the video, and stalling) were compared statistically to find significant differences between the streaming over QUIC and the streaming over TCP. Surprisingly, no evidence for any QoE improvement of QUIC over TCP in the context of YouTube streaming could be found.

  @inproceedings{info3-inproceedings-2019-7, abstract = {The introduction of the QUIC (Quick UDP Internet Connections) transport protocol by Google aimed to improve the Quality of Experience (QoE) with web services compared to the prevailing Transport Control Protocol (TCP). Nowadays, QUIC has become the default protocol to communicate between the Google Chrome browser and Google servers and accounts for an increasing share of the Internet traffic. This work investigates whether the promised QoE benefits of QUIC are indeed noticeable for end users or not. A measurement study was conducted for YouTube video streaming in two mobile and two fixed access networks, in which a defined set of videos was streamed back-to-back with QUIC and TCP in randomized order. QoE factors of video streaming (such as initial delay, the visual quality of the video, and stalling) were compared statistically to find significant differences between the streaming over QUIC and the streaming over TCP. Surprisingly, no evidence for any QoE improvement of QUIC over TCP in the context of YouTube streaming could be found.}, address = {Paris, France}, author = {Seufert, Michael and Schatz, Raimund and Wehner, Nikolas and Casas, Pedro}, booktitle = {3rd International Workshop on Quality of Experience Management (QoE-Management)}, keywords = {ua-net}, month = 2, title = {QUICker or not? - an Empirical Analysis of QUIC vs TCP for Video Streaming QoE Provisioning}, year = 2019 }

  %0 Conference Paper %1 info3-inproceedings-2019-7 %A Seufert, Michael %A Schatz, Raimund %A Wehner, Nikolas %A Casas, Pedro %B 3rd International Workshop on Quality of Experience Management (QoE-Management) %C Paris, France %D 2019 %T QUICker or not? - an Empirical Analysis of QUIC vs TCP for Video Streaming QoE Provisioning %X The introduction of the QUIC (Quick UDP Internet Connections) transport protocol by Google aimed to improve the Quality of Experience (QoE) with web services compared to the prevailing Transport Control Protocol (TCP). Nowadays, QUIC has become the default protocol to communicate between the Google Chrome browser and Google servers and accounts for an increasing share of the Internet traffic. This work investigates whether the promised QoE benefits of QUIC are indeed noticeable for end users or not. A measurement study was conducted for YouTube video streaming in two mobile and two fixed access networks, in which a defined set of videos was streamed back-to-back with QUIC and TCP in randomized order. QoE factors of video streaming (such as initial delay, the visual quality of the video, and stalling) were compared statistically to find significant differences between the streaming over QUIC and the streaming over TCP. Surprisingly, no evidence for any QoE improvement of QUIC over TCP in the context of YouTube streaming could be found.

• A Fair Share for All: Novel Adaptation Logic for QoE Fairness of HTTP Adaptive Video Streaming Seufert, Michael; Wehner, Nikolas; Casas, Pedro; Wamser, Florian in 14th International Conference on Network and Service Management (CNSM), Best Paper Award (2018).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  This paper presents a novel adaptation logic for HTTP adaptive streaming (HAS), which achieves not only a high Quality of Experience (QoE) but also high QoE fairness among independent and heterogeneous clients. The algorithm forces video clients to adapt the requested quality level based on the current network conditions and their individual bit rate requirements, such that the overall quality levels selected by all currently active streaming clients are fairly distributed, i.e., they do not diverge too much. The design of the algorithm is inspired by the well-known Transmission Control Protocol (TCP) congestion control, and drives heterogeneous clients to independently converge on similar quality levels without the need for communicating with each other and/or with a centralized controller in the network. By defining quality levels with equal visual quality, and preparing video representations accordingly, the quality level fairness is extended to QoE fairness. In this work, the design of the algorithm is described and a simulative performance evaluation is conducted to compare the QoE and QoE fairness of the proposed algorithm with other HAS adaptation logics.

  @inproceedings{info3-inproceedings-2018-38, abstract = {This paper presents a novel adaptation logic for HTTP adaptive streaming (HAS), which achieves not only a high Quality of Experience (QoE) but also high QoE fairness among independent and heterogeneous clients. The algorithm forces video clients to adapt the requested quality level based on the current network conditions and their individual bit rate requirements, such that the overall quality levels selected by all currently active streaming clients are fairly distributed, i.e., they do not diverge too much. The design of the algorithm is inspired by the well-known Transmission Control Protocol (TCP) congestion control, and drives heterogeneous clients to independently converge on similar quality levels without the need for communicating with each other and/or with a centralized controller in the network. By defining quality levels with equal visual quality, and preparing video representations accordingly, the quality level fairness is extended to QoE fairness. In this work, the design of the algorithm is described and a simulative performance evaluation is conducted to compare the QoE and QoE fairness of the proposed algorithm with other HAS adaptation logics.}, address = {Rome, Italy}, author = {Seufert, Michael and Wehner, Nikolas and Casas, Pedro and Wamser, Florian}, booktitle = {14th International Conference on Network and Service Management (CNSM), Best Paper Award}, keywords = {can}, month = 11, title = {A Fair Share for All: Novel Adaptation Logic for QoE Fairness of HTTP Adaptive Video Streaming}, year = 2018 }

  %0 Conference Paper %1 info3-inproceedings-2018-38 %A Seufert, Michael %A Wehner, Nikolas %A Casas, Pedro %A Wamser, Florian %B 14th International Conference on Network and Service Management (CNSM), Best Paper Award %C Rome, Italy %D 2018 %T A Fair Share for All: Novel Adaptation Logic for QoE Fairness of HTTP Adaptive Video Streaming %X This paper presents a novel adaptation logic for HTTP adaptive streaming (HAS), which achieves not only a high Quality of Experience (QoE) but also high QoE fairness among independent and heterogeneous clients. The algorithm forces video clients to adapt the requested quality level based on the current network conditions and their individual bit rate requirements, such that the overall quality levels selected by all currently active streaming clients are fairly distributed, i.e., they do not diverge too much. The design of the algorithm is inspired by the well-known Transmission Control Protocol (TCP) congestion control, and drives heterogeneous clients to independently converge on similar quality levels without the need for communicating with each other and/or with a centralized controller in the network. By defining quality levels with equal visual quality, and preparing video representations accordingly, the quality level fairness is extended to QoE fairness. In this work, the design of the algorithm is described and a simulative performance evaluation is conducted to compare the QoE and QoE fairness of the proposed algorithm with other HAS adaptation logics.
• Beauty is in the Eye of the Smartphone Holder - A Data Driven Analysis of YouTube Mobile QoE Wehner, Nikolas; Wassermann, Sarah; Casas, Pedro; Seufert, Michael; Wamser, Florian in 14th International Conference on Network and Service Management (CNSM) (2018).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  Measuring the Quality of Experience (QoE) undergone by cellular network users has become paramount for cellular ISPs. Given its overwhelming dominance and ever-growing popularity, this paper focuses on the analysis of QoE for YouTube in mobile networks. Using a large-scale dataset of crowdsourced YouTube QoE measurements collected in smartphones with YoMoApp, we analyze the evolution of multiple relevant QoE-related metrics over time for YouTube mobile users. The dataset includes measurements from more than 360 users worldwide, spanning over the last five years. Our data-driven analysis shows a systematic performance and QoE improvement of YouTube in mobile devices over time, accompanied by an improvement of cellular network performance and by an optimization of the YouTube streaming behavior for smartphones.

  @inproceedings{info3-inproceedings-2018-39, abstract = {Measuring the Quality of Experience (QoE) undergone by cellular network users has become paramount for cellular ISPs. Given its overwhelming dominance and ever-growing popularity, this paper focuses on the analysis of QoE for YouTube in mobile networks. Using a large-scale dataset of crowdsourced YouTube QoE measurements collected in smartphones with YoMoApp, we analyze the evolution of multiple relevant QoE-related metrics over time for YouTube mobile users. The dataset includes measurements from more than 360 users worldwide, spanning over the last five years. Our data-driven analysis shows a systematic performance and QoE improvement of YouTube in mobile devices over time, accompanied by an improvement of cellular network performance and by an optimization of the YouTube streaming behavior for smartphones.}, address = {Rome, Italy}, author = {Wehner, Nikolas and Wassermann, Sarah and Casas, Pedro and Seufert, Michael and Wamser, Florian}, booktitle = {14th International Conference on Network and Service Management (CNSM)}, keywords = {can}, month = 11, title = {Beauty is in the Eye of the Smartphone Holder - A Data Driven Analysis of YouTube Mobile QoE}, year = 2018 }

  %0 Conference Paper %1 info3-inproceedings-2018-39 %A Wehner, Nikolas %A Wassermann, Sarah %A Casas, Pedro %A Seufert, Michael %A Wamser, Florian %B 14th International Conference on Network and Service Management (CNSM) %C Rome, Italy %D 2018 %T Beauty is in the Eye of the Smartphone Holder - A Data Driven Analysis of YouTube Mobile QoE %X Measuring the Quality of Experience (QoE) undergone by cellular network users has become paramount for cellular ISPs. Given its overwhelming dominance and ever-growing popularity, this paper focuses on the analysis of QoE for YouTube in mobile networks. Using a large-scale dataset of crowdsourced YouTube QoE measurements collected in smartphones with YoMoApp, we analyze the evolution of multiple relevant QoE-related metrics over time for YouTube mobile users. The dataset includes measurements from more than 360 users worldwide, spanning over the last five years. Our data-driven analysis shows a systematic performance and QoE improvement of YouTube in mobile devices over time, accompanied by an improvement of cellular network performance and by an optimization of the YouTube streaming behavior for smartphones.
• Studying the Impact of HAS QoE Factors on the Standardized QoE Model P.1203 Seufert, Michael; Wehner, Nikolas; Casas, Pedro in 3rd Workshop on QoE-based Analysis and Management of Data Communication Networks (Internet-QoE) (2018).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  P.1203 is a recent standardized model for assessing the Quality of Experience (QoE) of HTTP Adaptive Video Streaming (HAS). However, its complex definition does not allow for a straightforward identification of the underlying assumptions. To overcome this issue, this work investigates the impact of the well-known QoE factors of HAS, namely, initial delay, stalling, and adaptation, on the output QoE score of the model. Therefore, parameter studies are conducted using a reference implementation of P.1203, and the model response to variations of the input QoE factors are compared to results of previous QoE studies in order to get a deeper understanding of the standardized model and its inherent weighting of the QoE factors of HAS.

  @inproceedings{info3-inproceedings-2018-44, abstract = {P.1203 is a recent standardized model for assessing the Quality of Experience (QoE) of HTTP Adaptive Video Streaming (HAS). However, its complex definition does not allow for a straightforward identification of the underlying assumptions. To overcome this issue, this work investigates the impact of the well-known QoE factors of HAS, namely, initial delay, stalling, and adaptation, on the output QoE score of the model. Therefore, parameter studies are conducted using a reference implementation of P.1203, and the model response to variations of the input QoE factors are compared to results of previous QoE studies in order to get a deeper understanding of the standardized model and its inherent weighting of the QoE factors of HAS.}, address = {Vienna, Austria}, author = {Seufert, Michael and Wehner, Nikolas and Casas, Pedro}, booktitle = {3rd Workshop on QoE-based Analysis and Management of Data Communication Networks (Internet-QoE)}, keywords = {sys:hidden:inproceedings}, month = 7, title = {Studying the Impact of HAS QoE Factors on the Standardized QoE Model P.1203}, year = 2018 }

  %0 Conference Paper %1 info3-inproceedings-2018-44 %A Seufert, Michael %A Wehner, Nikolas %A Casas, Pedro %B 3rd Workshop on QoE-based Analysis and Management of Data Communication Networks (Internet-QoE) %C Vienna, Austria %D 2018 %T Studying the Impact of HAS QoE Factors on the Standardized QoE Model P.1203 %X P.1203 is a recent standardized model for assessing the Quality of Experience (QoE) of HTTP Adaptive Video Streaming (HAS). However, its complex definition does not allow for a straightforward identification of the underlying assumptions. To overcome this issue, this work investigates the impact of the well-known QoE factors of HAS, namely, initial delay, stalling, and adaptation, on the output QoE score of the model. Therefore, parameter studies are conducted using a reference implementation of P.1203, and the model response to variations of the input QoE factors are compared to results of previous QoE studies in order to get a deeper understanding of the standardized model and its inherent weighting of the QoE factors of HAS.
• Enhancing Machine Learning based QoE Prediction by Ensemble Models Casas, Pedro; Seufert, Michael; Wehner, Nikolas; Schwind, Anika; Wamser, Florian in 3rd Workshop on QoE-based Analysis and Management of Data Communication Networks (Internet-QoE) (2018).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  The number of smartphones connected to wireless networks and the volume of wireless network traffic generated by such devices have dramatically increased in the last few years, making it more challenging to tackle wireless network monitoring applications. The high-dimensionality of network data provided by current smartphone devices opens the door to the massive application of machine learning approaches to improve different wireless networking applications. In this paper we study the specific problem of Quality of Experience (QoE) prediction for popular smartphone apps, using machine learning models and in-smartphone measurements. We evaluate and compare different models for the analysis of smartphone generated data, including single models as well as machine learning ensembles such as bagging, boosting and stacking. Results suggest that, while decision-tree based models are the most accurate single models to predict QoE, ensemble learning models, and in particular stacking ones, are capable to significantly increase accuracy prediction and overall classification performance.

  @inproceedings{info3-inproceedings-2018-29, abstract = {The number of smartphones connected to wireless networks and the volume of wireless network traffic generated by such devices have dramatically increased in the last few years, making it more challenging to tackle wireless network monitoring applications. The high-dimensionality of network data provided by current smartphone devices opens the door to the massive application of machine learning approaches to improve different wireless networking applications. In this paper we study the specific problem of Quality of Experience (QoE) prediction for popular smartphone apps, using machine learning models and in-smartphone measurements. We evaluate and compare different models for the analysis of smartphone generated data, including single models as well as machine learning ensembles such as bagging, boosting and stacking. Results suggest that, while decision-tree based models are the most accurate single models to predict QoE, ensemble learning models, and in particular stacking ones, are capable to significantly increase accuracy prediction and overall classification performance.}, address = {Vienna, Austria}, author = {Casas, Pedro and Seufert, Michael and Wehner, Nikolas and Schwind, Anika and Wamser, Florian}, booktitle = {3rd Workshop on QoE-based Analysis and Management of Data Communication Networks (Internet-QoE)}, keywords = {can}, month = 7, title = {Enhancing Machine Learning based QoE Prediction by Ensemble Models}, year = 2018 }

  %0 Conference Paper %1 info3-inproceedings-2018-29 %A Casas, Pedro %A Seufert, Michael %A Wehner, Nikolas %A Schwind, Anika %A Wamser, Florian %B 3rd Workshop on QoE-based Analysis and Management of Data Communication Networks (Internet-QoE) %C Vienna, Austria %D 2018 %T Enhancing Machine Learning based QoE Prediction by Ensemble Models %X The number of smartphones connected to wireless networks and the volume of wireless network traffic generated by such devices have dramatically increased in the last few years, making it more challenging to tackle wireless network monitoring applications. The high-dimensionality of network data provided by current smartphone devices opens the door to the massive application of machine learning approaches to improve different wireless networking applications. In this paper we study the specific problem of Quality of Experience (QoE) prediction for popular smartphone apps, using machine learning models and in-smartphone measurements. We evaluate and compare different models for the analysis of smartphone generated data, including single models as well as machine learning ensembles such as bagging, boosting and stacking. Results suggest that, while decision-tree based models are the most accurate single models to predict QoE, ensemble learning models, and in particular stacking ones, are capable to significantly increase accuracy prediction and overall classification performance.

• Unsupervised QoE Field Study for Mobile YouTube Video Streaming with YoMoApp Seufert, Michael; Wehner, Nikolas; Wamser, Florian; Casas, Pedro; D’Alconzo, Alessandro; Tran-Gia, Phuoc in 9th International Conference on Quality of Multimedia Experience (QoMEX) (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  YoMoApp (YouTube Monitoring App) is an Android app to monitor mobile YouTube video streaming on both application- and network-layer. Additionally, it allows to collect subjective Quality of Experience (QoE) feedback of end users. During the development of the app, the stable versions of YoMoApp were already available in the Google Play Store, and the app was downloaded, installed, and used on many devices to monitor streaming sessions. As the app was not advertised in special campaigns or used for dedicated QoE studies, the monitored streaming sessions of this period compose the data set of a large unsupervised field study. The collected data set is evaluated to characterize current mobile YouTube streaming on both application and network layers. Furthermore, the problems and methodology to obtain QoE results from such unsupervised field study are discussed together with the actual QoE results. Correlations between QoE factors are investigated, and the QoE of clusters of similar streaming sessions is analyzed.

  @inproceedings{info3-inproceedings-2017-9, abstract = {YoMoApp (YouTube Monitoring App) is an Android app to monitor mobile YouTube video streaming on both application- and network-layer. Additionally, it allows to collect subjective Quality of Experience (QoE) feedback of end users. During the development of the app, the stable versions of YoMoApp were already available in the Google Play Store, and the app was downloaded, installed, and used on many devices to monitor streaming sessions. As the app was not advertised in special campaigns or used for dedicated QoE studies, the monitored streaming sessions of this period compose the data set of a large unsupervised field study. The collected data set is evaluated to characterize current mobile YouTube streaming on both application and network layers. Furthermore, the problems and methodology to obtain QoE results from such unsupervised field study are discussed together with the actual QoE results. Correlations between QoE factors are investigated, and the QoE of clusters of similar streaming sessions is analyzed.}, address = {Erfurt, Germany}, author = {Seufert, Michael and Wehner, Nikolas and Wamser, Florian and Casas, Pedro and D'Alconzo, Alessandro and Tran-Gia, Phuoc}, booktitle = {9th International Conference on Quality of Multimedia Experience (QoMEX)}, keywords = {can}, month = 5, title = {Unsupervised QoE Field Study for Mobile YouTube Video Streaming with YoMoApp}, year = 2017 }

  %0 Conference Paper %1 info3-inproceedings-2017-9 %A Seufert, Michael %A Wehner, Nikolas %A Wamser, Florian %A Casas, Pedro %A D'Alconzo, Alessandro %A Tran-Gia, Phuoc %B 9th International Conference on Quality of Multimedia Experience (QoMEX) %C Erfurt, Germany %D 2017 %T Unsupervised QoE Field Study for Mobile YouTube Video Streaming with YoMoApp %X YoMoApp (YouTube Monitoring App) is an Android app to monitor mobile YouTube video streaming on both application- and network-layer. Additionally, it allows to collect subjective Quality of Experience (QoE) feedback of end users. During the development of the app, the stable versions of YoMoApp were already available in the Google Play Store, and the app was downloaded, installed, and used on many devices to monitor streaming sessions. As the app was not advertised in special campaigns or used for dedicated QoE studies, the monitored streaming sessions of this period compose the data set of a large unsupervised field study. The collected data set is evaluated to characterize current mobile YouTube streaming on both application and network layers. Furthermore, the problems and methodology to obtain QoE results from such unsupervised field study are discussed together with the actual QoE results. Correlations between QoE factors are investigated, and the QoE of clusters of similar streaming sessions is analyzed.

• Application-Layer Monitoring of QoE Parameters for Mobile YouTube Video Streaming in the Field Seufert, Michael; Casas, Pedro; Wamser, Florian; Wehner, Nikolas; Schatz, Raimund; Tran-Gia, Phuoc in IEEE 6th International Conference on Communications and Electronics (ICCE) (2016).
  • [ Abstract ]
  • [ BibTeX ]
  • [ EndNote ]
  YouTube video streaming is one of the most popular and most demanding services in cellular networks. Thus, operators are concerned about the quality of the streaming delivered by their networks and would like to monitor the Quality of Experience (QoE) of the end users. In this work, we conduct a field study of mobile YouTube video streaming, in which both network flow parameters and application-layer streaming parameters were monitored, and present the characteristics of current mobile YouTube streaming. The impact of both approaches is investigated showing that monitoring network parameters is not sufficient to directly infer the resulting QoE. In contrast, the streaming parameters, which can be obtained from application-layer monitoring, show high correlations to the subjectively experienced quality, and thus, are better suited for QoE monitoring.

  @inproceedings{info3-inproceedings-2016-9, abstract = {YouTube video streaming is one of the most popular and most demanding services in cellular networks. Thus, operators are concerned about the quality of the streaming delivered by their networks and would like to monitor the Quality of Experience (QoE) of the end users. In this work, we conduct a field study of mobile YouTube video streaming, in which both network flow parameters and application-layer streaming parameters were monitored, and present the characteristics of current mobile YouTube streaming. The impact of both approaches is investigated showing that monitoring network parameters is not sufficient to directly infer the resulting QoE. In contrast, the streaming parameters, which can be obtained from application-layer monitoring, show high correlations to the subjectively experienced quality, and thus, are better suited for QoE monitoring.}, address = {Ha Long, Vietnam}, author = {Seufert, Michael and Casas, Pedro and Wamser, Florian and Wehner, Nikolas and Schatz, Raimund and Tran-Gia, Phuoc}, booktitle = {IEEE 6th International Conference on Communications and Electronics (ICCE)}, keywords = {can}, month = 7, title = {Application-Layer Monitoring of QoE Parameters for Mobile YouTube Video Streaming in the Field}, year = 2016 }

  %0 Conference Paper %1 info3-inproceedings-2016-9 %A Seufert, Michael %A Casas, Pedro %A Wamser, Florian %A Wehner, Nikolas %A Schatz, Raimund %A Tran-Gia, Phuoc %B IEEE 6th International Conference on Communications and Electronics (ICCE) %C Ha Long, Vietnam %D 2016 %T Application-Layer Monitoring of QoE Parameters for Mobile YouTube Video Streaming in the Field %X YouTube video streaming is one of the most popular and most demanding services in cellular networks. Thus, operators are concerned about the quality of the streaming delivered by their networks and would like to monitor the Quality of Experience (QoE) of the end users. In this work, we conduct a field study of mobile YouTube video streaming, in which both network flow parameters and application-layer streaming parameters were monitored, and present the characteristics of current mobile YouTube streaming. The impact of both approaches is investigated showing that monitoring network parameters is not sufficient to directly infer the resulting QoE. In contrast, the streaming parameters, which can be obtained from application-layer monitoring, show high correlations to the subjectively experienced quality, and thus, are better suited for QoE monitoring.