Stefan Wunderer

Mail	stefan.wunderer@uni-wuerzburg.de
Anschrift	Lehrstuhl für Informatik III Am Hubland D-97074 Würzburg

Dozent der Vorlesung mit integrierter Übung: "Nachhaltigkeit und Informatik" im Wintersemester 2021/22

2021[ to top ]

Valid Statements by the Crowd: Statistical Measures for Precision in Crowdsourced Mobile Measurements Wamser, Florian; Seufert, Anika; Hall, Andrew; Wunderer, Stefan; Hoßfeld, Tobias in Network (2021). 1(2) 215–232.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
Crowdsourced network measurements (CNMs) are becoming increasingly popular as they assess the performance of a mobile network from the end user’s perspective on a large scale. Here, network measurements are performed directly on the end-users’ devices, thus taking advantage of the real-world conditions end-users encounter. However, this type of uncontrolled measurement raises questions about its validity and reliability. The problem lies in the nature of this type of data collection. In CNMs, mobile network subscribers are involved to a large extent in the measurement process, and collect data themselves for the operator. The collection of data on user devices in arbitrary locations and at uncontrolled times requires means to ensure validity and reliability. To address this issue, our paper defines concepts and guidelines for analyzing the precision of CNMs; specifically, the number of measurements required to make valid statements. In addition to the formal definition of the aspect, we illustrate the problem and use an extensive sample data set to show possible assessment approaches. This data set consists of more than 20.4 million crowdsourced mobile measurements from across France, measured by a commercial data provider.

@article{info3-article-2021-5, abstract = {Crowdsourced network measurements (CNMs) are becoming increasingly popular as they assess the performance of a mobile network from the end user’s perspective on a large scale. Here, network measurements are performed directly on the end-users’ devices, thus taking advantage of the real-world conditions end-users encounter. However, this type of uncontrolled measurement raises questions about its validity and reliability. The problem lies in the nature of this type of data collection. In CNMs, mobile network subscribers are involved to a large extent in the measurement process, and collect data themselves for the operator. The collection of data on user devices in arbitrary locations and at uncontrolled times requires means to ensure validity and reliability. To address this issue, our paper defines concepts and guidelines for analyzing the precision of CNMs; specifically, the number of measurements required to make valid statements. In addition to the formal definition of the aspect, we illustrate the problem and use an extensive sample data set to show possible assessment approaches. This data set consists of more than 20.4 million crowdsourced mobile measurements from across France, measured by a commercial data provider.}, author = {Wamser, Florian and Seufert, Anika and Hall, Andrew and Wunderer, Stefan and Hoßfeld, Tobias}, journal = {Network}, keywords = {can}, month = 9, number = 2, pages = {215–232}, title = {Valid Statements by the Crowd: Statistical Measures for Precision in Crowdsourced Mobile Measurements}, volume = 1, year = 2021 }

%0 Journal Article %1 info3-article-2021-5 %A Wamser, Florian %A Seufert, Anika %A Hall, Andrew %A Wunderer, Stefan %A Hoßfeld, Tobias %D 2021 %J Network %N 2 %P 215--232 %T Valid Statements by the Crowd: Statistical Measures for Precision in Crowdsourced Mobile Measurements %V 1 %X Crowdsourced network measurements (CNMs) are becoming increasingly popular as they assess the performance of a mobile network from the end user’s perspective on a large scale. Here, network measurements are performed directly on the end-users’ devices, thus taking advantage of the real-world conditions end-users encounter. However, this type of uncontrolled measurement raises questions about its validity and reliability. The problem lies in the nature of this type of data collection. In CNMs, mobile network subscribers are involved to a large extent in the measurement process, and collect data themselves for the operator. The collection of data on user devices in arbitrary locations and at uncontrolled times requires means to ensure validity and reliability. To address this issue, our paper defines concepts and guidelines for analyzing the precision of CNMs; specifically, the number of measurements required to make valid statements. In addition to the formal definition of the aspect, we illustrate the problem and use an extensive sample data set to show possible assessment approaches. This data set consists of more than 20.4 million crowdsourced mobile measurements from across France, measured by a commercial data provider.
Trust but Verify: Crowdsourced Mobile Network Measurements and Statistical Validity Measures Seufert, Anika; Wamser, Florian; Wunderer, Stefan; Hall, Andrew; Hoßfeld, Tobias in 2021 Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit) (2021).
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
Network operators, regulators, and big data companies use crowdsourced measurements to study the performance of mobile networks on a large scale. Such a type of measurement is defined as the collection and processing of data measured by the crowd, here the crowd of mobile subscribers. Crowdsourced network measurements make it relatively easy and inexpensive to obtain large amounts of network data that also reflect the quality actually received by the end user. However, this measurement method also involves some uncertainties, since, for example, it is not possible to precisely control when, where and with which devices measurements are taken. Thus, there is a trade-off between the reliability of the individual measurement and the scope of the measurements. Therefore, how data of this type is analyzed is particularly important in order to obtain valid results. To address this issue, our paper defines concepts and guidelines for analyzing the validity of crowdsourced mobile network measurements. In particular, we address precision, for example the number of measurements needed to make valid statements, and also representativeness, for example the spatial and temporal distribution of the data. In addition to the formal definition of these two aspects, we illustrate the issue and possible evaluation approaches with the help of an extensive example data set. This data set consists of more than 11.7M crowdsourced mobile measurements from all over France, measured by a commercial mobile data provider. In the end, we provide an evaluation guideline and two possible use cases.

@inproceedings{info3-inproceedings-2021-11, abstract = {Network operators, regulators, and big data companies use crowdsourced measurements to study the performance of mobile networks on a large scale. Such a type of measurement is defined as the collection and processing of data measured by the crowd, here the crowd of mobile subscribers. Crowdsourced network measurements make it relatively easy and inexpensive to obtain large amounts of network data that also reflect the quality actually received by the end user. However, this measurement method also involves some uncertainties, since, for example, it is not possible to precisely control when, where and with which devices measurements are taken. Thus, there is a trade-off between the reliability of the individual measurement and the scope of the measurements. Therefore, how data of this type is analyzed is particularly important in order to obtain valid results. To address this issue, our paper defines concepts and guidelines for analyzing the validity of crowdsourced mobile network measurements. In particular, we address precision, for example the number of measurements needed to make valid statements, and also representativeness, for example the spatial and temporal distribution of the data. In addition to the formal definition of these two aspects, we illustrate the issue and possible evaluation approaches with the help of an extensive example data set. This data set consists of more than 11.7M crowdsourced mobile measurements from all over France, measured by a commercial mobile data provider. In the end, we provide an evaluation guideline and two possible use cases.}, address = {Porto, Portugal (virtual conference)}, author = {Seufert, Anika and Wamser, Florian and Wunderer, Stefan and Hall, Andrew and Hoßfeld, Tobias}, booktitle = {2021 Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit)}, keywords = {can}, month = 6, title = {Trust but Verify: Crowdsourced Mobile Network Measurements and Statistical Validity Measures}, year = 2021 }

%0 Conference Paper %1 info3-inproceedings-2021-11 %A Seufert, Anika %A Wamser, Florian %A Wunderer, Stefan %A Hall, Andrew %A Hoßfeld, Tobias %B 2021 Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit) %C Porto, Portugal (virtual conference) %D 2021 %T Trust but Verify: Crowdsourced Mobile Network Measurements and Statistical Validity Measures %X Network operators, regulators, and big data companies use crowdsourced measurements to study the performance of mobile networks on a large scale. Such a type of measurement is defined as the collection and processing of data measured by the crowd, here the crowd of mobile subscribers. Crowdsourced network measurements make it relatively easy and inexpensive to obtain large amounts of network data that also reflect the quality actually received by the end user. However, this measurement method also involves some uncertainties, since, for example, it is not possible to precisely control when, where and with which devices measurements are taken. Thus, there is a trade-off between the reliability of the individual measurement and the scope of the measurements. Therefore, how data of this type is analyzed is particularly important in order to obtain valid results. To address this issue, our paper defines concepts and guidelines for analyzing the validity of crowdsourced mobile network measurements. In particular, we address precision, for example the number of measurements needed to make valid statements, and also representativeness, for example the spatial and temporal distribution of the data. In addition to the formal definition of these two aspects, we illustrate the issue and possible evaluation approaches with the help of an extensive example data set. This data set consists of more than 11.7M crowdsourced mobile measurements from all over France, measured by a commercial mobile data provider. In the end, we provide an evaluation guideline and two possible use cases.

2020[ to top ]

Crowdsourced Network Measurements in Germany: Mobile Internet Experience from End User Perspective Schwind, Anika; Wamser, Florian; Hoßfeld, Tobias; Wunderer, Stefan; Tarnvik, Erik; Hall, Andy in Broadband Coverage in Germany; 14. ITG Symposium (2020).
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
Collecting and analyzing meaningful data in mobile networks is the key to assessing network performance. Crowdsourced Network Measurements (CNMs) provide insights beyond the network layer and offer performance and other measurements at the application and user-level towards Quality of Experience (QoE). In this paper, the mobile Internet experience for Germany is evaluated with the help of crowdsourcing from the perspective of an end user. We statistically analyze a dataset with throughput measurements on the end device from Tutela Ltd., which covers more than 2.5 million throughput tests across Germany from January to July 2019. We give insights into this emerging methodology and highlight the benefits of this method. The paper contains statistics and conclusions for several large cities as well as regions in Germany compared to general statements for Germany, since individual measurements and averages often only imprecisely reflect the situation. The goal is to give a holistic view of the performance of the current mobile network in Germany. Reading this paper, it becomes evident that reliable statements about the quality of the mobile network for Germany depend on a large number of peculiarities in different regions with their own performance characteristics due to different network deployments and population numbers.

@inproceedings{info3-inproceedings-2020-13, abstract = {Collecting and analyzing meaningful data in mobile networks is the key to assessing network performance. Crowdsourced Network Measurements (CNMs) provide insights beyond the network layer and offer performance and other measurements at the application and user-level towards Quality of Experience (QoE). In this paper, the mobile Internet experience for Germany is evaluated with the help of crowdsourcing from the perspective of an end user. We statistically analyze a dataset with throughput measurements on the end device from Tutela Ltd., which covers more than 2.5 million throughput tests across Germany from January to July 2019. We give insights into this emerging methodology and highlight the benefits of this method. The paper contains statistics and conclusions for several large cities as well as regions in Germany compared to general statements for Germany, since individual measurements and averages often only imprecisely reflect the situation. The goal is to give a holistic view of the performance of the current mobile network in Germany. Reading this paper, it becomes evident that reliable statements about the quality of the mobile network for Germany depend on a large number of peculiarities in different regions with their own performance characteristics due to different network deployments and population numbers.}, address = {Berlin}, author = {Schwind, Anika and Wamser, Florian and Hoßfeld, Tobias and Wunderer, Stefan and Tarnvik, Erik and Hall, Andy}, booktitle = {Broadband Coverage in Germany; 14. ITG Symposium}, keywords = {can}, month = 8, title = {Crowdsourced Network Measurements in Germany: Mobile Internet Experience from End User Perspective}, year = 2020 }

%0 Conference Paper %1 info3-inproceedings-2020-13 %A Schwind, Anika %A Wamser, Florian %A Hoßfeld, Tobias %A Wunderer, Stefan %A Tarnvik, Erik %A Hall, Andy %B Broadband Coverage in Germany; 14. ITG Symposium %C Berlin %D 2020 %T Crowdsourced Network Measurements in Germany: Mobile Internet Experience from End User Perspective %X Collecting and analyzing meaningful data in mobile networks is the key to assessing network performance. Crowdsourced Network Measurements (CNMs) provide insights beyond the network layer and offer performance and other measurements at the application and user-level towards Quality of Experience (QoE). In this paper, the mobile Internet experience for Germany is evaluated with the help of crowdsourcing from the perspective of an end user. We statistically analyze a dataset with throughput measurements on the end device from Tutela Ltd., which covers more than 2.5 million throughput tests across Germany from January to July 2019. We give insights into this emerging methodology and highlight the benefits of this method. The paper contains statistics and conclusions for several large cities as well as regions in Germany compared to general statements for Germany, since individual measurements and averages often only imprecisely reflect the situation. The goal is to give a holistic view of the performance of the current mobile network in Germany. Reading this paper, it becomes evident that reliable statements about the quality of the mobile network for Germany depend on a large number of peculiarities in different regions with their own performance characteristics due to different network deployments and population numbers.
White Paper on Crowdsourced Network and QoE Measurements – Definitions, Use Cases and Challenges Hoßfeld, Tobias; Wunderer, Stefan; Beyer, André; Hall, Andrew; Schwind, Anika; Gassner, Christian; Guillemin, Fabrice; Wamser, Florian; Wascinski, Krzysztof; Hirth, Matthias; Seufert, Michael; Casas, Pedro; Tran-Gia, Phuoc; Robitza, Werner; Wascinski, Wojciech; Ben Houidi, Zied (2020).
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
The goal of the white paper at hand is as follows. The definitions of the terms build a framework for discussions around the hype topic ‘crowdsourcing’. This serves as a basis for differentiation and a consistent view from different perspectives on crowdsourced network measurements, with the goal to provide a commonly accepted definition in the community. The focus is on the context of mobile and fixed network operators, but also on measurements of different layers (network, application, user layer). In addition, the white paper shows the value of crowdsourcing for selected use cases, e.g., to improve QoE or regulatory issues. Finally, the major challenges and issues for researchers and practitioners are highlighted. This white paper is the outcome of the Würzburg seminar on “Crowdsourced Network and QoE Measurements” which took place from 25-26 September 2019 in Würzburg, Germany. International experts were invited from industry and academia. They are well known in their communities, having different backgrounds in crowdsourcing, mobile networks, network measurements, network performance, Quality of Service (QoS), and Quality of Experience (QoE). The discussions in the seminar focused on how crowdsourcing will support vendors, operators, and regulators to determine the Quality of Experience in new 5G networks that enable various new applications and network architectures. As a result of the discussions, the need for a white paper manifested, with the goal of providing a scientific discussion of the terms “crowdsourced network measurements” and “crowdsourced QoE measurements”, describing relevant use cases for such crowdsourced data, and its underlying challenges. During the seminar, those main topics were identified, intensively discussed in break-out groups, and brought back into the plenum several times. The outcome of the seminar is this white paper at hand which is – to our knowledge – the first one covering the topic of crowdsourced network and QoE measurements.

@techreport{info3-misc-2020-1, abstract = {The goal of the white paper at hand is as follows. The definitions of the terms build a framework for discussions around the hype topic ‘crowdsourcing’. This serves as a basis for differentiation and a consistent view from different perspectives on crowdsourced network measurements, with the goal to provide a commonly accepted definition in the community. The focus is on the context of mobile and fixed network operators, but also on measurements of different layers (network, application, user layer). In addition, the white paper shows the value of crowdsourcing for selected use cases, e.g., to improve QoE or regulatory issues. Finally, the major challenges and issues for researchers and practitioners are highlighted. This white paper is the outcome of the Würzburg seminar on “Crowdsourced Network and QoE Measurements” which took place from 25-26 September 2019 in Würzburg, Germany. International experts were invited from industry and academia. They are well known in their communities, having different backgrounds in crowdsourcing, mobile networks, network measurements, network performance, Quality of Service (QoS), and Quality of Experience (QoE). The discussions in the seminar focused on how crowdsourcing will support vendors, operators, and regulators to determine the Quality of Experience in new 5G networks that enable various new applications and network architectures. As a result of the discussions, the need for a white paper manifested, with the goal of providing a scientific discussion of the terms “crowdsourced network measurements” and “crowdsourced QoE measurements”, describing relevant use cases for such crowdsourced data, and its underlying challenges. During the seminar, those main topics were identified, intensively discussed in break-out groups, and brought back into the plenum several times. The outcome of the seminar is this white paper at hand which is – to our knowledge – the first one covering the topic of crowdsourced network and QoE measurements.}, author = {Hoßfeld, Tobias and Wunderer, Stefan and Beyer, André and Hall, Andrew and Schwind, Anika and Gassner, Christian and Guillemin, Fabrice and Wamser, Florian and Wascinski, Krzysztof and Hirth, Matthias and Seufert, Michael and Casas, Pedro and Tran-Gia, Phuoc and Robitza, Werner and Wascinski, Wojciech and Ben Houidi, Zied}, institution = {Würzburg}, keywords = {ua-net}, month = 3, number = {10.25972/OPUS-20232}, title = {White Paper on Crowdsourced Network and QoE Measurements – Definitions, Use Cases and Challenges}, year = 2020 }

%0 Report %1 info3-misc-2020-1 %A Hoßfeld, Tobias %A Wunderer, Stefan %A Beyer, André %A Hall, Andrew %A Schwind, Anika %A Gassner, Christian %A Guillemin, Fabrice %A Wamser, Florian %A Wascinski, Krzysztof %A Hirth, Matthias %A Seufert, Michael %A Casas, Pedro %A Tran-Gia, Phuoc %A Robitza, Werner %A Wascinski, Wojciech %A Ben Houidi, Zied %D 2020 %N 10.25972/OPUS-20232 %T White Paper on Crowdsourced Network and QoE Measurements – Definitions, Use Cases and Challenges %X The goal of the white paper at hand is as follows. The definitions of the terms build a framework for discussions around the hype topic ‘crowdsourcing’. This serves as a basis for differentiation and a consistent view from different perspectives on crowdsourced network measurements, with the goal to provide a commonly accepted definition in the community. The focus is on the context of mobile and fixed network operators, but also on measurements of different layers (network, application, user layer). In addition, the white paper shows the value of crowdsourcing for selected use cases, e.g., to improve QoE or regulatory issues. Finally, the major challenges and issues for researchers and practitioners are highlighted. This white paper is the outcome of the Würzburg seminar on “Crowdsourced Network and QoE Measurements” which took place from 25-26 September 2019 in Würzburg, Germany. International experts were invited from industry and academia. They are well known in their communities, having different backgrounds in crowdsourcing, mobile networks, network measurements, network performance, Quality of Service (QoS), and Quality of Experience (QoE). The discussions in the seminar focused on how crowdsourcing will support vendors, operators, and regulators to determine the Quality of Experience in new 5G networks that enable various new applications and network architectures. As a result of the discussions, the need for a white paper manifested, with the goal of providing a scientific discussion of the terms “crowdsourced network measurements” and “crowdsourced QoE measurements”, describing relevant use cases for such crowdsourced data, and its underlying challenges. During the seminar, those main topics were identified, intensively discussed in break-out groups, and brought back into the plenum several times. The outcome of the seminar is this white paper at hand which is – to our knowledge – the first one covering the topic of crowdsourced network and QoE measurements.

Hubland Süd, Geb. M2

Stefan Wunderer

Hinweis zum Datenschutz

Hinweis zum Datenschutz

Bildnachweise