Andrzej Dulny, M.Sc.

Chair of Data Science (Informatik X)
University of Würzburg
Am Hubland
97074 Würzburg
Germany

Email: andrzej.dulny@uni-wuerzburg.de

Phone: (+49 931) 31 - 81316

Fax: (+49 931) 31 81316-0

Projects and Research Interests

I have been part of the DMIR Research Group since early 2021, after I received my master's degree in Mathematics at the University of Würzburg. Currently pursuing research at Prof. Hotho's research group, I am passionate about leveraging machine learning techniques to predict the evolution of physical systems, extract the physical equations governing the system, and enhance simulation methods with machine learning. My work lies at the intersection of machine learning and physics, with a particular emphasis on developing and applying novel deep learning methods to solve complex problems in the field of dynamical systems using:

Physics-Informed Neural Networks (PINNs)
NeuralODEs
Graph Neural Networks
Transformer-based models for spatial data

Furthermore, I am particularily interested in handling non-grid and low-resolution data.

The methods I develop in my research can be applied to improve simulation and prediction of a variety of physical systems including weather prediction, climate models, wave simulations, fluid dynamics etc. Additionally, I apply the results of my research within the MAGNET4cardiac7T research project, where I aim to train neural networks for simulating electromagnetic fields within a MRI scanner.

Education

2017–2020: M.Sc. Mathematics at the University of Würzburg
2013–2016: B.Sc. Mathematics at the Jagiellonian University in Cracow

Publications

2023[ to top ]

Heinisch, P., Dulny, A., Krause, A., Hotho, A. (2023) TaylorPDENet: Learning PDEs from non-grid Data, available: http://arxiv.org/abs/2306.14511.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
Modeling data obtained from dynamical systems has gained attention in recent years as a challenging task for machine learning models. Previous approaches assume the measurements to be distributed on a grid. However, for real-world applications like weather prediction, the observations are taken from arbitrary locations within the spatial domain. In this paper, we propose TaylorPDENet - a novel machine learning method that is designed to overcome this challenge. Our algorithm uses the multidimensional Taylor expansion of a dynamical system at each observation point to estimate the spatial derivatives to perform predictions. TaylorPDENet is able to accomplish two objectives simultaneously: accurately forecast the evolution of a complex dynamical system and explicitly reconstruct the underlying differential equation describing the system. We evaluate our model on a variety of advection-diffusion equations with different parameters and show that it performs similarly to equivalent approaches on grid-structured data while being able to process unstructured data as well.

@misc{heinisch2023taylorpdenet, abstract = {Modeling data obtained from dynamical systems has gained attention in recent years as a challenging task for machine learning models. Previous approaches assume the measurements to be distributed on a grid. However, for real-world applications like weather prediction, the observations are taken from arbitrary locations within the spatial domain. In this paper, we propose TaylorPDENet - a novel machine learning method that is designed to overcome this challenge. Our algorithm uses the multidimensional Taylor expansion of a dynamical system at each observation point to estimate the spatial derivatives to perform predictions. TaylorPDENet is able to accomplish two objectives simultaneously: accurately forecast the evolution of a complex dynamical system and explicitly reconstruct the underlying differential equation describing the system. We evaluate our model on a variety of advection-diffusion equations with different parameters and show that it performs similarly to equivalent approaches on grid-structured data while being able to process unstructured data as well.}, author = {Heinisch, Paul and Dulny, Andrzej and Krause, Anna and Hotho, Andreas}, keywords = {neural-ode}, note = {cite arxiv:2306.14511}, title = {TaylorPDENet: Learning PDEs from non-grid Data}, year = 2023 }
Dulny, A., Hotho, A., Krause, A. (2023) DynaBench: A Benchmark Dataset for Learning Dynamical Systems from Low-Resolution Data, in Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E. and Bonchi, F., eds., Machine Learning and Knowledge Discovery in Databases: Research Track, Cham: Springer Nature Switzerland, 438–455, available: http://arxiv.org/abs/2306.05805.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
Previous work on learning physical systems from data has focused on high-resolution grid-structured measurements. However, real-world knowledge of such systems (e.g. weather data) relies on sparsely scattered measuring stations. In this paper, we introduce a novel simulated benchmark dataset, DynaBench, for learning dynamical systems directly from sparsely scattered data without prior knowledge of the equations. The dataset focuses on predicting the evolution of a dynamical system from low-resolution, unstructured measurements. We simulate six different partial differential equations covering a variety of physical systems commonly used in the literature and evaluate several machine learning models, including traditional graph neural networks and point cloud processing models, with the task of predicting the evolution of the system. The proposed benchmark dataset is expected to advance the state of art as an out-of-the-box easy-to-use tool for evaluating models in a setting where only unstructured low-resolution observations are available. The benchmark is available at https://professor-x.de/dynabench.

@inproceedings{10.1007/978-3-031-43412-9_26, abstract = {Previous work on learning physical systems from data has focused on high-resolution grid-structured measurements. However, real-world knowledge of such systems (e.g. weather data) relies on sparsely scattered measuring stations. In this paper, we introduce a novel simulated benchmark dataset, DynaBench, for learning dynamical systems directly from sparsely scattered data without prior knowledge of the equations. The dataset focuses on predicting the evolution of a dynamical system from low-resolution, unstructured measurements. We simulate six different partial differential equations covering a variety of physical systems commonly used in the literature and evaluate several machine learning models, including traditional graph neural networks and point cloud processing models, with the task of predicting the evolution of the system. The proposed benchmark dataset is expected to advance the state of art as an out-of-the-box easy-to-use tool for evaluating models in a setting where only unstructured low-resolution observations are available. The benchmark is available at https://professor-x.de/dynabench.}, address = {Cham}, author = {Dulny, Andrzej and Hotho, Andreas and Krause, Anna}, booktitle = {Machine Learning and Knowledge Discovery in Databases: Research Track}, editor = {Koutra, Danai and Plant, Claudia and Gomez Rodriguez, Manuel and Baralis, Elena and Bonchi, Francesco}, keywords = {neural-ode}, pages = {438–455}, publisher = {Springer Nature Switzerland}, title = {DynaBench: A Benchmark Dataset for Learning Dynamical Systems from Low-Resolution Data}, year = 2023 }

2022[ to top ]

Dulny, A., Hotho, A., Krause, A. (2022) NeuralPDE: Modelling Dynamical Systems from Data, in Bergmann, R., Malburg, L., Rodermund, S.C. and Timm, I.J., eds., KI 2022: Advances in Artificial Intelligence, Cham: Springer International Publishing, 75–89.
- [ Abstract ]
- [ BibTeX ]
Many physical processes such as weather phenomena or fluid mechanics are governed by partial differential equations (PDEs). Modelling such dynamical systems using Neural Networks is an active research field. However, current methods are still very limited, as they do not exploit the knowledge about the dynamical nature of the system, require extensive prior knowledge about the governing equations or are limited to linear or first-order equations. In this work we make the observation that the Method of Lines used to solve PDEs can be represented using convolutions which makes convolutional neural networks (CNNs) the natural choice to parametrize arbitrary PDE dynamics. We combine this parametrization with differentiable ODE solvers to form the NeuralPDE Model, which explicitly takes into account the fact that the data is governed by differential equations. We show in several experiments on toy and real-world data that our model consistently outperforms state-of-the-art models used to learn dynamical systems.

@inproceedings{10.1007/978-3-031-15791-2_8, abstract = {Many physical processes such as weather phenomena or fluid mechanics are governed by partial differential equations (PDEs). Modelling such dynamical systems using Neural Networks is an active research field. However, current methods are still very limited, as they do not exploit the knowledge about the dynamical nature of the system, require extensive prior knowledge about the governing equations or are limited to linear or first-order equations. In this work we make the observation that the Method of Lines used to solve PDEs can be represented using convolutions which makes convolutional neural networks (CNNs) the natural choice to parametrize arbitrary PDE dynamics. We combine this parametrization with differentiable ODE solvers to form the NeuralPDE Model, which explicitly takes into account the fact that the data is governed by differential equations. We show in several experiments on toy and real-world data that our model consistently outperforms state-of-the-art models used to learn dynamical systems.}, address = {Cham}, author = {Dulny, Andrzej and Hotho, Andreas and Krause, Anna}, booktitle = {KI 2022: Advances in Artificial Intelligence}, editor = {Bergmann, Ralph and Malburg, Lukas and Rodermund, Stephanie C. and Timm, Ingo J.}, keywords = {neuralpde}, pages = {75–89}, publisher = {Springer International Publishing}, title = {NeuralPDE: Modelling Dynamical Systems from Data}, year = 2022 }

2021[ to top ]

Wankerl, S., Dulny, A., Götz, G., Hotho, A. (2021) Learning Mathematical Relations Using Deep Tree Models, in 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), 1681–1687, available: https://doi.org/10.1109/ICMLA52953.2021.00268.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
To the present day, computer algebra systems used for calculating mathematical relations on a symbolic level are mainly rule-based systems. Only recently, deep learning has been applied to the task of symbolic mathematics. Researchers succeeded in developing neural networks which can do symbolic calculations of the equivalence of multiple pairing of equations. However, the generator used to create mathematical terms in previous research has several drawbacks and as with all deep learning tasks, the quality of the data used for training the models has a significant impact on the quality of the results. In this work, we propose a new generator for polynomials. We use it to train several recursive neural networks to recognize the equivalence, derivative, or variable substitution between pairs of polynomials for the first time. Our results indicate that these mathematical relations are identifiable with the help of deep learning.

@inproceedings{9680206, abstract = {To the present day, computer algebra systems used for calculating mathematical relations on a symbolic level are mainly rule-based systems. Only recently, deep learning has been applied to the task of symbolic mathematics. Researchers succeeded in developing neural networks which can do symbolic calculations of the equivalence of multiple pairing of equations. However, the generator used to create mathematical terms in previous research has several drawbacks and as with all deep learning tasks, the quality of the data used for training the models has a significant impact on the quality of the results. In this work, we propose a new generator for polynomials. We use it to train several recursive neural networks to recognize the equivalence, derivative, or variable substitution between pairs of polynomials for the first time. Our results indicate that these mathematical relations are identifiable with the help of deep learning.}, author = {Wankerl, Sebastian and Dulny, Andrzej and Götz, Gerhard and Hotho, Andreas}, booktitle = {2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA)}, keywords = {treemodels}, month = {dec}, pages = {1681-1687}, title = {Learning Mathematical Relations Using Deep Tree Models}, year = 2021 }
Kobs, K., Steininger, M., Dulny, A., Hotho, A. (2021) Do Different Deep Metric Learning Losses Lead to Similar Learned Features?, Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 10644–10654.
- [ BibTeX ]
@article{noauthororeditor, author = {Kobs, Konstantin and Steininger, Michael and Dulny, Andrzej and Hotho, Andreas}, journal = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)}, keywords = {myown}, month = {October}, pages = {10644-10654}, title = {Do Different Deep Metric Learning Losses Lead to Similar Learned Features?}, year = 2021 }

2020[ to top ]

Dulny, A., Steininger, M., Lautenschlager, F., Krause, A., Hotho, A. (2020) Evaluating the multi-task learning approach for land use regression modelling of air pollution, in International Conference on Frontiers of Artificial Intelligence and Machine Learning, IASED.
- [ Abstract ]
- [ BibTeX ]
Air pollution has been linked to several health problems including heart disease, stroke and lung cancer. Modelling and analyzing this dependency requires reliable and accurate air pollutant measurements collected by stationary air monitoring stations. However, usually only a low number of such stations are present within a single city. To retrieve pollution concentrations for unmeasured locations, researchers rely on land use regression (LUR) models. Those models are typically developed for one pollutant only. However, as results in different areas have shown, modelling several related output variables through multi-task learning can improve the prediction results of the models significantly. In this work, we compared prediction results from single-task and multi-task learning multilayer perceptron models on measurements taken from the OpenSense dataset and the London Atmospheric Emissions Inventory dataset. LUR features were generated from OpenStreetMap using OpenLUR and used to train hard parameter sharing multilayer perceptron models. The results show multi-task learning with sufficient data significantly improves the performance of a LUR model.

@inproceedings{dulny2020evaluating, abstract = {Air pollution has been linked to several health problems including heart disease, stroke and lung cancer. Modelling and analyzing this dependency requires reliable and accurate air pollutant measurements collected by stationary air monitoring stations. However, usually only a low number of such stations are present within a single city. To retrieve pollution concentrations for unmeasured locations, researchers rely on land use regression (LUR) models. Those models are typically developed for one pollutant only. However, as results in different areas have shown, modelling several related output variables through multi-task learning can improve the prediction results of the models significantly. In this work, we compared prediction results from single-task and multi-task learning multilayer perceptron models on measurements taken from the OpenSense dataset and the London Atmospheric Emissions Inventory dataset. LUR features were generated from OpenStreetMap using OpenLUR and used to train hard parameter sharing multilayer perceptron models. The results show multi-task learning with sufficient data significantly improves the performance of a LUR model.}, author = {Dulny, Andrzej and Steininger, Michael and Lautenschlager, Florian and Krause, Anna and Hotho, Andreas}, booktitle = {International Conference on Frontiers of Artificial Intelligence and Machine Learning}, keywords = {myown}, organization = {IASED}, title = {Evaluating the multi-task learning approach for land use regression modelling of air pollution}, year = 2020 }

Andrzej Dulny, M.Sc.