Carlo D'Eramo
Prof. Dr. Carlo D'Eramo
John-Skilton-Straße 8a
About me
I am the head of the professorship for Reinforcement Learning and Computational Decision-Making at the Center for Artificial Intelligence and Data Science of Julius-Maximilians-Universität Würzburg. I am also independent group leader of hessian.AI.
The research of my LiteRL group revolves around the problem of how agents can efficiently acquire expert skills that account for the complexity of the real world. To answer this question, we are investigating lightweight methods to obtain adaptive autonomous agents, focusing on several RL topics including multi-task, curriculum, adversarial, options, and multi-agent RL.
Previously, I have studied Computer Engineering at Politecnico di Milano (Italy), where I obtained a B.Sc. degree in 2011 and an M.Sc. degree in 2015. I obtained a double degree in Computer Science at University of Illinois at Chicago (USA) in 2015. Afterwards, I conducted a Ph.D. in Information Technology at Politecnico di Milano (Italy), where I graduated in February 2019 with a thesis "On the exploitation of uncertainty to improve Bellman updates and exploration in Reinforcement Learning". Then, I have been a postdoctoral researcher at the Intelligent Autonomous Systems (IAS) group at TU Darmstadt from April 2019 to October 2022.
Follow me on Twitter: @CarloDeramo
- 2022-now: Professor of Reinforcement Learning and Computational Decision-Making at University of Würzburg (Germany)
- 2022-now: Independent research group leader of the LiteRL group at hessian.AI (Germany)
- 2019-2022: Postdoctoral researcher in Reinforcement Learning at TU Darmstadt (Germany)
- 2015-2019: Ph.D. student of Information Technology at Politecnico di Milano (Italy)
- 2011-2015: M.Sc. student of Computer Science at University of Illinois at Chicago (USA)
- 2011-2015: M.Sc. student of Computer Engineering at Politecnico di Milano (Italy)
- 2008-2011: B.Sc. student of Computer Engineering at Politecnico di Milano (Italy)
Journal articles
-
Simone Parisi, ..., Carlo D'Eramo, et al. "Long-Term Visitation Value for Deep Exploration in Sparse-Reward Reinforcement Learning." Algorithms 15.3 (2022): 81.
-
Pascal Klink, ..., Carlo D'Eramo, et al. "A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learning." J. Mach. Learn. Res. 22 (2021): 182-1.
-
Carlo D'Eramo, et al. "Gaussian Approximation for Bias Reduction in Q-Learning." Journal of Machine Learning Research 22 (2021): 1-51.
-
Carlo D'Eramo, et al. "MushroomRL: Simplifying Reinforcement Learning Research." Journal of Machine Learning Research 22 (2021): 1-5.
-
Dorothea Koert, ..., Carlo D'Eramo, et al. "Multi-channel interactive reinforcement learning for sequential tasks." Frontiers in Robotics and AI (2020): 97.
Conference proceedings
-
Carlo D'Eramo, and Georgia Chalvatzaki. "Prioritized Sampling with Intrinsic Motivation in Multi-Task Reinforcement Learning." 2022 International Joint Conference on Neural Networks (IJCNN). IEEE, 2022.
-
Pascal Klink, ..., Carlo D'Eramo et al. "Curriculum reinforcement learning via constrained optimal transport." International Conference on Machine Learning. PMLR, 2022.
-
Pascal Klink, Carlo D'Eramo, et al. "Boosted Curriculum Reinforcement Learning." International Conference on Learning Representations. 2022.
-
Tuan Dam, Carlo D'Eramo, et al. "Convex Regularization in Monte-Carlo Tree Search." International Conference on Machine Learning. PMLR, 2021.
-
Julen Urain, ..., Carlo D'Eramo, et al. "Composable energy policies for reactive motion generation and reinforcement learning." Robotics: Science and Systems (RSS), 2021.
-
Andrew S. Morgan, ..., Carlo D'Eramo, et al. "Model predictive actor-critic: Accelerating robot skill acquisition with deep reinforcement learning." 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021.
-
Carlo D'Eramo, et al. "Sharing knowledge in multi-task deep reinforcement learning." 8th International Conference on Learning Representations, 2020.
-
Pascal Klink, Carlo D'Eramo, et al. "Self-paced deep reinforcement learning." Advances in Neural Information Processing Systems 33 (2020): 9216-9227.
-
Tuan Dam, Carlo D'Eramo, et al. "Generalized mean estimation in Monte-Carlo tree search." Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence. 2021.
-
Samuele Tosatto, Carlo D'Eramo, et al. "Exploration driven by an optimistic bellman equation." 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 2019.
-
Carlo D'Eramo, Andrea Cini, and Marcello Restelli. "Exploiting action-value uncertainty to drive exploration in reinforcement learning." 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 2019.
-
Davide Tateo, Carlo D'Eramo, et al. "Exploiting structure and uncertainty of Bellman updates in Markov decision processes." 2017 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, 2017.
-
Samuele Tosatto, Carlo D'Eramo, et al. "Boosted fitted q-iteration." International Conference on Machine Learning. PMLR, 2017.
-
Carlo D'Eramo, et al. "Estimating the maximum expected value in continuous reinforcement learning problems." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 31. No. 1. 2017.
-
Carlo D'Eramo, Marcello Restelli, and Alessandro Nuara. "Estimating maximum expected value through gaussian approximation." International Conference on Machine Learning. PMLR, 2016.
Workshop papers
- Oliver Järnefelt, Carlo D'Eramo. "Modular Value Function Factorization in Multi-Agent Reinforcement Learning". Workshop on Decision-Making in Multi-Agent Systems. IEEE IROS. 2022.
Theses
-
Carlo D'Eramo. "On the exploitation of uncertainty to improve Bellman updates and exploration in Reinforcement Learning." Ph.D. Thesis (2019).
-
Carlo D'Eramo. "On the use of deep Boltzmann machines for road signs classification." M.Sc. Thesis. Politecnico di Milano (2015).
Teaching
2022 - TU Darmstadt: Introduction to Reinforcement Learning: from foundations to deep approaches
