piwik-script

Intern
Reinforcement Learning and Computational Decision-Making

Carlo D'Eramo

Prof. Dr. Carlo D'Eramo

Head
Reinforcement Learning and Computational Decision-Making
Julius-Maximilians-Universität
John-Skilton-Straße 8a
97074 Würzburg
Deutschland
Gebäude: JS8
Raum: 04.016
Telefon: +49 931 31-83457

 

 

About me

I am the head of the professorship for Reinforcement Learning and Computational Decision-Making at the Center for Artificial Intelligence and Data Science of Julius-Maximilians-Universität Würzburg. I am also independent group leader of hessian.AI.

The research of my LiteRL group revolves around the problem of how agents can efficiently acquire expert skills that account for the complexity of the real world. To answer this question, we are investigating lightweight methods to obtain adaptive autonomous agents, focusing on several RL topics including multi-task, curriculum, adversarial, options, and multi-agent RL.

Previously, I have studied Computer Engineering at Politecnico di Milano (Italy), where I obtained a B.Sc. degree in 2011 and an M.Sc. degree in 2015. I obtained a double degree in Computer Science at University of Illinois at Chicago (USA) in 2015. Afterwards, I conducted a Ph.D. in Information Technology at Politecnico di Milano (Italy), where I graduated in February 2019 with a thesis "On the exploitation of uncertainty to improve Bellman updates and exploration in Reinforcement Learning". Then, I have been a postdoctoral researcher at the Intelligent Autonomous Systems (IAS) group at TU Darmstadt from April 2019 to October 2022.

Follow me on Twitter: @CarloDeramo

Check out my CV here.

Area chair: AAAI, ACML, AISTATS, NeurIPS.

Journal articles

  • Julen Urain, ..., Carlo D'Eramo, et al. "Composable Energy Policies for Reactive Motion Generation and Reinforcement Learning". International Journal of Robotics Research (2023).

  • Simone Parisi, ..., Carlo D'Eramo, et al. "Long-Term Visitation Value for Deep Exploration in Sparse-Reward Reinforcement Learning." Algorithms 15.3 (2022): 81.

  • Pascal Klink, ..., Carlo D'Eramo, et al. "A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learning." Journal of Machine Learning Research 22 (2021): 182-1.

  • Carlo D'Eramo, et al. "Gaussian Approximation for Bias Reduction in Q-Learning." Journal of Machine Learning Research 22 (2021): 1-51.

  • Carlo D'Eramo, et al. "MushroomRL: Simplifying Reinforcement Learning Research." Journal of Machine Learning Research 22 (2021): 1-5.

  • Dorothea Koert, ..., Carlo D'Eramo, et al. "Multi-channel interactive reinforcement learning for sequential tasks." Frontiers in Robotics and AI (2020): 97.

Conference proceedings

  •  Carlo D'Eramo, and Georgia Chalvatzaki. "Prioritized Sampling with Intrinsic Motivation in Multi-Task Reinforcement Learning." 2022 International Joint Conference on Neural Networks (IJCNN). IEEE, 2022.

  •  Pascal Klink, ..., Carlo D'Eramo et al. "Curriculum reinforcement learning via constrained optimal transport." International Conference on Machine Learning. PMLR, 2022.

  •  Pascal Klink, Carlo D'Eramo, et al. "Boosted Curriculum Reinforcement Learning." International Conference on Learning Representations. 2022.

  •  Tuan Dam, Carlo D'Eramo, et al. "Convex Regularization in Monte-Carlo Tree Search." International Conference on Machine Learning. PMLR, 2021.

  •  Julen Urain, ..., Carlo D'Eramo, et al. "Composable energy policies for reactive motion generation and reinforcement learning." Robotics: Science and Systems (RSS), 2021.

  •  Andrew S. Morgan, ..., Carlo D'Eramo, et al. "Model predictive actor-critic: Accelerating robot skill acquisition with deep reinforcement learning." 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021.

  •  Carlo D'Eramo, et al. "Sharing knowledge in multi-task deep reinforcement learning." 8th International Conference on Learning Representations, 2020.

  •  Pascal Klink, Carlo D'Eramo, et al. "Self-paced deep reinforcement learning." Advances in Neural Information Processing Systems 33 (2020): 9216-9227.

  •  Tuan Dam, Carlo D'Eramo, et al. "Generalized mean estimation in Monte-Carlo tree search." Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence. 2021.

  •  Samuele Tosatto, Carlo D'Eramo, et al. "Exploration driven by an optimistic bellman equation." 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 2019.

  •  Carlo D'Eramo, Andrea Cini, and Marcello Restelli. "Exploiting action-value uncertainty to drive exploration in reinforcement learning." 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 2019.

  •  Davide Tateo, Carlo D'Eramo, et al. "Exploiting structure and uncertainty of Bellman updates in Markov decision processes." 2017 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, 2017.

  •  Samuele Tosatto, Carlo D'Eramo, et al. "Boosted fitted q-iteration." International Conference on Machine Learning. PMLR, 2017.

  •  Carlo D'Eramo, et al. "Estimating the maximum expected value in continuous reinforcement learning problems." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 31. No. 1. 2017.

  •  Carlo D'Eramo, Marcello Restelli, and Alessandro Nuara. "Estimating maximum expected value through gaussian approximation." International Conference on Machine Learning. PMLR, 2016.

Workshop papers

  • Tuan Dam, Carlo D'Eramo, Joni Pajarinen, Jan Peters. A Unified Perspective on Value Backup and Exploration in Monte-Carlo Tree Search. European Workshop on Reinforcement Learning (EWRL). 2023.
  • Théo Vincent, Boris Belousov, Carlo D'Eramo, Jan Peters. Iterated Deep Q-Network: Efficient Learning of Bellman Iterations for Deep Reinforcement Learning. European Workshop on Reinforcement Learning (EWRL). 2023.
  • Théo Vincent, Alberto Maria Metelli, Jan Peters, Marcello Restelli, Carlo D'Eramo. "Parameterized projected Bellman operator". ICML-Frontiers4LCD. 2023.
  • Oliver Järnefelt, Carlo D'Eramo. "Modular Value Function Factorization in Multi-Agent Reinforcement Learning". Workshop on Decision-Making in Multi-Agent Systems. IEEE IROS. 2022.
  • Pascal Klink, Haoyi Yang, Carlo D'Eramo, Jan Peters, Joni Pajarinen. "Curriculum Reinforcement Learning via Constrained Optimal Transport". European Workshop on Reinforcement Learning (EWRL). 2022.

Theses

  • Carlo D'Eramo. "On the exploitation of uncertainty to improve Bellman updates and exploration in Reinforcement Learning." Ph.D. Thesis (2019).

  • Carlo D'Eramo. "On the use of deep Boltzmann machines for road signs classification." M.Sc. Thesis. Politecnico di Milano (2015).

Teaching
  • 2023 - JMU: Reinforcement Learning and Computational Decision-Making;
  • 2023 - JMU: Seminar Introduction to Reinforcement Learning: from foundations to modern approaches;
  • 2022 - TU Darmstadt: Introduction to Reinforcement Learning: from foundations to deep approaches.