We live in a fast-paced, networked world. In this world, the need for self-adaptive software is growing, i.e. software that is able to adapt independently to changing environmental situations. Examples include fluctuating transmission bandwidths, a changing number of users, or changing user preferences.

An essential task in the development of self-adaptive software is to specify when and how an adaptation should take place. This is difficult for software engineers because at the time of development they are usually not able to predict all possible environmental situations the software might encounter. The scientists from Prof. Klaus Pohl's paluno working group are responding to this challenge by using online reinforcement learning, a method of artificial intelligence. The idea behind this is that the software itself learns which adaptation is the best in which situation by collecting and evaluating feedback at runtime. Good adjustments lead to positive feedback, while bad adjustments result in negative feedback. In this way, the software learns to collect the best possible feedback by trying it out. Thus, the previously manual activity of software engineers to determine when and how an adaptation should take place is taken over by artificial intelligence.

"Previous methods of online reinforcement learning still have one problem", says Dr. Andreas Metzger, head of the Adaptive Systems department at paluno. " The so-called exploration rate must be fine-tuned manually. This is the likelihood that the software will try a new customization instead of limiting itself to known, secure customizations. " Because this is at the expense of automation, the paluno team uses a novel learning algorithm: It is called Policy-based Reinforcement Learning and does not require a fine-tuning of the exploration rate. Initial tests on self-adaptive systems from the areas of business process management and web applications showed that the software is able to learn suitable adaptations for different environmental situations with the help of this algorithm. The scientists now also want to use their findings for the development of self-adaptive transport management systems and smart home systems. Together with European partners, they address these areas in the EU projects DataPorts and ENACT.

Presentation at CAiSE 2020 and BPM 2020

The learning approach and the application to self-adaptive web applications were presented by Alexander Palm at the 32nd International Conference on Advanced Information Systems Engineering (CAiSE 2020) in June 2020. The extended learning approach and the application to self-adaptive business process management will be presented by Andreas Metzger at the 18th International Conference on Business Process Management (BPM 2020) in September 2020.