Conceptional architecture of policy-based approach

Online Reinforcement Learning für selbst-adaptive Informationssysteme

Unsicherheiten zur Designzeiten erschweren die Entwicklung selbstadaptiver Systeme. Alexander Palm stellt auf der virtuellen CAiSE'20 einen KI-Ansatz vor, der die Entwicklung der Adaptionslogik automatisiert.

Die 32nd International Conference on Advanced Information Systems Engineering (CAiSE’20) findet vom 8.-12. Juni 2020 statt. Alexander Palm wird die Arbeit aufgrund der aktuellen Situation online präsentieren.

Alexander Palm, Andreas Metzger und Klaus Pohl: Online Reinforcement Learning for Self-Adaptive Information Systems


A self-adaptive information system is capable of maintaining its quality requirements in the presence of dynamic environment changes. To develop a self-adaptive information system, information system engineers have to create self-adaptation logic that encodes when and how the system should adapt itself. However, developing self-adaptation logic may be difficult due to design time uncertainty; e.g., anticipating all potential environment changes at design time is in most cases infeasible. Online reinforcement learning addresses design time uncertainty by learning the effectiveness of adaptation actions through interactions with the system's environment at run time, thereby automating the development of self-adaptation logic. Existing online reinforcement learning approaches for self-adaptive information systems exhibit two shortcomings that limit the degree of automation: they require manually fine-tuning the exploration rate and may require manually quantizing environment states to foster scalability. We introduce an approach to automate the aforementioned manual activities by employing policy-based RL as a fundamentally different type of RL. We demonstrate the feasibility and applicability of our approach using two self-adaptive information system exemplars.

Kontakt: Alexander Palm, E-mail: alexander.palm(at), Tel.: +49 201 18-34659