Context-Driven Cooperative Intelligent Control for Distributed Cyber-Physical Actuation Platforms Using CTDE Multi-Agent Reinforcement Learning

Abstract

Modern ubiquitous and intelligent environments rely on distributed cyber-physical actuation platforms, which are difficult to control because of tight integration between actuators and partial observability as well as would not be achievable to net out centralised and fixed notions of control in dynamical environments. Centralised strategies are lacking in scalability as well as robustness limits whereas independent controllers fail to reduce cross-actuator interference. This paper suggests using Centralised Training and Decentralised Exercises (CTDE) Multi-Agents reinforcement learning (MARL) context-based control framework to propose a cooperative framework of intelligent control. Training the agents use globally available context information such as couplings strength and environmental conditions to acquire coordinated control policy during training, though execution is completely decentralised based on local observations only. The agent state representation incorporates the idea of context embedding to facilitate adaptive decision making and decision making that is aware of interference. The results of extensive simulations indicate that the proposed framework can significantly enhance control precision and reduce any interference cross-actuator in addition to ensuring stable convergence at alternative coupling levels in comparison with centralised and non-cooperative baselines. These findings demonstrate the efficiency of CTDE based cooperative learning of scalable and robust control and demonstrate its applicability to next generation ubiquitous computing and intelligent cyber-physical spaces.