Perception-based Co-evolutionary Reinforcement Learning for UAV Sensor Allocation

"Perception-based reasoning will be an important capability of future intelligent systems. One of the key aspects of this theory is granulation, which will allow agents to process imprecise information in large complex state spaces. In complex and uncertainenvironments, autonomous agents will need to adapt the initial definitions of granules and perception structures according to new information received from the environment. We propose a general Perception-based Reinforcement Learning framework fordynamically adapting strategic decisions about motion and sensor management in Unmanned Aerial Vehicles (UAVs). As a motivating example, we will consider a reconnaissance mission by UAVs with the goal of identifying and tracking relevant targets. Such amission requires sensor allocation at two levels: the individual UAV and the team-wide allocation. At the individual level, the total available UAV sensor resources need to be dynamically allocated to tracking appearing and disappearing targets. At theteam level, each UAV is treated as a composite sensor, and these composite sensors need to be allocated to different regions of the search space in proportion to the density and importance of targets there. In Phase I of this research we will study thefeasibility of co-evolutionary adaptation of individual and team-level sensor allocation policies. The algorithm for allocating UAV sensor resources to dynamically appearing targets can be applied with minor modifications to many resource allocationproblems in telecommunication networks, e.g. call admission in circuit switched networks and multimedia flow allocation under DiffServ QoS paradigm in the Internet. The team level algorithm for allocating UAVs to different regions of the search space canbe used in distributed dynamic load balancing, e.g. moving cached content on the Web closer to "hot spots.""