DEC-POMDP Stochastic Game Approach for Uncertain MultiAgent Systems

The key innovation builds on the decentralized partially observable markov decision processes theory to model unmanned vehicles (UVs) that engage in stochastic game formulations for collaborative teaming and computing joint optimal policies. Prior models consider purely collaborative agents which have identical payoff functions contrary to the human-centric models. We incorporate agent self-interestedness in the payoff functions and maximize expected team reward. Learning components are incorporated in the agents to reduce the search space for optimal actions given a history of world observations. It is our intuition that such a framework would provide computationally tractable performance even though DEC-POMDP are shown to be NEXP-complete with no communication. The framework would have host of generic algorithms that can be easily adapted as per the scenario definition like target tracking, formation flying, planning etc. The proposed approach can be implemented on CybeleTM agent DSSI (Decision support system infrastructure) capability built by IAI to model agent beliefs, actions and rewards with hooks for the environment simulation. Cybele infrastructure has been tested on CDC environment (e.g., wireless PDAs), which allows users to develop agent applications and algorithms, simulate the algorithms and directly deploy the software on wireless networked environment, enabling hardware in the loop simulations.