Multi-agent active information gathering in discrete and continuous-state decentralized POMDPs by policy graph improvement

Mikko Lauri, Joni Pajarinen, Jan Peters

Research output: Contribution to journalArticleScientificpeer-review

15 Citations (Scopus)
25 Downloads (Pure)

Abstract

Decentralized policies for information gathering are required when multiple autonomous agents are deployed to collect data about a phenomenon of interest when constant communication cannot be assumed. This is common in tasks involving information gathering with multiple independently operating sensor devices that may operate over large physical distances, such as unmanned aerial vehicles, or in communication limited environments such as in the case of autonomous underwater vehicles. In this paper, we frame the information gathering task as a general decentralized partially observable Markov decision process (Dec-POMDP). The Dec-POMDP is a principled model for co-operative decentralized multi-agent decision-making. An optimal solution of a Dec-POMDP is a set of local policies, one for each agent, which maximizes the expected sum of rewards over time. In contrast to most prior work on Dec-POMDPs, we set the reward as a non-linear function of the agents’ state information, for example the negative Shannon entropy. We argue that such reward functions are well-suited for decentralized information gathering problems. We prove that if the reward function is convex, then the finite-horizon value function of the Dec-POMDP is also convex. We propose the first heuristic anytime algorithm for information gathering Dec-POMDPs, and empirically prove its effectiveness by solving discrete problems an order of magnitude larger than previous state-of-the-art. We also propose an extension to continuous-state problems with finite action and observation spaces by employing particle filtering. The effectiveness of the proposed algorithms is verified in domains such as decentralized target tracking, scientific survey planning, and signal source localization.

Original languageEnglish
Article number42
JournalAUTONOMOUS AGENTS AND MULTI-AGENT SYSTEMS
Volume34
Issue number2
DOIs
Publication statusPublished - 1 Oct 2020
Publication typeA1 Journal article-refereed

Keywords

  • Active perception
  • Decentralized POMDP
  • Information gathering
  • Planning under uncertainty

Publication forum classification

  • Publication forum level 2

ASJC Scopus subject areas

  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Multi-agent active information gathering in discrete and continuous-state decentralized POMDPs by policy graph improvement'. Together they form a unique fingerprint.

Cite this