Reference:
Joni Pajarinen and Jaakko Peltonen. Periodic Finite State Controllers for Efficient POMDP and DEC-POMDP Planning. In Proceedings of the 25th Annual Conference on Neural Information Processing Systems (NIPS), pages 2636–2644, December 2011.
Abstract:
Applications such as robot control and wireless communication require planning under uncertainty. Partially observable Markov decision processes (POMDPs) plan policies for single agents under uncertainty and their decentralized versions (DEC-POMDPs) find a policy for multiple agents. The policy in infinite-horizon POMDP and DEC-POMDP problems has been represented as finite state controllers (FSCs). We introduce a novel class of periodic FSCs, composed of layers connected only to the previous and next layer. Our periodic FSC method finds a deterministic finite-horizon policy and converts it to an initial periodic infinite-horizon policy. This policy is optimized by a new infinite-horizon algorithm to yield deterministic periodic policies, and by a new expectation maximization algorithm to yield stochastic periodic policies. Our method yields better results than earlier planning methods and can compute larger solutions than with regular FSCs.
Suggested BibTeX entry:
@inproceedings{pajarinen11c,
author = {Pajarinen, Joni and Peltonen, Jaakko},
booktitle = {Proceedings of the 25th Annual Conference on Neural Information Processing Systems (NIPS)},
language = {eng},
month = {December},
pages = {2636--2644},
title = {{Periodic Finite State Controllers for Efficient POMDP and DEC-POMDP Planning}},
year = {2011},
}
|