Reference:

Joni Pajarinen and Jaakko Peltonen. Periodic Finite State Controllers for Efficient POMDP and DEC-POMDP Planning. In Proceedings of the 25th Annual Conference on Neural Information Processing Systems (NIPS), pages 2636–2644, December 2011.

Abstract:

Applications such as robot control and wireless communication require planning under uncertainty. Partially observable Markov decision processes (POMDPs) plan policies for single agents under uncertainty and their decentralized versions (DEC-POMDPs) find a policy for multiple agents. The policy in infinite-horizon POMDP and DEC-POMDP problems has been represented as finite state controllers (FSCs). We introduce a novel class of periodic FSCs, composed of layers connected only to the previous and next layer. Our periodic FSC method finds a deterministic finite-horizon policy and converts it to an initial periodic infinite-horizon policy. This policy is optimized by a new infinite-horizon algorithm to yield deterministic periodic policies, and by a new expectation maximization algorithm to yield stochastic periodic policies. Our method yields better results than earlier planning methods and can compute larger solutions than with regular FSCs.

Suggested BibTeX entry:

@inproceedings{pajarinen11c,
    author = {Pajarinen, Joni and Peltonen, Jaakko},
    booktitle = {Proceedings of the 25th Annual Conference on Neural Information Processing Systems (NIPS)},
    language = {eng},
    month = {December},
    pages = {2636--2644},
    title = {{Periodic Finite State Controllers for Efficient POMDP and DEC-POMDP Planning}},
    year = {2011},
}

PDF (148 kB)
See books.nips.cc ...