Expectation optimization with probabilistic guarantees in POMDPs with discounted-sum objectives Conference Paper


Author(s): Chatterjee, Krishnendu; Elgyutt, Adrian; Novotný, Petr; Rouillé, Owen
Title: Expectation optimization with probabilistic guarantees in POMDPs with discounted-sum objectives
Affiliation IST Austria
Abstract: Partially-observable Markov decision processes (POMDPs) with discounted-sum payoff are a standard framework to model a wide range of problems related to decision making under uncertainty. Traditionally, the goal has been to obtain policies that optimize the expectation of the discounted-sum payoff. A key drawback of the expectation measure is that even low probability events with extreme payoff can significantly affect the expectation, and thus the obtained policies are not necessarily risk-averse. An alternate approach is to optimize the probability that the payoff is above a certain threshold, which allows obtaining risk-averse policies, but ignores optimization of the expectation. We consider the expectation optimization with probabilistic guarantee (EOPG) problem, where the goal is to optimize the expectation ensuring that the payoff is above a given threshold with at least a specified probability. We present several results on the EOPG problem, including the first algorithm to solve it.
Conference Title: IJCAI: International Joint Conference on Artificial Intelligence
Volume: 2018
Conference Dates: July 13-19, 2018
Conference Location: Stockholm, Sweden
ISBN: 978-099924112-7
Publisher: IJCAI  
Start Page: 4692
End Page: 4699
DOI: 10.24963/ijcai.2018/652
Notes: This research was supported by the Vienna Science and Technology Fund (WWTF) grant ICT15-003; Austrian Science Fund (FWF): S11407-N23(RiSE/SHiNE);and an ERC Start Grant (279307:Graph Games).
Open access: yes (repository)
IST Austria Authors
  1. Petr Novotny
    11 Novotny
Related IST Austria Work